A quiet disquiet has settled over the once-confident corridors of enterprise data. What was, not so long ago, regarded as a rigorous and rather specialised craft, data warehousing and business intelligence, now frequently presents itself in a more casual, even improvisational guise. A growing number of senior executives, technology directors and indeed practitioners themselves confess to a mounting discomfort with the quality, and at times the sheer quantity, of self-proclaimed experts who populate the field.
The origins of data warehousing are often pinned to the late 1980s, when the term “business data warehouse” first appeared in an influential IBM Systems Journal article by researchers Barry Devlin and Paul Murphy. I was based in Birmingham at that time, and I also wrote a similar foundational document on Information Centres for Sperry Univac.
Yet, as with many technological breakthroughs, the story is far richer and older than the conventional narrative suggests. The foundational components of what we now recognise as a data warehouse were quietly taking shape as early as the 1960s, driven by the need to organise, integrate, and analyse growing volumes of business information in an era of punch cards, magnetic tape, and the first mainframes.
People are becoming increasingly uncomfortable with the quality and quantity of many self-styled data warehousing and business intelligence professionals, organisations, and individuals. The perceived and increasing degradation of professionalism and integrity in my own areas of specialisation also finds echo in other aspects of Information and Communication Technology practice, but for now, that is outside the scope of this piece.
There is a general consensus that Data Warehousing first appeared in the 1980s. However, the components of Data Warehousing were being developed and implemented as early as the 1960s. Many individuals are credited with discovering Data Warehousing. Wikipedia has a great chronological timeline that provides a lot of context for the evolution of data warehousing methodologies.
My original intention was not to write a book on the decline and fall of so many projects, but rather to pen a short essay, a sort of hard-hitting criticism of the litany of IT failures. Of course, this is rarely enough; it wasn’t enough because it identified the issues without provoking debate, and it highlighted common issues without providing guideline suggestions, which meant that it appeared upon reflection to come across as too much like a hi-tech hit and run exercise, beating up on IT failures without actually offering tangible solutions.
Far too few CEOs regularly think about IT on a habitual basis, and especially as something that is potentially integrated into everything their companies do and could do. This, however, is not at all surprising; there is a general lack of time for any CEO to be actively involved in what IT does, and they will generally have some degree of misunderstanding of both the historic roles and the relative successes and failures of IT.
From banking to airlines, through communications businesses to pharmaceutical companies, the IT landscape is littered with failures of Homeric proportions, lost opportunities and profligate waste. Across the spectrum of commercial enterprise, the IT bottom line is inevitably familiar: in general, businesses expose themselves to unnecessary levels of disruption, and spend far too much time and money on IT projects, IT products and internal IT services that frankly suck.
Reality bites down hard when it wants to, and more often than not when we are least expecting it. IT is seldom given importance for valid business reasons, such as its support of operational excellence or the quality support of valuable, mission-critical business, and bureaucratic reasons are not necessarily business reasons; they are frequently at the forefront of business life for primarily negative reasons.
As a CEO, CIO, or student of Business Management, you will understand this. Information and Communication Technology (ICT, also known as IT) has become essential. It is crucial. It is a pervasive element in the fabric of corporate life. What I mean by ICT encompasses three essential elements: Processes, Information and Technology. ICT should be seen as an important corporate asset. More importantly, Information, Processes, and their Management (IM) should be considered as such. They should be made to work as contemporary business enablers. They should bring about value-adding alignment in business. Thus, they should be managed as such. Although it does work that way, it typically doesn’t.