1.       To begin at the beginning

“A thing is not necessarily true because a man dies for it.”

Oscar Wilde

For well over two decades one of the most talked about benefits of Enterprise Data Warehousing (EDW) has been that it gives us a single source of business truth.

In the past, people working in the field of Information and Communications Technologies, in common with other industrial and technological sectors, have striven to maximise what could perhaps be described as a relatively flexible ‘one method fits all’ approach to providing services and products.

In the field of Information Management the search for a harmonised, sanitised and superficially conformed view of business data has achieved its highest form of expression in Enterprise Data Warehousing.

So what led us to view EDW as being a valuable asset in the corporate toolbox?

In short, EDW met a real and present need in the marketplace and in the workplace. It achieved success precisely because it wasn’t a technology looking for a solution but a technology based solution driven fundamentally by business demand and market needs.

The business imperative was simple; businesses wanted a single accurate, reliable and trustworthy view of corporate data, a true reflection of the state of the enterprise. EDW arrived on the scene to meet that demand.

An opportunity arises, the business moves, the market reacts, and the data warehouse captures the events. With the solid foundations of a Data Warehouse, we can report the past, monitor the present and try to predict the future.

Now, the suggestion that a database can contain ‘truth’ is a bit of an contentious claim, no matter how factual the data in the database might be, or how ‘open to interpretation’ the term “truth” is.

We should take arbitrary claims of truth, which form part of a wider social phenomenon, with a pillar of salt. It’s not our fault if we are suckered by words; after all, we are the day to day consumers of spin, a product of the all-pervasive and fashion conscious media industry. Term abuse is no less prominent in my own field of Information Technology (IT), where we have a proven tendency to openly embrace and extend the fast and loose, the tendentious and illusory, the marginal and transient, of old worlds and new.

However, if we stick to the basics of this debate, what is significant about the ” single source of truth” claim is that all too often reality does not live up to the promise. In some instances, business people disagree that there are automatic benefits for a corporation that implements a Data Warehouse, and often the critics have very good reasons for taking the positions they do. However, this should not detract from the proven fact that Enterprise Data Warehousing, when done right and for the right reasons, has the potential to deliver real business benefits.

One of the most popular storylines used to highlight the self-inflicted pain of not having a “single version of the truth” when it comes to corporate data, involves a hypothetical business meeting and two possible scenarios – one terribly bad of course, and the second a thinly veiled representation of business data nirvana, if one can imagine such a thing.

In the first of the two scenarios, the nasty scenario, an executive management meeting is called, and like so many management meetings, it triggers a demand for departmental statistics, forecasts, tactical reviews and position reports; a set of requirements that, by twists and turns, evolves into everyone’s favourite nightmare.

2.      Scenario Number 1

“My sources are unreliable, but their information is fascinating”

Ashleigh Brilliant

I am at the head offices of McWidget Associates, a regular corporation in a regular corporate climate. McWidget Associates holds a regular executive meeting, in which heads of marketing, finance and production, and so on, present and discuss business performance and profitability statistics. As I mentioned, in this, the first of our scenarios, we see the redeployment of whole teams of people in their respective departments, frantically working away preparing the reports and the statistics for the executive meeting. The work is essentially long, arduous and tedious.

Whilst the traitorous clock on the wall marks internet time, a loyal band of employee’s battle with complexity, fatigue and monotony in order to produce their departments very own pixel-perfect executive reports.

As the time marches on, reports are constructed based on data painstakingly extracted from departmental computer systems, spread-sheets and personal files. This intellectually mind-numbing productive output is then checked, double-checked and checked again; the mantra is that if you must be wrong just make sure you are consistently wrong, and that your consistency has been validated.

The output of a small Amazonian rainforest is indelibly etched with data that has been massaged by many minds, cleaned for accuracy and processed by each department’s carefully developed statistical analysis programs.

Several minutes before the meeting is due to start, the finalised reports are reviewed, approved and then reprinted in colour for posterity. All of this output is the tangible result of immense effort, the absence of real asset management, a failure to adopt good business practices and a lack of foresight.

Nervous executives, armed with their data, rush off to the conference room. The scene is set, and the game begins. As is predictable, a short way into the meeting, an intense and vitriolic argument breaks out between some of the participants; accusations of cooking the books, sexing up statistics, overplaying the competition and downplaying success, get bandied about, as one report after the next is trashed, and the veracity of certain data is hotly disputed.

Now, there are various reasons why meetings end in verbal bust ups, but in this story, we will keep it simple. Anomalies between the sales and revenue data in the reports presented by each department head have caused the friction and fractiousness. No one agrees on the exact sales or revenue figures for the first quarter. It has become a meeting in which three distinct versions of the truth are discussed but never reconciled; of course, all of this is elegantly framed and presented using charts, more charts, animated graphics and other favourites.

The bottom line result of this “confusion of data” is that everything is a phenomenal mess. The department heads start getting overly snippy and overtly political, unwisely coffee is served, and a PowerPoint battle ensues, in which the CEO is left wondering “what on earth is going on”. In short, the meeting becomes an exercise in engineered confusion, worthy of Machiavelli. All because Marketing took its data from the marketing system, Finance from the finance system and Production from the production system, and so on and so forth. Therefore, the basic data, or better said, the differences in the basic data, from department to department, simply failed to pass the truth and reconciliation test.

Now, the story does not end there. The first scenario, the nasty scenario, leads, almost predictably, to a second scenario, the happy scenario, one that illustrates quite nicely that the smart organisation and its supply chain of information would sensibly and naturally leverage Enterprise Data Warehousing.

3.      Scenario Number 2

“Sameness is the mother of disgust, variety the cure”

Francesco Petrarch

In my second and shorter scenario, you see the director of the marketing department in her glass, steel, white and black minimalist office. An office that is spacious, cool and efficient. We have from the madness of incongruence to a sane and workable environment in a sane and workable world, courtesy of Knoll International, Bauhaus, B&O, Apple and Elle.

The director is discussing new and dynamic business opportunities with members of her team, who are as equally dynamic and full of ideas as the landscape and culture in which they work.

In that moment, an electronic message pops-up on the screen of a matt platinum tablet, a simple reminder that the interdepartmental meeting is due to start in fifteen minutes time. In a couple of virtual mouse clicks, we see a simple dialogue window, the director’s assistant selects a report type from a drop down menu, then a quarterly period, and then clicks on the “generate report” button. In an instant, the Business Intelligence application retrieves data from one of the data marts in the Enterprise Data Warehousing infrastructure, this then leads to very rapid in-memory calculations and formatting, and the final report directly appears on the tablet screen in spread-sheet format. The team quickly reviews the report, and adds any contextual information that is required; the report parameters are stored for use later. The director picks up her limited edition of Ive’s swish little gadget and heads for the meeting; all the key department heads have repeated this exercise.

People wander into the meeting room. Small talk ensues, pleasantries exchanged, and the mood is relaxed and professional.

The meeting develops in a perfectly acceptable manner, as we would expect; nobody disputes the veracity of the data, just the rationale and logic of some of the interpretations.

People ask questions about strategy and operational tactics; this is a forum to raise issues of concern, and immediate access to information ensured that they dealt with on a case by cases basis, effectively and efficiently.

In any case, in our second scenario the executives have access to the Enterprise Data Warehouse infrastructure directly from their laptops, tablets and assorted mobile computing devices. So even when questions of detail arise they can be answered there and then, either qualitatively, by interpretation of the facts, or quantitatively, by drilling down through the underlying data, or indeed, through the creation of interactive “what if” scenarios.

What is important in the second scenario is how it highlights the benefits of a single source of the truth. It’s all about knowledge and trust, knowing that you have the data and trusting the validity of that data that you have, across the enterprise.

The message is simple. Enterprise Data Warehousing can take you and your organisation, towards total flexibility, total information availability, and total coherence and cohesiveness. In short, the second scenario demonstrates the power of adequate, appropriate and timely information. Without this capability, an organisation is just not going to be as agile or as responsive as it might otherwise be.

4.      Review

“I never read a book before reviewing it – it prejudices a man so”

Sydney Smith

Of the two scenarios, the second is clearly preferable. Having the choir sing from the same music sheet is preferable to constant dissonance or unhelpful improvisation. But, one nagging question always remains, is it truly desirable to have the members of the choir always singing in perfect harmony?

There are a number of business people, such as accountants, administrators and generalists, who have the occasional tendency to dismiss the idea of “a single version of the truth”. Typically, their dismissal is based on unconvincing grounds – a misunderstanding of the data warehouse process, that are truly unimportant, are simply unsound – aversion to the questioning of financial accounting practices and certain data, or that are just objections that do not apply.

Constructive criticism can be helpful, but some criticisms of the benefits of Enterprise Data Warehousing are so facile as to be absurd. One example that illustrates the immaturity that sometimes erupts on the data-warehousing scene is an example of an accountant, who is supposed to have claimed that, “you can never have one report that will be everything to everybody”. This criticism of the “single source of the truth” just goes to exemplify the point, that both the exaltation of truth in Enterprise Data Warehousing and the subsequent criticism of the claim both miss the point of Enterprise Data Warehousing.

Maybe, instead of using the term “single version of the truth”, it would simplify matters if we used the term “trusted source of integrated enterprise data”. This does not add anything to or take away anything from the value propositions of the data warehouse paradigm, but it may serve to anchor sanity in a safer haven, especially in these curious times, where the usage of super-sized claims, of dubious validity, is imperceptibly going out of fashion.

Using an Enterprise Data Warehouse to impose a single clean source of integrated enterprise data, to be used for reporting, collaboration, planning, and data analytics and decision making, is very useful. Nevertheless, it is also possibly a mistake to believe that the story starts and ends there, and to assume that an Enterprise Data Warehouse constitutes only one version of the truth. For example, if an Enterprise Data Warehouse is used as the data integration hub for regulatory reporting, compliance reporting, taxation reporting, accounting reports and management reports, and so on and so forth, one may already start to see that a single version of the truth might not meet all of these requirements.

However, when critics of the “single source of the truth” catchphrase use it to leverage their criticism of the Enterprise Data Warehouse paradigm in general, they are probably being somewhat easy on the pocket in terms of their degrees of accuracy. An Enterprise Data Warehouse is the perfect hub for integrating enterprise information; and it is rarely deployed as the last repository in the reporting chain, as this is typically the role of a Data Mart. This is precisely where the arguments of certain critics of the Enterprise Data Warehouse paradigm also fall down, as a lot of financial reporting is performed using data stored in Data Marts or more frequently in Operational Systems.

5.      Wrap up

“What sets worlds in motion is the interplay of differences”

Octavio Paz

What I want to achieve with the focus of this discussion is to help bring clarity to the muddied waters of the concept of having a single source of the truth, and having done that, to explain its relationship to contemporary Enterprise Data Warehousing.

In order for an enterprise to benefit from an Enterprise Data Warehouse, in ways that allow for divergences in reason, logic, interpretation as well as data, the organisation and the members in the organisation that use the data, must have reached a certain level of maturity with respect to divergences, what those divergences represent, and what they will effect.

That is, the organisation must have a clear and good understanding of its data. Not just in order to be able to cope with the associated pitfalls, but also to be able to grasp the benefits of the value obtained from certain divergences that may be present in differing viewpoints and interpretations of data. This is all very academic, because there is no single commercial enterprise that is close to achieving this level of maturity in its use of data and information. Expressed in a different way, the organisation that has reached this level of maturity has yet to become a reality.

For the moment, the Enterprise Data Warehouse model, together with its Data Mart component, is the most versatile and cost effective method we have for providing information for a range of reporting and analysis requirements. More to the point, it works. So, if it makes more sense to describe the Enterprise Data Warehouse as “single source of integrated enterprise data” rather than a “single source of truth”, then we should do that, and with the added assurance that, if we do data warehousing the right way, and for all the right reasons, then accruable value will be intrinsic.

So, at the end of the day we must cater for two requirements; a single version of the truth, and multiple versions of the truth. Technically this is not such a big issue, but conceptually it is likely to give quite a few technicians a headache, so, probably the best thing to do is to give the responsibility for this conundrum to someone who can comfortable manage and reconcile seemingly conflicting and disparate dialectics, such as someone with a solid grounding in classical philosophy.

At the end of the day, the occasional criticism that the Enterprise Data Warehouse model doesn’t provide what isn’t actually needed is pointless, sterile and quite irrelevant. Rather like complaining about not being able to make a decent cup of tea with a Porsche.

File under: Good Strat, Good Strategy, Martyn Richard Jones, Martyn Jones, Cambriano Energy, Iniciativa Consulting, Iniciativa para Data Warehouse, Tiki Taka Pro