Martyn Richard Jones

Dusseldorf, August 2006

Data Warehousing provides possibly one of the best opportunities for IT organizations to deliver a valuable business solution in order to address a set of business needs; requirements that go well beyond the area of day to day operational support, and traditional applications (web enabled or not), and when Data Warehousing is done the right way, and for the right reasons, its payback to all of its stakeholders can be positively significant.

In essence, a data warehouse project is typified in business and technology terms as follows:

• Provides a reliable source of consistent information to support strategic, tactical and operational decision making.
• It is supremely business focused.
• It is a highly “marketing oriented” initiative in terms of information.
• The development and delivery mechanism is technology based.
• It is developed and delivered in manageable iterations; iterations that are small enough to be achieved in a short space of time, and iterations that are large enough to be of significance to the business.
• It is repeatable. It can be used repeatedly, and in any corporate setting, to deliver: adequate, appropriate and timely information to the point of need, when it is needed, and to the quality required.
• It is cost effective.
• Its underlying design and architecture provides the necessary agility to cope satisfactorily with any changes required by the business, and to cater for almost any new requirements from business users.
• It is in essence, a market model applied to information demand and delivery.

The above is a representative list of good things that no corporate executive could possibly argue substantially against; however, things don’t always go according to expectations.

The down side to Data Warehousing is that more than forty percent of DW projects do not make the grade, in some way or another. This is, in my opinion, is an unacceptable level of failure, especially considering that DW is something that promises to deliver so much business value, costs so much when it fails, and frequently creates even more bad feeling between IT and the Business. So, what are the problems, and how can they be addressed?

There are a number of reasons why quite a few DW projects don’t live up to expectations; the following list illustrates some examples of why failure occurs (I will return to this list later in the book and provide additional details both on the reasons for failure and how to avoid or mitigate those failures). So, here are some of the reasons for failure:

• A tendency to trust in intuition rather than in knowledge and experience.
• Starting a data warehouse without having a qualified and committed purchaser (business sponsor).
• Carrying out requirements gathering exercises that focus solely on data and not on business process.
• Setting unrealistic expectations and not meeting promised delivery objectives.
• Providing users just with reports, instead of giving the business users the means to analyze data and prepare reports for themselves.
• Providing either more, or less, than what has been asked for, and what has been promised.
• Using SDLC (System Development Life Cycle) Methodology for DW.
• Not using an iterative approach for the DW project.
• Misunderstandings regarding what technologies to use, the cost/benefit ratios, and how to use them.
• Delivering unusable, unmanageable or unintelligible data models.
• Providing data that is not reliable enough to support decision making, or to produce reliable and accurate reports for regulatory reporting.
• Not providing adequate business requirements documents and functional specifications.
• Thinking that the DW project is over when the data is being delivered to the business users.
• Not training people in relevant aspects of Data Warehousing and Business Intelligence.
• Allowing for too much ambiguity in terms of project roles and responsibilities.
• Allowing Data Warehouse costs to get out of hand.
• Tolerating behavior that turns a Data Warehouse project into a political war zone, where organizational rivalries and battles are played out, to the detriment of the business.
• Forgetting that the DW is also about Quality.

The bottom line is this: Each and every one of these problems can be avoided or mitigated if adequate DW testing is in place.

Many thanks for reading.

 

If you want to find out more about testing for Data Warehousing, BI, Data Integration, Data Migration and Reporting, then give me a call or drop me a line: martyn.jones@cambriano.es

 

Just a few points before closing.

Firstly, please consider joining The Big Data Contrarians, here on LinkedIn: https://www.linkedin.com/groups/8338976

Secondly, keep in touch. My strategy blog is here http://www.goodstrat.com and I can be followed on Twitter at @GoodStratTweet. Please also connect on LinkedIn if you wish. If you have any follow-up questions then leave a comment or send me an email on martyn.jones@cambriano.es

Thirdly, you may be interested in other articles I have written, such as:

You may also be interested in some other articles I have written on the subject of Data Warehousing.

Data Warehousing explained to Big Data friends – https://goodstrat.com/2015/07/20/data-warehousing-explained-to-big-data-friends/

Stuff a great data architect should know – https://goodstrat.com/2015/08/16/stuff-a-great-data-architect-should-know-how-to-be-a-professional-expert/

Big Data is not Data Warehousing – https://goodstrat.com/2015/03/06/consider-this-big-data-is-not-data-warehousing/

What can data warehousing do for us now – http://www.computerworld.com/article/3006473/big-data/what-can-data-warehousing-do-for-us-now.html

Looking for your most valuable data? Follow the money – http://www.computerworld.com/article/2982352/big-data/looking-for-your-most-valuable-data-follow-the-money.html