Young Einstein was awaiting the birth of his sister. His mother and father had been telling him for months to look forward to having somebody to play with. When Einstein eventually got to see his sister, he exclaimed disappointedly “But, it doesn’t have wheels!”

That happens a lot in IT. Young Einstein had a very good excuse, we don’t.

Do you want people to be thrilled and delighted by data warehousing and business intelligence? Then to start with, make sure you can check all of these:

• You’ve got business requirements from real business users? Check!

• You’ve scoped the requirements? Check!

• Including the ubiquitous dimensional model or reporting model? Check!

• It’s small enough to be doable in 3 months or less? Check!

• And, big enough to be significant for the commissioning business users? Check!

• You’ve got management and budgetary approval to proceed? Check!

• A Data Warehouse (Entity/Relationship or Conformed Dimensions and Facts) will be an integral part of your architecture? Check!

• The four fundamental design principles of the Data Warehouse will be strictly adhered to? Check!

• The data in the Data Warehouse will be subject oriented? Check!

• The data in the Data Warehouse will be integrated? Check!

• The data in the Data Warehouse will be time-variant? Check!

• The data in the Data Warehouse will be non-volatile? Check!

• Your business users will be working with you, every step of the way? Check!

Pretty straightforward, right? There is no rocket surgery involved, and getting the above right will cover up to 90% of the major fundamentals of getting Data Warehouse right, at least, at the beginning – even if you are back at the beginning for the umpteenth time.

Later we can address other aspects, such data quality, testing and performance, but right now, I want to focus on the core essentials, because when Data Warehousing fails, it is well before issues of data quality, testing and performance become problematic.

So, why do Data Warehouse projects fail, and why so many of those botches can be put down to failures to check one or more of the twelve points made above?

1. The first exhibit can be summed up as: I don’t need Inmon or Kimball to inform me on how to do this stuff! Incredible but true, there are actually people out there who will claim, without blushing that “I don’t need no West Coast stuffed shirt liberal academics to tell me how to do Data Warehousing. I know data, I know warehousing and I know my business, and no jumped up punk is going to tell me otherwise”.
2. Next up comes the classic demotivating cry of “our business users are too stupid to know what they want, and too stupid to do things for themselves”. When it comes to being creative and effective with a narrow set of tools and technology, business users can leave pure technology folk in the dust. Business users are smart, are savvy and will use anything that is available to do their job. Treating your business users as idiots is a sure fire way to fail.
3. Even worse than treating the business users as innocent fools, is the approach of not communicating with business users at all, the term I use for this is Organisational Autism. It’s not an insult, it’s a social and behavioural organisational fact, especially in IT departments and IT technology and service vendors.
4. When ‘no’ becomes the hardest word. Stopping scope creep before it starts is easy, unless of course you can’t say no. Emphasising the need for proper scope from the outset, even before one gets into the details, and then reiterating it and applying it at every stage, will avoid a lot of false expectations and grief down the line. That said, many Data Warehouse projects fail because people who should know better don’t have the social skills to be able to effectively and persuasively say no.
5. We built it, and they didn’t come – the ungrateful schmucks! You didn’t get real requirements, you didn’t scope the requirements, and you didn’t even try and imagine what might be useful. You went to the business users, like some techy Santa Claus with a shiny new reporting system that you insist on calling Data Warehousing with Business Intelligence, and they told you to shove it. Well, fair’s fair, that’s a reasonable response to costly and unreasonable behaviour – and a clear message that hubris is ineffective and passé.
6. “But, you didn’t tell us you wanted to do analysis of data over time” – If I had a dollar for every time some Data Warehouse architect or Technical Project Manager has come out with that howler, then I would be well up in my dollar account. Data Warehouse projects have failed because people forgot to incorporate true time variance into the data. How come Data Warehouse professionals forget such a fundamental factor? I know, it beggars belief. So far I can only think it’s because DW and BI are plagued with a surfeit of charlatans, populists and opportunists.
7. The dimensional model that is not. I have seen so called dimensional models that have been so complex and unintuitive that not even the designer has been able to fully understand what they have done, never mind it being simple and intuitive for the business user to grasp. What happens next? “Jaysus! How do we get the data into that?”, “Blimey! How do we produce our month-end summary from this?” and “Are you having a laugh, sunshine?” Big Data Warehouse programmes have actually floundered because some bright spark has fucked up the data model, a data model that must be – by design – simple enough that the business can understand what it’s all about and solid enough to provide what is required.
8. The data warehouse that isn’t or “Look Ma! I’m flying”. As the father of Data Warehousing states, a data warehouse is subject-oriented, integrated, time-variant and non-volatile. A lot of organisations will take this in, and shelve it. “Oh, we didn’t have time for all if the nonsense”, and they go and dump a whole pile of operational data on a server somewhere, and declare victory. What’s wrong with that? Nothing. It’s not Data Warehousing, it should not be called Data Warehousing, and people should not pretend otherwise, even if it is giving people what they want. Fact is, most of the time this approach doesn’t work either, or if it does, only for a month or three.

Bottom line, a majority of Data Warehouse and Business Intelligence projects are fucked up, and they are fucked up because people fuck them up.

In most cases the fuck ups are avoidable. Even just following the simple, accessible and sensible guiding fundamentals of DW father Bill Inmon and dimensional modelling guru Ralph Kimball, you can’t go far wrong.

Mozart was asked by a young bloke on how to compose a symphony. Mozart suggested that he first worked on ballads. “But, but” explained the young man, “You were writing symphonies when you were ten!”, “Ah yes” replied Mozart “But, I didn’t have to ask how”.

So, final words. It’s possible to get Data Warehousing and Business Intelligence right from the get go. So, first get it right, and then again, and again, and again… after that, you might try something else, but in a very controlled environment that limits the exposure and impact of your experiments, nevertheless, for most or all iterations, just keep on doing the proven, the reliable and the effective… and remember, it’s not brain science.

Many thanks for reading.

---

File under: Good Strat, Good Strategy, Martyn Richard Jones, Martyn Jones, Cambriano Energy, Iniciativa Consulting, Iniciativa para Data Warehouse, Tiki Taka Pro