, , ,

Simple question, why do I write this blog? To enlighten? To inform? To help? To save mother earth?

Well, no, of course not.

I write about Good Strategy to get people’s attention, I want people to hire me to do paid work, and preferably in strategy, leadership and coaching, and then for them to pay me.

The purpose is to sell my services and to accrue commensurate compensation.

Kicking out the bad and replacing it with the good is my stock in trade, and one of my key focuses is on Data Warehousing, where more than 85% of these projects are partially or completely dysfunctional. I have chosen Data Warehousing to be one of my key focuses, because in spite of all the bullshit to the contrary, Data Warehousing is about business process and business demand, far more than the technology enablers and the latest management consulting fads and dogmas.

So, you may ask, “What’s all this about then, Marty?” But, let’s be honest. Wherever you are in the world, if your Data Warehouse or Business Intelligence project or programme isn’t in a mess, hasn’t failed, isn’t failing or is on a long death-march to nowhere,  then either your organisation is one of the lucky few, or someone in your organisation is not being entirely candid.

So, the trillion dollar question today is: Why are so many Data Warehouse projects failing?

Let’s compare the market…

Many data warehouse projects fail, even if they are declared a resounding success – more on that later – and much has been written about causes of failures. But what is usually omitted from such insight is the elephant in the Portaloo – people.

Yes, it is people who are the biggest contributors to the screwing up of so many Data Warehouse projects, and in spite of wishful thinking, and therefore unscrewing a screwed Data Warehouse project isn’t as simple as unscrewing a light bulb, because there are political, social and psychological issues at play.

I know that there is a plethora of lists, all usually ignored, that outline reasons why Data Warehouses fail, and there is a degree of merit to what they focus on, but there is also a down side. By framing failure as being centred on things such as technology, data quality and techniques, they miss the point, because it deflects attention away from the really big problem; the people problem.

I have my own guiding principles when it comes to Data Warehousing, and the key principles for me are:

  1. Don’t do anything unless it is driven by business demand. It’s okay to go and sell ideas to create a demand, backed by business justification, but we are not in the business to be Santa Claus, but to help drive business value.
  2. Do what is required. Never extend the scope of what needs to be done, but tailor the scope to something that is small enough to deliver and large enough to be significant.
  3. Don’t have too many irons in the fire. Don’t try and address a plethora of requirements at the same time.
  4. Iterate the hell out of everything. Deliver early, deliver often, and get the users involved in every step of the way, so that when the big day comes, the big day is merely a formality. An excuse for pizza and beer, not a day for hands-on firefighting, arm-waving and bluster.

When I tell business executives this, they love it. Why? Because it makes good sense, and it is something that rational, reasonable and coherent people can relate to. When I explain the same things to IT executives and managers, they generally loathe it, and in most cases they respond is as if I had peed on their carpets.

Unfortunately IT is full of under-socialised knee-jerk reactionaries, who will keenly jump on any passing fad or fashion, because “it needs to be done”, and there is always a plethora of technology and service vendors to ensure that the habit is nurtured, lauded and well supplied. But the problem is not the suppliers of technological crack, but those who go out in search of it.

Sure, technology and service suppliers will bend the facts, play with smoke and mirrors and will tell IT people what they want to hear, but they will never lie to their captive audience as much as the captive audience will lie to itself.

I once heard Larry Ellison talk about some of the crazy dot com schemes that were going down in Silicon Valley, like the ‘get rich fast’ scheme of selling dog food over the internet. Smart guys know when an idea is nuts, but that doesn’t stop them from selling the crazy guys a stack of high-value software licenses.

Another problem is that Data Warehousing is conceptually a walk in the park. Data Warehousing at the highest level is easy to explain, the problem is there. Most people who think they know about Data Warehousing do not bother to go past the trivial conceptual level. At a trivial level Data Warehousing looks easy and almost intuitive, you can almost second guess what comes next. But this is analogous to watching Rafa Nadal and Roger Federer slogging it out in a tennis final, and thinking “well, I can do that as well”.

When I bluntly tell people that they cannot hope to be the Nadal or Federer of Data Warehousing – even if to date they have spent millions on trying to get there – they hate it, and the response is typically along the lines of an immature mixture of bluff, bluster and rage.

However, tantrums aside, you still cannot bullshit simplicity – telling things straight – and this is why so many people who are ‘fannying about’ in the areas of Data Warehousing and Business Intelligence – like 80% or more – react to simplicity like Superman reacts to kryptonite.

The evidence is out there, in your organisation, in your department. I have seen many of the worst avoidable nightmares in IT and business history, and in these highly volatile and competitive times there is no margin for snow-jobs, obfuscation or procrastination.

Technology and IT service vendors will sell you all the high-priced crack you can consume, and then some, but they are not the ones to blame. The problem is in the organisations that do Data Warehousing for all the wrong reasons, the problem is in the people who get organisations to do Data Warehousing for all the wrong reasons. Which is precisely why I called this blog post “Be more idiot – business as usual in Data Warehousing”.

So, now if you want to unscrew your screwed Data Warehouse or Business Intelligence programme or project, and are not afraid of hearing awkward questions and some home truths, then just give me a bell.

Be more idiot – business as usual in Data Warehousing – by Martyn Richard Jones