It’s 9:45 on a cold and wintry Monday morning; crowds of people are huddled around the water fountains as gossip spreads throughout the building like wildfire. There has been a multiple data pile up in the Widget and Wodget data centre, there are a few casualties but no reported fatalities. No one knows what quite is going on, as shadowy rumour hints at flying pigs, rebellious ducks and rolling heads.

It was to be a big day for Widget and Wodget, “your neighbourhood’s favourite Prime Brokers”, it was billed as the Senior Mega Monday to end all Mega Mondays. A mega coup for the hallowed financial institution of W&W; today would be the day that would herald in the culmination of two major IPOs and one multi-billion global M&A deal signed off, as well as the launch of a raft of CDOs.

However, all was not well in the hallowed tech halls. Due to a series of systematic failures that struck the Data Warehousing architecture and its infrastructure, the world of IPOs and M&As rapidly turned from a superficially well automated and fuzzy dream to one of lurid, cold and angst ridden realism, as more and more failures and fiascos announce themselves, like so many gophers popping up on a lush series of putting greens.

Data quality issues alone have meant that Widget and Wodget have grossly misrepresented the underlying risk in their CDO offerings; have misstated the initial valuations of the IPOs; and, have failed to reconcile data provided by the involved parties in an M&A deal. In addition, the infrastructure will not allow rapid recovery and recalibration, nor will it allow for bad data to be unwound from the data warehouse, nor will it allow for a clean reset and restart. In sum, it’s a complete and utter disaster.

There are many things that IT departments do in the name of expediency and common-sense, as they shy away doing the right thing right, because – I wish I had a dollar for every time I have heard this said – it is just too “academic”. So they do something else, ignoring the virtues of everything from established and proven methods and architectures, to old axioms such as “a stitch in time, saves nine”.

Worst still is the attitude that is best related by this simple anecdote. In the early eighties, the powers that be in the UK made it compulsory to wear seat belts in the car whilst driving. Nothing new, I was used to doing that anyway. However, at this time I was travelling quite a lot to continental Europe (France, Spain, Greece, Portugal, etcetera) and after landing at my destination I would usually grab a taxi and we’d head off to the city.

Invariably I would use the “clunk click”[1] seatbelt procedure before the start of every car journey, and invariably the taxi driver would insist that it wasn’t necessary, that it was enough to pretend to be wearing a seatbelt, you know, just in case the police checked (but only then, outside of the city). Now, the taxi drivers in some of these places used to be pretty macho, and for them wearing a seat-belt wearing seemed to be a sure sign that a guy might be gay. To say nothing of what might be uttered if one refused the generous offer of a full strength black-tobacco cigarette, you know, to relax during the crazy testosterone[2] boosted journey to the centre of town.

This is exactly what you get in IT, although the scenarios and inferences might be different, the inferences are still there. You want testing? You want data quality checks? You want requirements? What’s the matter with you, chicken?

However, what your great grandparents might have taken for granted does not come easy to IT organisations. They can’t distinguish between playing smart, lean and agile, and being artless bodgers[3] in charge of large budgets and mission critical systems, so they guess. “Oh, we don’t do data quality, the data has always served us well”, “We don’t need to do testing. What? You want us to be here for the next twenty years testing this crap?”, “Reference architectures! What reference architectures? I didn’t get to where I am today by using best principles, best practices and reference architectures…”, “It’s all very well for Inmon and Kimball to say that, but they don’t have to work in this company…” and so on and so forth.

[1] http://en.wikipedia.org/wiki/Clunk_Click_Every_Trip Part of the slogan of a series of British public information films sponsored by the Royal Society for the Prevention of Accidents.

[2] C19H28O2

[3] Bodger; someone who makes or repairs something badly.

Many thanks for reading.

---

File under: Good Strat, Good Strategy, Martyn Richard Jones, Martyn Jones, Cambriano Energy, Iniciativa Consulting, Iniciativa para Data Warehouse, Tiki Taka Pro