Martyn Rhisiart Jones
A quiet disquiet has settled over the once-confident corridors of enterprise data. What was, not so long ago, regarded as a rigorous and rather specialised craft, data warehousing and business intelligence, now frequently presents itself in a more casual, even improvisational guise. A growing number of senior executives, technology directors and indeed practitioners themselves confess to a mounting discomfort with the quality, and at times the sheer quantity, of self-proclaimed experts who populate the field.
The symptoms are by now familiar, if seldom discussed in polite company. Certification mills churn out credentials with the efficiency of a high-street coffee chain; LinkedIn profiles brim with titles that stretch the definition of “architect” or “guru” to breaking point; and project war stories abound of dashboards that dazzle on first viewing yet crumble under serious scrutiny, of warehouses that resemble attics more than precision-engineered repositories, and of transformation programmes that quietly mutate into expensive exercises in report-proliferation.
This is not merely a matter of technical standards slipping. It speaks to something deeper: a perceived erosion of professional seriousness and intellectual integrity in an area that once prided itself on both. The discipline that gave the world Kimball’s dimensional modelling, Inmon’s corporate information factory and the patient art of slowly building conformed layers of trusted data now often feels like a race to the bottom, where speed-to-demo trumps durability, and the loudest voice in the Zoom room wins the contract.
Nor is the unease confined to data circles alone. Parallel conversations can be heard across much of the broader Information and Communication Technology landscape—among software engineers weary of copy-paste architecture, among cybersecurity professionals watching threat models give way to checkbox compliance, among architects who remember when enterprise systems were designed rather than merely assembled. Yet for the moment, those wider currents lie beyond the scope of this reflection.
What is striking about the present moment in data warehousing and BI is the contrast with its own recent history. Only a decade or so ago, the field still carried the aura of a serious profession: long apprenticeships in large organisations, deep familiarity with the trade-offs between performance and flexibility, a certain pride in building systems that would last ten years rather than ten sprints. Today, the democratisation of tools (no bad thing in itself) has been accompanied by a kind of commoditisation of expertise. The barrier to entry has fallen so far that almost anyone with a working knowledge of SQL, a cloud account and a weekend tutorial can style themselves a “data engineer” or “analytics leader”.The consequences are not abstract. Organisations find themselves wrestling with proliferating data silos disguised as “modern data stacks”, with governance that exists chiefly in slide decks, with BI teams that produce beautiful visuals while quietly losing the thread of where the numbers actually come from. The promise of data-driven decision-making, so compelling on the conference circuit, too often turns out in practice to be data-chaos-driven decision-making.
One is tempted to ask whether the pendulum has swung too far. The answer, probably, is yes—and yet the forces that drove it there are not easily reversed. The hunger for quick results, the pressure on margins, the relentless marketing of low-code and no-code solutions, the rise of the “citizen data scientist” (a phrase that once sounded utopian and now frequently feels euphemistic): all have played their part.
Still, the discomfort is real and growing. Those who have spent decades in the quieter, more methodical corners of the discipline sense that something valuable is being diluted. The question now is whether the field can rediscover a balance: one that preserves the accessibility and innovation of recent years while quietly reasserting the importance of rigour, accountability and, yes, a certain modesty about what we really know.
For the time being, many practitioners and clients alike appear to be holding their breath, waiting to see whether the current exuberance will correct itself, or whether the profession will need a more deliberate act of reclamation. Either way, the conversation, until recently conducted in murmurs, is beginning to find its voice.
The issues that I think need addressing are:
1. Why do some organisations take on Data Warehouse or Business Intelligence development contracts when they are clearly ill-equipped to take on such endeavours and have little or no satisfactory experience in the field?
2. Why do some organisations make alarmingly huge and false claims about the depth and breadth of their experience and knowledge in the field of data warehousing?
3. Why do some organisations put forward “consultants” as their data warehouse experts, when these self-same ‘consultants’ barely have a clue about what data warehousing is about?
4. Why do some people consent to being touted around as professional consultants in the field of data warehousing, when clearly they are not?
5. Why do some people consider themselves professional consultants in the field of data warehousing, when clearly they are not?
6. The contribution to a fundamental lack of professionalism (including integrity and honesty) to the failures of data warehouse and business intelligence projects.
7. The contribution of professional malpractice to the business world’s view of data warehousing and business intelligence, and the companies and individuals involved in these “fields”.
The problems with all of the above are not that these organisations and individuals are doing what they do; the problem is the impact they are having on highly professional individuals and organisations:
1. Companies stung by data warehousing scams tend to blame data warehousing, rather than the cowboys they hired to provide them with a data warehousing process.
2. The bad reputation created by lies, shoddy work and lack of professionalism can hit everyone, not just the guilty.
3. Some large organisations can buy or bluff their way out of failure, disgrace and malpractice; small professional organisations are more prone to systemic damning of their chosen line of business.
Proposal:
1. The industry should have a professional body for accrediting data warehouse and business intelligence service providers.
2. The industry organisation should be empowered to investigate potential malpractice in the profession and to apply sanctions as appropriate.
3. This industry organisation should have the teeth to name and shame cowboys in the profession – so that they either shape up, or ship out.
What do you think about the perception of a growing lack of professionalism in data warehousing and business intelligence projects and practices?
Many thanks for reading.
File under: Good Strat, Good Strategy, Martyn Richard Jones, Martyn Jones, Cambriano Energy, Iniciativa Consulting, Iniciativa para Data Warehouse, Tiki Taka Pro
Discover more from GOOD STRATEGY
Subscribe to get the latest posts sent to your email.
GOOD STRATEGY 