The best way to avoid creating data silos is to develop a plethora of data silos?

Narrator: Data Warehousing is an excellent integrator; it is the cohesive melting pot of data and the best-known way of providing multiple valid versions of the truth depending on who is looking at the data, when they are doing so, and from what perspective.

Dud: Pete, do data warehouses create siloed systems of data?

Pete: Why do you ask, Dudley?

Dud: That’s what the data mesh folk seem to be claiming.

Pete: I think you should be able to see through the flimsy logical gun carriage this nonsense is mounted on.

Dud: Enlighten me, Pete.

Pete: Before getting involved in the data mesh cult, many code jockeys created applications that siloed data. When I have pointed this out to them or asked them about it, they always tell me they have no mandate to produce quality applications, well-designed and engineered data models, and fragmented data sources. It’s pretty shocking to hear such things from so-called professionals, Dud.

Dud: Data mesh created the conceptual idea of data that is not siloed. But using what are, for all intents and purposes, data silos?

Pete: Careful there, Dud. You are treading on dangerous ground. There is a lot of money tied up in these technology fads.

Dud: I have to laugh at the claim that “I didn’t do a quality job because that wasn’t a requirement”.

Pete: Absolutely right, Dud. And I have learned from a lot of experience that doing a quality job is often quicker than doing a crap job, especially in the long run when functionality and additional integration are added to an application.

Dud: But so many people in IT simply don’t get the good sense of that approach, and they will often bitch about being constrained by sound engineering practice as if it were a curse or something worse.

Pete: Forgive them, Dud. For they do not know what they are cocking up.

Dud: So, Pete, how do we correct this lamentable state of affairs?

Pete: The short answer is to stop doing something wrong. Also, don’t trust people and providers who have proven to you, sometimes, many times, that they cannot be trusted.

Dud: How do you think we can get that message to those who need to notice, understand, and take it on board.

Pete: It’s a hard call, Dud. We live in a world where people lie and mislead, and vendors will buy the favours of corporate buyers or purchase influencers. Also, many big-ticket decisions are made based on pure arrogance and ignorance. What’s even worse is that we see managers make decisions that should not be part of their competence. They lack the breadth and depth of knowledge and experience to make calls related to sophisticated architectures and technologies.

Dud: Good points, Pete. I’ve noticed that the vice of some project managers is to get too involved in solutions architecture development.

Pete: Tell me about it, Dud. They are a bloody pain in the arse if you ask me.

Dud: Can we still safely use data warehousing as an appropriate reference architecture?

Pete: Most certainly, Dud.