Data Warehousing means monolithic databases?

Martyn Jones, Brooklyn, 27th September 2024

Narrator: Another false meme doing the rounds is that Data Warehousing necessarily means monolithic databases. This is not what data warehouses have been for many businesses.

Martyn: Part of the problem with this claim is that even the term monolithic is being corrupted to mean “any architecture that I don’t like”.

Having worked with massive clusters of computing power, with tons of nodes and disks and memory and ultra-fast mesh communications backplanes, I instinctively know this monolithic labelling is dubious. In addition, we could isolate data at various levels of abstraction. Indeed, we can organise singular and interdependent subject/data domain areas using database technology that has been available for decades. So distributed data storage and computing is nothing new and certainly nothing new to data warehousing.

So, the notion that “data mesh is an emerging architecture that establishes an alternative, decentralised pattern to the data warehouse” is nonsense born of ignorance and immaturity.

Narrator: So, what do Pete, Dud and Mel think of all of this?

Dud: Hello, Mel. How are you, me old mucker?

Narrator: Mel is from Brooklyn.

Mel: Oh, boy. Oh, boy! Not too bad, not too bad. I arrived in London a few hours ago, and I’m still lagged by the jet. And how about you, Dudley?

Dud: Can’t complain, Mel. Fancy a pint? Oh, look, here comes Pete.

Pete: Hello, you two. What’s going on then, guys?

Mel: I had a long, deep and engaging conversation on the flight from New York. Yes, I, and the guy sitting next to me, had a few drinks and Martinis and shot the breeze. And very soon, the topics turned to data and information. And we were getting along like a house on fire. Then, out of the blue, apropos of nothing, he says, “Data warehousing means monolithic databases”. Just like that!

Pete: The cheeky sod. What did you say to him? Oh, and get the drinks in, Dud.

Mel: Let’s wait until Pete gets back, and I’ll tell both of you guys simultaneously.

Pete: So what else have you been up to, Mel? Are there any new projects in the pipeline?

Mel: Nah, I’m too old for that now, Pete. Well, I still have my projects but nothing in the cinematic sense of the term, or anything even close.

Narrator: They chat about nothing in particular, as an easygoing mood of satisfaction washes over them.

Pete: Here he is. Thanks Dud. You should have said, I would have helped you with the drinks.

Narrator: They sit down and admire their surroundings and their affinity.

Dud: So, Mel, how did your monolithic database conversation go?

Mel: Oh, boy, that was such a strange thing. Well, the guy’s name was Joe, and this is roughly how our conversation went:

Joe-past: Mel-past, Data Warehousing means monolithic databases

Mel-past: If you are so of a mind, Joe-past, but I beg to disagree

Joe-past: Okay, Mel, change my mind.

Mel-past: I’ve got this friend working on a top-secret data warehousing project in China. His name is Martyn. Oh, damn, sorry, my bad. Scrub that, forget it. Say his name is Bernie and he’s working on this project in London.

Bernie designed this distributed database that spanned five massive data centres and an aggregation hub.

He told me once that “if there is anything we have learned from building massive data warehouses, it’s to avoid monolithic-anything like the plague”.

In this instance, the conceptual database was anything but monolithic, totally distributed and partitioned. Data warehousing doesn’t mean monolithic databases and models unless the architecture of the data warehousing storage is all wrong.

Joe-past: If you don’t mind me saying so, but your friend got it wrong. He just created a faux distributed database.

Mel-past: In what way did he get it wrong? In my opinion, if conceptually you have a conceptual whole. A unified view of the data, then the underlying distribution of data and processing, is of no interest to the user, but the performance it provides benefits everyone.

Joe-post: If I need to explain it to you, then it’s not worth it to explain it to you!

Mel-past: Sir, without meaning to offend, you seem ill-equipped to criticise the work of a leading global expert in this matter.

Joe-past: What do you mean ill-equipped? I have the hat, gloves, scarf and everything.

Me? Ill-equipped? Pah!

Mel: And that was it.

Pete: Well, you certainly put him in his place.

Dud: What a div.

Mel: Well, it helped pass the time. Transatlantic flights can be terribly dull.

Pete: G&T Mel?

Mel: I don’t mind if I do. Heavy on the Seagrams.

Pete: Any takeaways from this experience, Mel?

Mel: To reiterate, I would say this. If you want a good life, never run for a bus, never eat fried food, and never assume that others are as lousy at being solutions architects as you. Apart from that, practising the violin every day would be best.

Dud: Sound advice, Mel.

Pete: I think we must get you back in the conversation, Mel.

Mel: Cheers, guys! Long Island Teas for all?