If you enjoy this piece or find it useful then please consider joining The Big Data Contrarians: https://www.linkedin.com/grp/home?gid=8338976

Many thanks, Martyn.

If Princess Diana had been alive during the formative years of the Big Data revolution there would have been a plethora of influential Big Data bullshit babblers issuing their gushingly awful pieces in places like Forbes, the WSJ and professional blogging forums about the Big Data humanitarian causes closest to the heart of the peoples’ princess. And if tragedy had repeated itself, and had been reported not as paparazzi driven schmaltz or morbid vulgarity, but as something even more rancid and farcical, we would now have a Princess Diana Memorial Data Lake in some regal park in London or Milton Keynes –powered by Hadoop. Because, as the bullshit babblers would have it, “that is what she would have wanted”.

But, is this entirely fair? Should we view the outpourings of the biggest Big Data bullshit babblers on the entire internet as the inevitable result of free will, or is Big Data a message from God, in the same way that hard drugs are a signal to certain rock stars that they have too much available cash?

Which brings me to another issue. In a recent interview, I was given a list of data related terms, and was asked which one I preferred. Big Data, Smart Data, Small Data… you know what I mean. Anyway, I went off on a tangent about domestic pets and anthropomorphism. Okay, so it was logical entrapment, but I wanted to make a point. “Don´t you think that ascribing human behavior and thought to pets is a bit weird?” I asked. “No, came back the reply”. It wasn´t the answer I wanted, because the answer I wanted was “Yes, it certainly is” not a “No, that’s what my mum thinks as well”. I wanted to say see, people who ascribe human characteristics to dogs strike us as being a bit fanciful, but people who do the same for data? How can a bunch of recognizable symbols embody smartness? I mean, data by itself, of itself, is dumber than a rock.

So why do we pretend that the information, knowledge and the smarts are in the data and that data itself, without the need for any intervention (other than Hadoop, Sparke or Hive, etc.), is capable of revealing this smartness?

And the only thing I can think is that we are so desperate to sell useless crap that no one needs or wants, that we are even capable of saying the most dopiest of things in order to do so.

Anyway, I was at a Big Data conference recently, and every presenter selling a tool made exactly the same type of pitch. The amazing ways that their tools could establish correlations. Some of the examples of the correlations were so contrived, so obviously the creation of PR than the outcome of hands-off automated analysis, that it became seriously embarrassing, not as a professional, but as a human being. What´s more, no one mentioned the absent elephantine concept of causation, so everyone who went in clueless stayed happy in their ignorance throughout the whole wham-bam-tank-you-mam dog and pony session.

Now, I do think that the sort of data processing associated with Big Data does have a place in the old IT toolkit, but the levels of hype, misappropriation and downright lies is seriously queering the pitch. Just look at some of the Big Data articles in places like Forbes, Information Management and LinkedIn. If you haven’t yet noticed the tendency to use tremendous volumes, varieties and velocities of bullshit to push the Big Data envelope, then you really haven’t been paying enough attention.

Many thanks for reading.

If you enjoy this piece or find it useful then please consider joining The Big Data Contrarians: https://www.linkedin.com/grp/home?gid=8338976

Many thanks, Martyn.