11 Wed Nov 2015
Tags
For those of you who are familiar with the world of Big Data you will also be aware of the vanguard data community known as The Big Data Contrarians (the most fabulous Big Data community online).
Launched today (23 September 2015), the Big Data Contrarian’s Challenge is destined to fast become the most prestigious, enviable and prized challenge on the entire global world-wide-web. Continue reading
11 Wed Nov 2015
Posted in Big Data, Data Lake, hadoop, Inform, educate and entertain., Martyn Jones
Tags
If you enjoy this piece or find it useful then please consider joining The Big Data Contrarians: https://www.linkedin.com/grp/home?gid=8338976
Many thanks, Martyn.
If Princess Diana had been alive during the formative years of the Big Data revolution there would have been a plethora of influential Big Data bullshit babblers issuing their gushingly awful pieces in places like Forbes, the WSJ and professional blogging forums about the Big Data humanitarian causes closest to the heart of the peoples’ princess. And if tragedy had repeated itself, and had been reported not as paparazzi driven schmaltz or morbid vulgarity, but as something even more rancid and farcical, we would now have a Princess Diana Memorial Data Lake in some regal park in London or Milton Keynes –powered by Hadoop. Because, as the bullshit babblers would have it, “that is what she would have wanted”.
But, is this entirely fair? Should we view the outpourings of the biggest Big Data bullshit babblers on the entire internet as the inevitable result of free will, or is Big Data a message from God, in the same way that hard drugs are a signal to certain rock stars that they have too much available cash?
Which brings me to another issue. In a recent interview, I was given a list of data related terms, and was asked which one I preferred. Big Data, Smart Data, Small Data… you know what I mean. Anyway, I went off on a tangent about domestic pets and anthropomorphism. Okay, so it was logical entrapment, but I wanted to make a point. “Don´t you think that ascribing human behavior and thought to pets is a bit weird?” I asked. “No, came back the reply”. It wasn´t the answer I wanted, because the answer I wanted was “Yes, it certainly is” not a “No, that’s what my mum thinks as well”. I wanted to say see, people who ascribe human characteristics to dogs strike us as being a bit fanciful, but people who do the same for data? How can a bunch of recognizable symbols embody smartness? I mean, data by itself, of itself, is dumber than a rock.
So why do we pretend that the information, knowledge and the smarts are in the data and that data itself, without the need for any intervention (other than Hadoop, Sparke or Hive, etc.), is capable of revealing this smartness?
And the only thing I can think is that we are so desperate to sell useless crap that no one needs or wants, that we are even capable of saying the most dopiest of things in order to do so.
Anyway, I was at a Big Data conference recently, and every presenter selling a tool made exactly the same type of pitch. The amazing ways that their tools could establish correlations. Some of the examples of the correlations were so contrived, so obviously the creation of PR than the outcome of hands-off automated analysis, that it became seriously embarrassing, not as a professional, but as a human being. What´s more, no one mentioned the absent elephantine concept of causation, so everyone who went in clueless stayed happy in their ignorance throughout the whole wham-bam-tank-you-mam dog and pony session.
Now, I do think that the sort of data processing associated with Big Data does have a place in the old IT toolkit, but the levels of hype, misappropriation and downright lies is seriously queering the pitch. Just look at some of the Big Data articles in places like Forbes, Information Management and LinkedIn. If you haven’t yet noticed the tendency to use tremendous volumes, varieties and velocities of bullshit to push the Big Data envelope, then you really haven’t been paying enough attention.
Many thanks for reading.
If you enjoy this piece or find it useful then please consider joining The Big Data Contrarians: https://www.linkedin.com/grp/home?gid=8338976
Many thanks, Martyn.
11 Wed Nov 2015
Posted in All Data, Big Data, Good Strategy, goodstrat, Martyn Jones
If you enjoy this piece or find it useful then please consider joining The Big Data Contrarians: https://www.linkedin.com/grp/home?gid=8338976
Many thanks, Martyn.
Pundits far and wide are hailing the end of the period of big data babble, hyperbole and bullshit and are looking forward to an epoch of practical, tangible and verifiable Big Data success stories.
Gartner themselves came out some time ago and declared that Big Data was no longer in the hype cycle. Some took this as a sign that the Big Data bullshit bonanza was over, others were more cynical and suspected a highly orchestrated ruse, a move to the next level in the game plan.
But does this new attitude towards Big Data really ring true?
Accompanying this apparent bold openness, frankness and humility in the ranks of the rehabilitated Big Data bullshit babblers there is an awful lot of what appears to be ‘more of the same’. Or as the people of Thailand might say, “same, same, but different”.
As some of you might know, I am the administrative owner of The Big Data Contrarians community group on LinkedIn, and even I was somewhat taken aback by a recent piece by Bernard Marr entitled 20 Stupid Claims About Big Data. So much so that I wrote a fairly complimentary comment on LinkedIn about it. The thing is, even as a posted it I was thinking to myself “you’ll be sorry”.
Today I read yet another Big Data ‘reformation’ piece on LinkedIn Pulse, this time from Matthew Reaney and with the compelling title of The 5 Myths of Big Data.
Call me naïve, call me illusory, and a believer in humankinds need for basic decency, but I frequently have the idea that praising moderately acceptable behaviour leads to even more good behaviour. But it was not to be, and as fast as one could say ‘what the hell is going on here?’ back came a surfeit of astroturfed Big Data bananas – from all directions – bigger, brasher and more bogus than ever before.
Make no mistake, Big Data hype hasn’t gone away, it has become more subtle, more cunning and even more misleading.
Leading the charge is the initiative to discredit Data Warehousing by all means possible, and the amount of bullshit, disinformation and blatant lies doing the rounds is beginning to look like Big Data hype reflecting Big Data itself, if only in terms of the vast volumes, varieties and velocities that this Big Data babbling bullshit comes in.
But seriously, we are simply getting more of the same, as the end of the Big Data hype war is declared, we are subject to a bombardment of Big Data boloney via Cloud, IoT, the Hadoop ecosphere (as if using Hadoop was someone linked to ecology and saving the planet), and especially this incredibly obnoxious and dopey vehicle for Big Data tripe known widely as the Data Lake – more on that stupidity at some other time. But onwards and upwards…
This all reminds me of a joke from many decades ago, retold in part from memory.
A teacher was looking for a subject about which her class pupils could write, to set as a homework exercise.
After much deliberation she decided to as ask the children to write about what they thought of the police?
Sure, not a good question, I know, and as I stated, this was many decades ago, when even grown-ups could be innocent and naïve and hopeful.
Anyway, when the children had handed in all their essays, the teacher read the essays and was disappointed to find that most of them were very wishy-washy and that the children were almost all unanimously indifferent or grudgingly respectful of the police, except for one. One of the children, let’s call him Dave, was very critical and had written “I don’t think much of the police.” When the teacher asked Dave why he had written that, he replied “All police is bastards, Miss”. The teacher was vexed by the reply, but being a good and caring teacher she considered how she could change this obviously hostile view of the bobby on the beat and the police detective taking evil doers out of circulation, so she decided to do something about it.
She had a bright idea and took her problem to the police and discussed what could be done to give the children a much more positive view of the police and the work they did, so they would see the police as a necessary part of society, to be respected but not feared.
As a result, the teacher and the police organised a police day at the school. It was a big party, with lots of free goodies, badges and posters, rides in patrol cars, sirens, interesting stories and a movie, and a big discussion with the police dog handler and his faithful and brave police-dog, Ajax. The police took special interest in Dave, he was the one they wanted to convince the most, and he was the one they made the most fuss of.
At the end of the day, the teacher again asked the children to write about what they got from the school police day that she had organised.
The following Monday, after all the essays had been handed in by the children, she sought out and read Dave’s essay, eager with anticipation.
This time it contained the surprising phrase of “I really, really don’t think much of the police.”
Again, the teacher asked Dave why he had written what he had wrote, especially considering all the effort the police had gone to in order to leave a good and lasting impression with the children in general, and Dave in particular.
He simply replied “the Police is cunning bastards, Miss.”
Personally, I have respect for the professionalism, courage and hard work of many officers in our police forces, but when it comes to my view of certain Big Data pundits – and naming no names, just watch my eyes – the feeling is not the same.
Make of that what you will.
Many thanks for reading.
If you enjoyed this piece or found it useful then please consider joining The Big Data Contrarians: https://www.linkedin.com/grp/home?gid=8338976
Many thanks,
Martyn.
03 Mon Aug 2015
Posted in Big Data, Consider this, good start, goodstart, Martyn Jones, Strategy
If you enjoy this piece or find it useful then please consider joining The Big Data Contrarians:
Join The Big Data Contrarians here: https://www.linkedin.com/grp/home?gid=8338976
Many thanks.
Now everyone is doing Big Data you don’t want to be the odd one out, right? Of course not.
Now, if you are serious about looking at Big Data from a business perspective then I will try and lend you some advice. If you are doing it from an IT or technology perspective, then I wish you good luck, and I hope that your Big Data initiative doesn’t turn into another tech crash-and-burn show.
Now some Big Data pros are telling us that the place to start with Big Data is with strategy. Now, I’m too polite to call this out as abject bullshit, even though it is, and will instead content myself by offering an alternative and simple approach to approaching and addressing Big Data.
My first piece of advice is this. DON’T START WITH STRATEGY!
Strategy is a coherent, cohesive and executable response to a significant challenge.
Strategy is not a definition of objective, a wish list of what you are trying to achieve or aspirational goals of a nebulous nature. No, strategy is not the objective but a means of reaching that objective. Strategy is real, tangible and executable. Strategy is doing.
So what is a Big Data strategy?
If a company is looking at the Big Data options, the last place they should want to start out from is from strategy. That is as silly idea as they come. Starting with strategy on the road to formulating viable responses to significant challenges and opportunities is like saying that before we choose strategic options and a realisable strategy, then you must have a strategy in place.
Strategy is not working out what you want to achieve. That sort of thing should happen prior to any strategic work. Neither is strategy an exercise in establishing starting points, nor formulating questions nor understanding the challenges. All of this should come well before the major strategy aspects even kicks-in.
Big Data strategy is a realisable, tangible and manageable response to a significant challenge, one that depends heavily on the availability, usability and credibility of Big Data (or Very Large Data Bases) and the business value of processing that Big Data.
So, a word of advice. If you are thinking of embarking on a Big Data initiative, do not start with strategy. That is a really daft place to start.
Start here instead. With real business imperatives. This is where you are thinking about the big and significant challenges to the business, and how, at a high level of abstraction, you could go about meeting those challenges. Here you identify your challenges and your responses, aligned to your objectives.
If you can identify business imperatives that make it absolutely necessary to include elements of Big Data, then go forward with that mandatory requirement in mind. If not, then don’t try to shoe-horn Big Data into a place where it really isn’t needed or wanted. Because if you go against the grain in this way it may well hurt you and your business, in more ways than you bargained for.
In order to go out looking for data requirements driven by business imperatives, we really need to know what we are looking for.
What we are looking for maybe highly tangible or less so. We may have to derive the data we are looking for by refining, aggregating, enriching, filtering and cleansing. Therefore, with those and other aspects in mind, we can go out and find what we need.
From looking at the data requirements, you should have a good idea of potential sources of that data. Agility in this aspect is predicated on the premise that one knows the systems on the IT landscape, the business processes and all the potential sources of data – at a high level at least. So, this is not the sort of work you can do remotely with little or no knowledge of the clients business, IT setup, processes or culture.
But anyway, after you identify the sources you move on to the next step.
Here you discuss aspects of the data you require with the database / application platform owners to ensure that:
So far so good. Once passed these hurdles (and don’t forget this is a super-simplification) we move in to the next.
So, now we know:
Therefore, we go ahead and create a proof of concept or three. Simples!
However, make sure that all prototypes are governed by these simple timeless guidelines:
You run your proof of concept. You analyse, assess and represent your outcomes. You socialise, present and interpret.
When you’ve done that you are in now in a good position to estimate the usefulness of the exercise, from both a qualitative and quantitative perspective.
I did not want to touch in specific aspects of technology in this piece, in part, because I did not consider it a central issue in the theme of things. Of course, as part of creating proofs of concepts and pilot schemes you may want to experiment with the swatch (swaith? oh for auto-correction) of technologies out there. So go ahead and evaluate ‘Big Data’ technologies, and don’t forget, the answer to every Big Data technology question isn’t an automatic ‘Hadoop’. There are other valid Big Data technology options around, such as Lustre and GPFS, or even Oracle, Teradata or EXASol. Also, remember this, if all you are working on is a prototype, a proof of concept or a pilot then you can try and negotiate a free license with any of the major DBMS vendors for that initiative. So negotiate, bargain and get the most appropriate technologies with the best deals.
Finally I will leave you with three guidelines to consider:
Many thanks for reading.
In subsequent blog pieces I will be sharing my views on the evolution of information management in general, and the incorporation novel and innovative techniques, technologies and methods into well architected mainstream information supply frameworks, for primarily strategic and tactical objectives.
As always, please reach out and share your questions, views and criticisms on this piece using the comment box below. I frequently write about strategy, organisational, leadership and information technology topics, trends and tendencies. You are more than welcome to keep up with my posts by clicking the ‘Follow’ link and perhaps you will even consider sending me a LinkedIn invite if you feel our data interests coincide. Also feel free to connect via Twitter, Facebook and the Cambriano Energy website.
For more on this and other topics, check out some of my other posts:
Absolutely Fabulous Big Data Roles – https://www.linkedin.com/pulse/absolutely-fabulous-big-data-roles-martyn-jones?trk=prof-post
Not banking on Big Data? – https://www.linkedin.com/pulse/banking-big-data-martyn-jones?trk=prof-post
10 amazing reasons to join The Big Data Contrarians –https://www.linkedin.com/pulse/10-amazing-reasons-join-big-data-contrarians-martyn-jones?trk=prof-post
Amazing Data Warehousing with Hadoop and Big Data –https://www.linkedin.com/pulse/cloudera-kimball-dw-building-disinformation-factory-martyn-jones?trk=prof-post
The Big Data Contrarians: The Agora for Big Data dialogue –https://www.linkedin.com/pulse/big-data-contrarians-agora-dialogue-martyn-jones?trk=mp-reader-card
The Big Data Shell Game – https://www.linkedin.com/pulse/big-data-shell-game-martyn-jones?trk=mp-reader-card
Aligning Data Warehousing and Big Data –https://www.linkedin.com/pulse/aligning-data-warehousing-big-martyn-jones?trk=mp-reader-card
Big Data Luddites – https://www.linkedin.com/pulse/big-data-luddites-martyn-jones?trk=mp-reader-card
Data Warehousing Explained to Big Data Friends –https://www.linkedin.com/pulse/data-warehousing-explained-big-friends-martyn-jones?trk=mp-reader-card
Big Data, a promised land where the Big Bucks grow –https://www.linkedin.com/pulse/big-data-promised-land-where-bucks-grow-martyn-jones-6023459994031177728?trk=mp-reader-card
The Big Data Contrarians – https://www.linkedin.com/pulse/big-data-contrarians-martyn-jones?trk=mp-reader-card
Is big data really for you? Things to consider before diving in –https://www.linkedin.com/pulse/big-data-really-you-things-consider-before-diving-martyn-jones?trk=mp-reader-card
Big Data Explained to My Grandchildren – https://www.linkedin.com/pulse/big-data-explained-my-grandchildren-martyn-jones?trk=mp-reader-card
If you enjoy this piece or find it useful then please consider joining The Big Data Contrarians:
Join The Big Data Contrarians here: https://www.linkedin.com/grp/home?gid=8338976
Many thanks.
03 Mon Aug 2015
Posted in Big Data, Consider this, good start, goodstart, Martyn Jones, Strategy
Plus ça change, plus c’est la même chose.
Jean-Baptiste Alphonse Karr
I wrote a piece called ‘7 New Big Data Roles for 2015’. I published it on LinkedIn. Many people read it. Some people made suggestions. Others politely ignored it.
I listened to the suggestions, comment and criticisms, and revised the piece as a result.
So here, it is… I hope you like it. And if not, I might try again in six months’ time.
Continue reading12 Tue May 2015
Posted in Big Data, Consider this, good start, goodstart, goostart, Martyn Jones, Stories
Bernice and the Martians, BATM for short, were an incredibly popular progressive-rock band.
Their first big commercial success came with the release of their first album and their planned promotional tour, which took in all continents.
The manager of the band was none other than effable polymath, Renaissance man and good all-round rogue, Ricky Jonesy – an obsessive control freak, lover of fine wines and darling of predictive analytics. He really loved his numbers, his social media and his sentiment analysis.
In fact, much of the early success BATM came about due to Ricky’s unparalleled passion for the ‘Big Data’.
Ricky was the band’s architect. He had major input into their material: what they composed; how they composed; their stage sets and lighting; where they performed; the way they played; how they dressed; were photographed; spoke; walked; and, ate and drank. In short, he controlled the whole BATM enchilada. It was like being in data-driven heaven.
As I said, their first album, a progressive-rock masterpiece called ‘Your Hole’, achieved major critical acclaim even before it was bolting out of the stalls and across the interwebs. Overnight the band became big property, and their notional market value ran higher than Twitter on steroids.
The band members were really please. The presses interviewed Bernice right, left and centre and he made no bones about the fact that a major part of their success was due to Ricky and his Big Data mojo.
Articles about the phenomenon appeared in all the major social media sites. Facebook, LinkedIn and BubbaToons. Ricky was named Supreme Data Scientist of the year by the Gardener Group, hailed as a messiah by the Big Data Front and lauded by all and sundry.
Then the band went on tour. Blazing a trail of ones and zeros across the face of the planet.
They were 5 gigs into their tour and Ricky decided to call a band meeting.
“Hi, guys” said Ricky “I’ve been analysing the stats, and I see that those yokes Big Blokes in Tights are trending strongly on the social media, coinciding with the release of their new single Never Stick A Banger In Your Ear”.
“Oh, whoa” chimes in Bernice, “tell us what we gotta do then, Ricky”.
Back comes Ricky. “Well, this is what I thought we might do”
“We take the old Fester and Ailin song Tropical Diseases, we practice it as much as can, and then we play it at the next gig in Birmingham, this weekend”
“But, Ricky!” pipes up Marty Smarty, “it’s an Irish country and western song. It doesn’t fit in with what we do, does it? And, anyway, we only have three days to get it prepared.”
Ricky responds. “Ah, you don’t want to be worrying your little head over that. Trust me. Learn the song. It’ll be great. The public will love it.”
So, BATM learn the song. It’s perfect. At the Saturday gig, they play it as the encore. The fans love it to bits and there’s not a cold cigarette lighter in the place.
Then they fly off to Palma de Mallorca for a bit of a rest before their next gig in Madrid.
The guys and gals are lounging at the poolside at the legendary Don Pimpón Espinete Plaza complex. The weather is glorious, the food is glorious, the scenery is glorious, and even the orchestra is glorious.
Then along comes Ricky, calling yet another band meeting.
“Hi, guys” said Ricky “I’ve been analysing the stats again, and I see that those yokes Spanky’s Magic Piano are trending strongly on the social media with their cover version of Engel Humpadink’s The Monkey Song”
“Oh yeah, what’s that mean for us, Ricky” chimes in Halo Popette, the bands keyboardist.
Back comes Ricky. “Well, this is what I thought we might do”
“We take the old Fester and Ailin song There’s A Dead Man Up The Chimney, and we rewrite it in the style of Tom Jones when he made that album of his, Little Fockers, was it? Then we practice it as much as can, until it’s perfect, and then we play it at the next gig in Madrid, this weekend”
“But, Ricky!” pipes up Brian McGarsical, “It’s a bit of an odd one isn’t it? I mean to say, it doesn’t fit in with what we do, does it? And, anyway, we only have four days to get it prepared.”
Ricky responds as fast as a chalked-up cat going down a drainpipe. “Ah, you don’t want to be worrying your little head over that. Trust me. Learn the song. It’ll be great. The public will love it. And anyways, it will fit nicely on the playlist, up there with Tropical Diseases.”
The band rewrite the song, and practice the Bedejaysus out of it. Ricky likes it so much that he gets the stats to confirm that this has to be number one on the next gig playlist.
Come the day of the gig, and BATM kick off, not with a progressive-rock anthem, but with There’s A Dead Man Up The Chimney. A group of young people at the front clearly are loving this new sound, but quite a few people are starting at the stage in fright, and it’s not from skunk induced paranoia either.
Two guys are having a conversation at the back of the hall.
“Yo, lunchbox, hurry this gig up, I thought this band was all progressive-rock and stuff, not this wiener schnitzel stuff.”
“No comment.”
Having divided the crowd with their first song, they play songs from their album. Again, they encore with Tropical Diseases. The crowd at the front go wild. The progressive rockers look on, bemused.
“Well, that was a mixed bag” says Bernice.
“Take it from your man Ricky. It all went fine lads. Just needs some fine-tuning of the songs and the analytics need to be a bit more real time. Take me word for it.”
Back comes a unison of “Okay, Ricky. We believe yas!”
So, off they go to Bonn, to prepare for the following weeks gig at the Live Music Hall in Cologne.
The band goes out visiting the museums, they have lunch at Brauhaus Bonnsch, and after a leisurely walk along the banks of Rhine they are taking a beer or three in a lovely little beer garden close to the United Nations campus.
Then out of the blue, a familiar voice can be heard.
“Hi, guys! We’re all goin’ on a summer ‘oliday”. It’s the voice of Ricky. “Anyway, Good news guys. I’ve been analysing the amazin’ Big Data stats again, and I see that those mensch Die Zahnarzt are trending strongly on the social media, especially on Swotter and Titter, with their amazon’ cover version of Podge and Rodge’s chillout mix of Currywurst and Microchips.”
Silence. No one says a word for the best part of infinity.
Ricky continues… “As you’re not going to ask, lads, I’ll tell you. We take the old song A Great Day for the Washing, and we rewrite it in the style of techno-Buddah-bar-chill. Then we practice it as much as can, until it’s perfect, and then we bang it out at the next gig in Cologne, this Friday. Innit. Come on lads, it’s 20 minutes of stage magic, and it’s a breeze.”
Come the day of the gig, and the band arrive early at the hall. Ricky is already there. He’s changed the stage set completely and has a new wardrobe for the lads – Bavarian romantic. They’ll soon be all Princed and Smiley Virused up to the eyeballs, wrecking ball included.
and BATM kick off, not with a progressive-rock anthem or chill, but again with There’s A Dead Man Up The Chimney. Again, a group of young people at the front clearly are loving this new sound, but quite a few people are starting at the stage in drug induced awe. Then they follow that up with A Great Day for Washing. By the time they get to the encore of There’s A Dead Man Up The Chimney, boisterous arguments are breaking out everywhere and empty crisp packets and used sticks of chalk are being thrown at the stage. It’s a disaster.
Four guys are having a conversation at the back of the hall.
“I liked the first song”
“No! The first was terrible. Minging! I want my prog rock back.”
“It’s like the choice of leprosy or the plague.”
“Down with this sort of thing.”
Next day Bernice calls an urgent meeting of the band.
Ricky kicks off.
“Well, lads bit of mid-week game yesterday wasn’t it?”
Bernice comes back with a “You can say that again, Rick”
“Don’t worry, I have analysed the social-media Big Data from all of the concerts, and we’re doing good guys. It’s in the analytics”
“We have to go back to our roots and drop all the changes we made”
A stranger in the lounge where they are having the meeting walks up to them and in simple language explains to them what has happened.
“You created a great product, a great brand, with some interesting progressive music”
“Your music was acclaimed and your world tour was eagerly anticipated by all your fans”
“But then you went wrong”
“You became data driven, dopey and data driven”
“You chased fads, tendencies and styles, and it became a mish-mash”
“People don’t want mish-mashes. Not your base. They wanted good progressive music”
“You’ve lost all credibility. No, you’re just an eccentric band of brothers and sisters that no one will really want to see more than once, if at all”
“Your former fan base is acutely embarrassed by you. That’s your bread, butter, vodka and caviar… in your terms”
“Data drive, Big Data, Big Data analytics in real time?”
“You people have no idea the damage you can do, and so easily”
To be continued…
Many thanks for reading.
20 Fri Mar 2015
Posted in Big Data, Consider this, good start, Good Strat, goodstart, Martyn Jones
Most of us would probably like to work in a profession recognised for its legality, decency and honesty. At least I hope so. In my line of work, what we have right now is palpable evidence that the IT industry lacks a moral compass.
Imagine this. A major sensationalist tabloid pulls together a team of diverse journalists who are set to work on a national campaign to promote very high usage of sunbeds as a cure for cancer. Why? The newspaper owner’s son owns the sunbed franchise.
The health experts criticise the publisher for being irresponsible, unprofessional and lacking in scruples.
The public is mainly undecided, but many take the story on face value and adopt the fad. The intensive use of sunbeds sharply increases. Elsewhere, in unrelated news, the cases of skin cancer show a marked increase. Some blame it on EU legislation for bangers and bananas.
In spite of protests, the press campaign continues over many months.
Eventually, and based on the evidence of recognised health experts and bodies, the press regulatory association tries to get the offending publisher to temper their claims, but without any success. It is only when the government’s lawyers step in and threaten the newspaper owners with legal proceedings, do they freeze their campaign. Much later, the editor resigns and the board of directors issue a short apology on the back pages of their much vaunted organ.
We have that in IT. Our current sunbed cure for cancer, if you believe those who are ‘bigging it up’, is undoubtedly Big Data.
I occasionally post content to Linkedin, some of it (maybe even this piece) gets promoted through the Pulse Big Data channel. There are some reasonable pieces pinned to that channel, but unfortunately, for much of the time what we get is total and moronic Big Data astroturfing. Tantamount to the equivalent of Big Data’s very own Big Lie campaign.
The Linkedin Big Data channel reflects life, and it is full of self-aggrandising and shameless marketing guff, shot-through with scandalously flimsy promotions of tendentious success stories, specious claims of value, half-truths about realisable benefits and embarrassing conjecture about the importance of social media and internet logs.
What I am referring to mainly are superficially neutral (yet virally toxic) pieces placed in the public domain in order to promote Big Data at any cost.
Now let’s step back a bit.
For over 125 years, the Financial Times (FT) has built up a solid professional reputation for accurate reporting, reliable journalism and informative editorials. The FT is a newspaper trusted by its discerning readership and admired everywhere. In fact, I could not imagine their journalists writing about markets, securities and financial houses the same way that pundits elsewhere write about Big Data, Dark Data and the Internet of Things. Because the FT knows, that maintaining the trust of their readership is far more important than winning the short-term favours of a few market players.
So consider this; if we in IT cannot bring our standards of communicating with the public up to the levels of the financial industry, at minimum, you know what that means don’t you?
Exactly. The IT industry will have a far worse public image problem than the bankers and the solicitors currently have, and we all understand the general public appreciation of those professions.
Now, call me old fashioned, but for me that possibility is worthy of serious consideration, and especially by those in IT who confuse no holds barred pimping of fads, trends and technology, in which truth, decency and honesty are optional, for ethical, candid and informative analysis and reporting of the industry.
How will the industry take these criticisms?
To go back to the sunbed analogy what we will most certainly get comments in this vein:
Whilst those who rail against ‘the cancer curing advantages of sunbed use’ may be right – or at least partially right – the sunbed revolution will continue, just as the IT revolution industry has done, and in spite of people saying that the age of computing would be a passing mania.
So, when someone tells you “intensive sunbed use is just a dangerous fad”, what they actually mean to say is that we don’t need the term any more, as intensive sunbed use is here to stay, as are those who are shrewd, unprincipled and cynical enough to cash-in on the public’s gullibility and wilful stupidity when it comes to fads.
Yes, it does get that bad.
We have people who seemingly spend all their waking lives working out not-so-original ways and means of riddling the IT industry with vacuous bullshit, and what Big Data promotion has shown us clearly is that what we have palpable and comprehensive evidence that the IT industry in general lacks a moral compass.
Is that a reflection of IT, of those who create and manipulate IT fads, or of society in general?
Many thanks for reading.
As always, please share your questions, views and criticisms on this piece using the comment box below. I frequently write about strategy, organisational leadership and information technology topics, trends and tendencies. You are more than welcome to keep up with my posts by clicking the ‘Follow’ link and perhaps even send me aLinkedIn invite. Also feel free to connect via Twitter, Facebook and the Cambriano Energy website.
14 Sat Mar 2015
Posted in Big Data, Consider this, dark data, Martyn Jones
Dans ce pays-ci, il est bon de tuer de temps en temps un amiral pour encourager les autres – Voltair
My gran used to tell me that honesty pays. Of course, she never really understood banking or IT, probably because she didn’t want to know anything about them, and she never lived to witness the amazing hype circuses, the spin doctors spiel or the focus-group dog-and-pony show of the 21st century. Indeed, if honesty were a guaranteed payer my gran would have amassed more wealth than even Warren Buffet himself.
If my gran lived today, she might reflect on what Big Data might be about – maybe she would even consider it benignly, as a sort of shelter for fallen men of once uncertain virtue. We will never know. So onwards and upwards.
The Harvard Business Review contemplated honesty in somewhat different terms:
“Honesty is, in fact, primarily a moral choice. Businesspeople do tell themselves that, in the long run, they will do well by doing good. But there is little factual or logical basis for this conviction. Without values, without a basic preference for right over wrong, trust based on such self-delusion would crumble in the face of temptation.”
In a marvellous book, A few good from Univac, David E. Lundstrom narrates the story of Sperry Univac in the 1960s, one of the true great innovators in the first forty years of IT, and includes an allegory taken from the engineering front-line. I will recount it here, edited to highlight the zeitgeist, for your entertainment and as Voltaire put it, “to encourage the others”:
In the beginning was the Big Data Plan.
And then came the Big Data Assumptions.
And the Assumptions were without form.
And the Plan was without substance.
And darkness was upon the face of the Workers.
And they spoke amongst themselves, saying: “It is a crock of shit, and it stinketh.”
And the workers went unto their Supervisors and said: “It is a pail of dung, and none may abide the odor thereof.”
And the Supervisors went unto their Managers, saying: “It is a container of excrement, and it is very strong, such that none may abide by it.”
And the Managers went unto their Directors, saying: “It is a vessel of fertilizer, and none may abide its strength.”
And the Directors spoke amongst themselves, saying to one another: “It contains that which aids plant growth, and it is very powerful.”
And the Vice Presidents went unto the President, saying unto him: “This new plan will actively promote the growth and vigor of the company, with powerful effects.”
And the President looked upon the Big Data Plan, and saw that it was good.
“But?” I hear you say, “why fight it, why not take advantage of the Big Data zeitgeist?”, “Why not cash in on the grand bonanza Big Data bandwagon?” or “Monetise the 3 three famous Vs of Big Data?”
Well, it had crossed my mind, briefly, and (outside of the USA) we’ve all done stuff we have not entirely believed in, so the temptation to cash in is present, capisci? This paraphrasing of a piece from My Blue Heaven might give you a better idea:
One of my best friends makes his living as a completely phony Big Data Scientist. For two hundred bucks he can make you a Data Scientist or a Big Data guru. Some guys give you an education but this guy gives you immediate access to high paying jobs, sex that would make the 256 trillion Shades of Blah blush and a life in the City, the Big Apple or a small town in Germany.
Moreover, for an extra 250 bucks (limited time offer) you can also become a certified Big Data Neuro Trainer, which will allow you to do unto others what has been done unto you.
I also considered Big Data Brokerage, Big Data Certification and Big Data Independent Trading (New York – Paris – Peckham). The opportunities are immense.
However, what happens when the Big Data well runs dry, and I (and many others get tarnished with the mark of Big Data) become pariah by complicity, collusion or simple association?
That question I will leave for another day. But just consider the following.
All right, I admit, I am a big long-time fan of comic genius Mel Brooks, who has a knack of capturing deep insight from the human condition, especially when the human condition is off guard and shallow. In that vein, this is how I like to think the dialogue from the Dole Office scene from The History of the World Part Two would have gone, if he were to write that today:
Dole Office Clerk: Occupation?
Data Magnus Comicus: Stand-up Big Data scientist.
Dole Office Clerk: What?
Data Magnus Comicus: Stand-up Big Data scientist. I coalesce the vaporous datas of the human interaction with the social-media networking, Internet of Everything, and always-connected experience into a… viable, analytical and meaningful predictive-comprehension.
Dole Office Clerk: Oh, a Big Data bullshit artist!
Data Magnus Comicus: *Grumble*…
Dole Office Clerk: Did you bullshit Big Data last week?
Data Magnus Comicus: No.
Dole Office Clerk: Did you try to bullshit Big Data last week?
Data Magnus Comicus: Yes!
Finally, I leave you with some wise words from Israeli American professor of psychology and behavioural economics, Dan Ariely:
“Big data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it…”
Many thanks for reading.
06 Fri Mar 2015
Posted in Big Data, Consider this, Data Warehousing, Good Strat, hadoop, hdfs, Martyn Jones
Tags
Big Data, enterprise data warehousing, Good Strat, Good Strategy, Martyn Jones, Martyn Richard Jones
Hold this thought: To paraphrase the great Bob Hoffman, just when you think that if the Big Data babblers were to generate one more ounce of bull**** the entire f****** solar system would explode, what do they do? Exceed expectations.
I am a mild mannered person, but if there is one thing that irks me, it is when I hear variations on the theme of “Data Warehousing is Big Data”, “Big data is in many ways an evolution of data warehousing” and “with Big Data you no longer need a Data Warehouse”.
Big Data is not Data Warehousing, it is not the evolution of Data Warehousing and it is not a sensible and coherent alternative to Data Warehousing. No matter what certain vendors will put in their marketing brochures or stick up their noses.
In spite of all of the high-visibility screw-ups that have carried the name of Data Warehousing, even when they were not Data Warehouse projects at all, the definition, strategy, benefits and success stories of data warehousing are known, they are in the public domain and they are tangible.
Data Warehousing is a practical, rational and coherent way of providing information needed for strategic and tactical option-formulation and decision-making.
Data Warehousing is a strategy driven, business oriented and technology based business process.
We stock Data Warehouses with data that, in one way or another, comes from internal and optional external sources, and from structured and optional unstructured data. The process of getting data from a data source to the target Data Warehouse, involves extraction, scrubbing, transformation and loading, ETL for short.
Data Warehousing’s defining characteristics are:
Subject Oriented: Operational databases, such as order processing and payroll databases and ERP databases, are organized around business processes or functional areas. These databases grew out of the applications they served. Thus, the data was relative to the order processing application or the payroll application. Data on a particular subject, such as products or employees, was maintained separately (and usually inconsistently) in a number of different databases. In contrast, a data warehouse is organized around subjects. This subject orientation presents the data in a much easier-to-understand format for end users and non-IT business analysts.
Integrated: Integration of data within a warehouse is accomplished by making the data consistent in format, naming and other aspects. Operational databases, for historic reasons, often have major inconsistencies in data representation. For example, a set of operational databases may represent “male” and “female” by using codes such as “m” and “f”, by “1” and “2”, or by “b” and “g”. Often, the inconsistencies are more complex and subtle. In a Data Warehouse, on the other hand, data is always maintained in a consistent fashion.
Time Variant: Data warehouses are time variant in the sense that they maintain both historical and (nearly) current data. Operational databases, in contrast, contain only the most current, up-to-date data values. Furthermore, they generally maintain this information for no more than a year (and often much less). In contrast, data warehouses contain data that is generally loaded from the operational databases daily, weekly, or monthly, which is then typically maintained for a period of 3 to 10 years. This is a major difference between the two types of environments.
Historical information is of high importance to decision makers, who often want to understand trends and relationships between data. For example, the product manager for a Liquefied Natural Gas soda drink may want to see the relationship between coupon promotions and sales. This is information that is almost impossible – and certainly in most cases not cost effective – to determine with an operational database.
Non-Volatile: Non-volatility means that after the data warehouse is loaded there are no changes, inserts, or deletes performed against the informational database. The Data Warehouse is, of course, first loaded with cleaned, integrated and transformed data that originated in the operational databases.
We build Data Warehouses iteratively, a piece or two at a time, and each iteration is primarily a result of business requirements, and not technological considerations.
Each iteration of a Data Warehouse is well bound and understood – small enough to be deliverable in a short iteration, and large enough to be significant.
Conversely, Big Data is characterised as being about:
Massive volumes: so great are they that mainstream relational products and technologies such as Oracle, DB2 and Teradata just can’t hack it, and
High variety: not only structured data, but also the whole range of digital data, and
High velocity: the speed at which data is generated, transmitted and received.
These are known as the three Vs of Big Data, and they are subject to significant and debilitating contradictions, even amongst the gurus of Big Data (as I have commented elsewhere: Contradictions of Big Data).
From time to time, Big Data pundits slam Data Warehousing for not being able to cope with the Big Data type hacking that they are apparently used to carrying out, but this is a mistake of those who fail to recognise a false Data Warehouse when they see one.
So let’s call these false flag Data Warehouse projects something else, such as Data Doghouses.
“Data Doghouse, meet Pig Data.”
Failed or failing Data Doghouses fail for the same reasons that Big Data projects will frequently fail. Both will almost invariably fail to deliver artefacts on time and to expectations; there will be failures to deliver value or even simply to return a break even in costs versus benefits; and of course, there will be failures to deliver any recognisable insight.
Failure happens in Data Doghousing (and quite possibly in Big Data as well) because there is a lack of coherent and cohesive arguments for embarking on such endeavours in the first place; a lack of real business drivers; and, a lack of sense and sensibility.
There is also a willing tendency to ignore the advice of people who warn against joining in the Big Data hubris. Why do some many ignore the ulterior motives of interested parties who are solely engaged in riding on the faddish Big Data bandwagon to maximise the revenue they can milk off punters? Why do we entertain pundits and charlatans who ‘big up’ Big Data whilst simultaneously cultivating an ignorance of data architecture, data management and business realities?
Some people say that the main difference between Big Data and Data Warehousing is that Big Data is technology, and Data Warehousing is architecture.
Now, whilst I totally respect the views of the father of Data Warehousing himself, I also think that he was being far too kind to the Big Data technology camp. However, of course, that is Bill’s choice.
Let me put it this way, if Oracle gave me the code for Oracle 3, I could add 256 bit support, parallel processing and give it an interface makeover, and it would be 1000 times better than any Big Data technology currently in the market (and that version of Oracle is from about 1983).
Therefore, Data Warehousing has no serious competing paragon. Data Warehousing is a real architecture, it has real process methodologies, it is tried and proven, it has success stories that are no secrets, and these stories include details of data, applications and the names of the companies and people involved, and we can point at tangible benefits realised. It’s clear, it’s simple and it’s transparent.
Just like Big Data, right?
Well, no.
See what I mean?
Therefore, the next time someone says to you that Big Data will replace Data Warehousing or that Data Warehousing is Big Data, or any variations on that sort of ‘stupidity’ theme, you can now tell them to take a hike, in the confidence that you are on the side of reason.
Many thanks for reading.
Aligning Big Data: http://www.linkedin.com/pulse/aligning-big-data-martyn-jones
Big Data and the Analytics Data Store: http://www.linkedin.com/pulse/big-data-analytics-store-martyn-jones
A Modern Manager’s Guide to Big Data:http://www.linkedin.com/pulse/managers-guide-big-data-context-martyn-jones
Accomodating Big Data
01 Sun Mar 2015
Posted in Big Data, Consider this, Good Strat, Good Strategy, Martyn Jones, Martyn Richard Jones
Please note: This is an edited version of a previous piece with a similar name, but focusing solely on the three main Vs of Big Data.
What we’ve been toldWe’ve been told that business Big Data is the greatest thing since sliced bread, and that its major characteristics are:
Which is a simple and straightforward means of classification. Big Data is about massive volumes, high variety and high velocity. Right?
It’s not about bigI have never bought into the idea that more data is necessarily better data, or that it provides better focus or leads to increased insight, in fact I have been quite vocal with my contrarian opinion, but now this view is getting some additional support, and from some surprising corners.
In a recent blog piece on IBM’s Big Data and Analytics Hub (Big data: Think Smarter, not bigger), Bernard Marr wrote that “the truth is, it isn’t how big your data is, it’s what you do with it that matters!”
Over at Fierce Big Data it was Pam Baker who stated that “the term big data is unfortunate because it’s really not about the size of the data”. (Big data is not about petabytes, but complex computing).
Elsewhere, SAS echoed similar sentiments on their web site: “The real issue is not that you are acquiring large amounts of data. It’s what you do with the data that counts.”
Well, apparently Big Data isn’t about “massive volumes” of data.
Strike 1!
It is claimed that 20% of digital data is structured, it is based on the problematic suggestion that structured data is uniquely relational.
It is also said that unstructured data includes CSV files and XML data, and this makes up far more than the 20% of the data generated. But this definition is wrong.
If anything, CSV data is structured, and XML data is highly structured, and it’s typically regular ASCII data. So there it does not add variety, even though it is not structured in the ways that some someone might expect, especially if that someone lacks the required knowledge and experience. Simply stated, CSV data is structured, it’s just that it lacks rich metadata, but that doesn’t make it unstructured.
“But”, I hear you say “what about all the non-textual data such as multi-media, and what about the masses of unstructured textual data?”
Take it from me, most businesses will not be basing their business strategies on the analysis of a glut of selfies, juvenile twittering, home videos of cute kittens, or the complete works of William Shakespeare. Almost all business analysis (whether done by a professional statistician or a data scientist) will continue to be carried out using structured data obtained primarily from internal operational systems and external structured data providers.
Variety, Sir? No problem.
Strike two!
So, if we accept that Big Data isn’t really about the massive data volumes or high data variety then that leaves us with velocity. Because if it isn’t about record breaking VLDB or significant data variety, then for most commercial businesses the management of data velocity becomes either less of an issue or just is no issue.
Even in some extreme circumstances, one can explore the suggestion that data sampling can remove issues with data volume as well as velocity.
However, the fact that some software vendors and IT service suppliers set up this‘straw man’ velocity argument and then knock it down with the ‘amazing powers’ of their products and services, is quite another matter.
So, is it really about velocity?
Strike three!
Big Data is a dopey term, applied necessarily ambiguously to a surfeit of tenuously connected vagaries, and its time has come and gone. Let’s dump the Big Data moniker, and the 3 Vs along with it, and embrace the fact that data is data, there will always be more of it.
So, let’s consider ‘all data’ and principally for its time and place utility.
If there is something that you are not sure about or have questions with then please leave a comment below or email me.
Thanks very much for reading.
GOOD STRATEGY 