11 Wed Nov 2015
Tags
Here are the second seven talking points in this series that deal with aspects of leadership, coaching and management. Enjoy! Continue reading
11 Wed Nov 2015
Tags
You love data. You eat, breathe and sleep data! You source it, clean it, integrate and then analyse it until it confesses. You represent, invent and present results. Data is your life and Big Data is your prophet. The Big Data Big Top is the place to be, and (passively) that is where you are headed. For you, Big Data drives everything we do! Is that the case?
Yes?
No worries, in spite of all of that, you too can also be a useful member of The Big Data Contrarians.
Continue reading11 Wed Nov 2015
Tags
First, a request, please consider joining The Big Data Contrarians.
I have worked with clients across the entirety of financial industry for most of my career, and although this may surprise some people, I believe that I fully understand why they are being conservative about Big Data in general and Hadoop in particular. I can also understand why some people want to keep up or even ramp-up even more the Big Data market buzz, but with such a dearth of meaningful, well described and verifiable Big Data ‘success stories’, neither the banks nor I are going to be speculating in any big way on Big Data or Hadoop, anytime soon.
Based in Spain for almost three decades, I have been up close and intimate with a few of the biggest players in the Spanish financial industry. Indeed, Spanish banks have not only lead the way in the effective, innovative and business driven use of technologies in the Spanish market, but have applied that financial industry nous around the world.
In recent times, the big financial players in Spain have entered into the Big Data fields and stratospheres. From what I know, which may not be all, or so much, they are still watching and investigating rather than putting tangible things in production. Nevertheless, there are some interesting Big Data application ideas floating around the financial world. These are still relatively early days for Big Data in finance, and it will take some time for the hype to fade away and the cream of financial Big Data to rise to the top.
However, it has happened before.
If there was ever a country that quietly, diligently and consciously implemented Data Warehousing and Business Intelligence, then it has been Spain. Spanish companies were not only early adopters but also early beneficiaries of implementing Data Warehousing. Not for nothing did Bill Inmon’s company Prism Solutions chose Madrid as a major hub for its European Data Warehousing consulting, sales and support activities. Bill being the father of Data Warehousing and Prism being one of his commercial babies.
As an aside, at Prism I had the opportunity of working alongside fantastic professionals and great people with knowledge, values and experience, such as Don and Katherine. That great gig, I will never forget.
Which brings me to this.
I knew what Data Warehousing would be good for, and amplified this knowledge through reasonable, rational and coherent ways of addressing a wide range of requirements. My aim was to support my claims with coherent, simple and verifiable examples of Data Warehousing success stories.
I knew how to explain Inmon’s Data Warehousing, in business, management and technical terms. I saw when a company could benefit from DW and also when a company was not ready for DW. However, try as I might, I cannot achieve the same intensity of understanding with Big Data. Believe me I have tried.
I’m not a contrarian just because, but isn’t it about time the Big Data BS babblers put up or shut up?
So, if you are like me, then join The Big Data Contrarians.
Many thanks for reading.
11 Wed Nov 2015
Posted in All Data, Analytics, Big Data, Data Lake, Inform, educate and entertain., Martyn Jones
Tags
If you enjoy this piece or find it useful then please consider joining The Big Data Contrarians:
Join The Big Data Contrarians here: https://www.linkedin.com/grp/home?gid=8338976
Being brutally honest, Big Data is bullshit. Not only is Big Data bullshit but it comes with a surfeit of pranksters, pundits and prissy big data bullshit-babblers, all willing (cue narration by Morgan Freeman) to “big-up Big Data in a vulgar, ill-mannered and predictably nauseating dance of professional-hustling… as old as time.”
However, is all Big Data bullshit? Is it all a fad, a load of old tripe and a confusion of weasels together with their surfeit of weasel words? Or, is there something of value, substance and tangibility to be found amongst the volumes, velocities and varieties of brazen and opportunistic self-aggrandizement, toxic speculation and opinions and unverifiable miracles?
For Google, Facebook and Twitter, Big Data certainly isn’t BS. For example, Google rely on Big Data as the biggest irreplaceable element in their colossal advertising business – so I am lead to believe. A business that accounts for more than 90% of Google’s revenue. So clearly, for the masters of web-based unstructured and complex search, Big Data is an essential element in their business model. The biggest essential element by far.
However, let´s be honest, we should consider the obvious. How many of us are really going to do business like Google?
Big Data technology and service vendors benefit tangibly from the Big Data movement, at least this is the impression that I get. Indeed, there is much talk about the relationship between Big Data, the Hadoop ecosphere and the big wild world of open source, but what is more interesting is that companies are bringing in revenue on the back of Big Data by offering battle-hardened business and enterprise versions of open source software. Then there is the business in consulting, with such a demand for Big Data gurus, Master Data Scientists and Number Conjurors, there must presumably be real people working in these roles, and paid handsomely for doing so.
But apart from the ‘success’ associated with the foundation of so many Big Data start-up businesses and the market-based commitment of some of ITs’ ‘great and good’ to the new digital zeitgeist of data volumes, velocities and varieties, just where are the other success stories of Big Data coming from?
To help us in our quest, I earlier compiled a not-so-exhaustive list of Big Data success stories, celebrity-chef like, to help us out. Here are some of the Big Data gems that I managed to track down:
These would be all great headlines for Big Data success stories, apart from one small flaw. None of them is exactly a Big Data success story in the Big Data defining characteristics of volumes, varieties and velocities of mainly unstructured data or in terms of the Hadoop technological kitchen-drawer ecosphere.
Something is happening, and it is not exactly legitimate. Can you guess what it is yet?
When it rains it pours, and when it rains Big Data hype it quickly turn into a monsoon of cloying hysteria. Spotting and pointing at Big Data bullshit babblers on forums like LinkedIn Pulse, Forbes and Information Management is no fun, unless your fun is nuking a school of intellectually challenged fish floundering in a barrel of vintage Malmsey.
However, it not only is no fun, but also more times than not it is a complete and utter waste of time trying to get people to adopt a more critical approach to thinking. Because for every Big Data bullshit babbler, there is a battalion of intransigent Big Data believers stuck in untenable and absurd positions, marooned from reason and ways back to rationality. You can’t use logic against belief, and you can´t turn back a rising tide of IT refugees who are desperately seeking succour in the apparently safer-havens of Big Data, Data Science and Data-driven voodoo.
Only the other day I read that “The emergence of Big Data is now allowing CEOs to increasingly base decisions on current “reality” rather than past experience, but the risks in the integrity and fullness of the data that they are “seeing” and “hearing” is often a barrier to getting a clear picture of what is actually going on.” This is really taking shameless baloney and wilful ignorance to all new heights, but it doesn’t stop there.
Elsewhere another eminent Big Data bullshit babbler wrote, “Clearly big data and AI will change almost every industry this decade… but none more than these”, referring vaguely and vacuously to “Healthcare, Finance and Insurance”. What species of shameless and fatuous willy-waving goes so far out on a limb that it becomes massively removed from even being a grandiose and beguiling ‘bigging-up’ of a fad?
Finally yet importantly, I almost choked on my supersized Big Data popcorn the other day when I read, “Today, with the rise of the Internet, we capture “data” on everything. Therefore, the new term “Big Data” is honestly like 1985 again. But this time, Big Data will actually be really big and by some forecasts, be a $40 billion industry by 2018.”
This is not hype, it is not even simple deceit, it is astroturfing of 22 carat bullshit, and in most cases it’s clearly deliberate, it´s intentional and it´s grossly misleading. So why do people do it?
Given that Big Data is very much a niche technology, with very much a niche appeal, why do so many buffoons go around pretending that Big Data is for all of us? Like as if it was some sort of digital universal-panacea, when at the moment, and at best, it is a walk on bit-player with just a couple of lies who aspires to B actor status. In this sense, at present Big Data isn´t even the hero´s best friend.
Before I close the piece, I will leave you with the thoughts of Dan Ariely. Why? Because it just irritates the hell out of a section of the community of Big Data bullshit babblers, and it´s actually very accurate. Here it is:
“Big data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it…”
Many thanks for reading.
In subsequent blog pieces I will be sharing my views on the evolution of information management in general, and the incorporation novel and innovative techniques, technologies and methods into well architected mainstream information supply frameworks, for primarily strategic and tactical objectives.
As always, please reach out and share your questions, views and criticisms on this piece using the comment box below. I frequently write about strategy, organisational, leadership and information technology topics, trends and tendencies. You are more than welcome to keep up with my posts by clicking the ‘Follow’ link and perhaps you will even consider sending me a LinkedIn invite if you feel our data interests coincide. Also feel free to connect via Twitter, Facebook and the Cambriano Energy website.
For more on this and other topics, check out some of my other posts:
Looking for your most valuable data? Follow the money – http://www.itworld.com/article/2982352/big-data/looking-for-your-most-valuable-data-follow-the-money.html
Stuff a great data architect should know –https://goodstrat.com/2015/08/16/stuff-a-great-data-architect-should-know-how-to-be-a-professional-expert
Big Data, the promised land where ‘smart’ is the new doh!–https://www.linkedin.com/pulse/big-data-promised-land-where-smart-new-doh-martyn-jones?trk=prof-post
Absolutely Fabulous Big Data Roles –https://www.linkedin.com/pulse/absolutely-fabulous-big-data-roles-martyn-jones?trk=prof-post
Not banking on Big Data? – https://www.linkedin.com/pulse/banking-big-data-martyn-jones?trk=prof-post
10 amazing reasons to join The Big Data Contrarians –https://www.linkedin.com/pulse/10-amazing-reasons-join-big-data-contrarians-martyn-jones?trk=prof-post
Amazing Data Warehousing with Hadoop and Big Data–https://www.linkedin.com/pulse/cloudera-kimball-dw-building-disinformation-factory-martyn-jones?trk=prof-post
The Big Data Contrarians: The Agora for Big Data dialogue–https://www.linkedin.com/pulse/big-data-contrarians-agora-dialogue-martyn-jones?trk=mp-reader-card
The Big Data Shell Game – https://www.linkedin.com/pulse/big-data-shell-game-martyn-jones?trk=mp-reader-card
Aligning Data Warehousing and Big Data –https://www.linkedin.com/pulse/aligning-data-warehousing-big-martyn-jones?trk=mp-reader-card
Big Data Luddites – https://www.linkedin.com/pulse/big-data-luddites-martyn-jones?trk=mp-reader-card
Data Warehousing Explained to Big Data Friends –https://www.linkedin.com/pulse/data-warehousing-explained-big-friends-martyn-jones?trk=mp-reader-card
Big Data, a promised land where the Big Bucks grow –https://www.linkedin.com/pulse/big-data-promised-land-where-bucks-grow-martyn-jones-6023459994031177728?trk=mp-reader-card
The Big Data Contrarians – https://www.linkedin.com/pulse/big-data-contrarians-martyn-jones?trk=mp-reader-card
Is big data really for you? Things to consider before diving in–https://www.linkedin.com/pulse/big-data-really-you-things-consider-before-diving-martyn-jones?trk=mp-reader-card
Big Data Explained to My Grandchildren –https://www.linkedin.com/pulse/big-data-explained-my-grandchildren-martyn-jones?trk=mp-reader-card
If you enjoy this piece or find it useful then please consider joining The Big Data Contrarians:
Join The Big Data Contrarians here: https://www.linkedin.com/grp/home?gid=8338976
Many thanks, Martyn.
11 Wed Nov 2015
Tags
If you enjoy this piece or find it useful then please consider joining The Big Data Contrarians: https://www.linkedin.com/grp/home?gid=8338976
Many thanks, Martyn.
“Contrary to slanderous Eastern opinion, much of Iowa is not flat, but rolling hills country with a lot of timber, a handsome and imaginative landscape, crowded with constant small changes of scene and full of little creeks winding with pools where shiners, crappies and catfish hover.”
Paul Engle
Catfish are said to be named because of their passing resemblance to land-roving felines. Admittedly, it’s not like any cat I’ve seen around the house, but if you simultaneously squint your eyes – impressionist style, guzzle a quart of bourbon and smoke a stash of ganja then maybe the resemblance becomes more obvious.
Catfish come in all sizes and varieties, at times they are native and other times they are classed as an alien species, rather like this Welshman who finds himself living in the Spain of Evo Morales, Kirchner and King Mohammed. Nonetheless, you won’t find many thrilling and delightful catfish videos on YouTube nor will you see many entered for the best of breed category at the International Cat Show.
So, what have catfish got to do with Big Data?
Well, there’s loads of them, they come in many varieties, and when they aren’t eating, they can be quite swift. But that’s not what I really wanted to discuss.
Now imagine this. Given the immense geographic dispersion, varieties and volumes of catfish around the world, wouldn’t it be interesting to carry out the Ma and Pa of all Big Data experiments?
We capture – over time of course, this is not the work of one day – all the catfish in the world, and we not only electronically tag them but we also fit them with IoT (Internet of Things) devices that will tell us:
We could then collect this data, in centers all around the world, and then bring it all together in a massive Catfish Big Data Processing Centre in, for example, Coney Island.
Then the data we have so carefully collected, multiplied twice, and then searched and word-counted, in parallel, can be put to revolutionary, evolutionary and amazing uses such as:
Over time, the competence, repertoire and agility of Catfish of all varieties, species, volumes and velocities (did anyone mention Catfish voracity and veracity?) could be augmented, potentiated and expanded by invasive, elliptical and sublime manipulation and neuro-retraining. We could then start with in-aqua interactive stimulus, menu variation and programming and extra-sensory passivation. Later the experiments could be more complex and more all-inclusive, reaching greater and greater degrees of perfection and inclusivity and exclusivity as the Catfish Big Data bandwagon rolled on… Waterlogged, waylaid and none the wiser. Indeed, in the future, all individual decisions will also rely on Catfish input, insight and turbo-charged predictive analytics of great and lasting charm.
Diet manipulation, an habituation test, and chemical analysis of urinary free amino acids were used to demonstrate that bullhead catfish (Ictalurus nebulosus) naturally detect the body odors of conspecifics and respond to them in a predictable fashion. These signals are used in dominance and territorial relationships and lead to increased aggression toward chemical “strangers.” The results support the general notion that nonspecific metabolites, as well as specific pheromones, are important in chemical mediation of social behavior.
There is also one very important thing about catfish that not many people know – apart from Michael Caine, who of course is a leading authority on catfish – and not many people know that either. But, anyway… Catfish are also bottom feeders, this is because of some complex physiological configuration that I won’t go into here – for fear of hurting the sensibilities of the puerilely prudish and wasting valuable drinking time – so in terms of data, the Catfish are able to plumb the depths of the most obtuse, dark and murky data, gobble it up, transform it and… err… load it into Hadoop, to be analyzed with Spark and presented in Excel… or something like that.
So, you’re not convinced by this story? Okay, I didn’t want to tell you this, but here it goes…
Many of us worry about leveraging all data, and mainly we worry because we don’t really have a clue about what we are bullshitting about. We see Big Data, and we believe that is good, whether we know this to be true or not. We are grasping at straws like so many bottom feeders, so many feces-eating walking-catfish, motivated by ideas of maximizing the sale of useless and outdated crap to ignorant people who don’t need it and can’t derive any tangible benefit from it in the first place. This is the biggest takeaway from this current schizophrenic Big Data BS Kulturkampf. Beyond a limited set of interest stories and an even more limited set of peripheral benefits accruable in very specific circumstances, there is nothing tangible that really grabs the attention, apart from the razzle-dazzle, smoke and mirrors of vacuous cant dressed up as showmanship.
The biggest problem with Big Data isn’t so much the plethora of technology (which is more and more reminding me of box of half-eaten chocolates,) nor even the niche applications – for as miraculous and mysterious as most of them are. It’s more about Big Data being turned into a seriously creepy religion, where belief is paramount, and where there is little or no questioning of the tenets, the fables, the dogma and the liturgy, and where one person’s willful ignorance is just as valid as another person’s aspiration to gain knowledge and experience.
Make no mistake, Big Data can be useful for certain businesses and for certain situations. But for many of us in practice it’s either a peripheral player or doesn’t even make it to the bench.
A final thought. Treating Big Data as a religion is foolish, unhelpful and ultimately doomed to failure and ignominy. You have been warned!
For what it’s worth, I am currently writing the Ma and Pa of all Big Data parallel-analytics languages (details to follow), and I might even call it catfish (it’s sorta catchy) and I will have it represented by a muddy-looking open-source cartoon catfish, one worthy of a spot on YouTube.
Many thanks for reading.
If you enjoy this piece or find it useful then please consider joining The Big Data Contrarians: https://www.linkedin.com/grp/home?gid=8338976
Many thanks, Martyn.
19 Wed Aug 2015
Posted in Big Data, good start, goodstart, goodstrat
If you enjoy this piece or find it useful then please consider joining The Big Data Contrarians:
Join The Big Data Contrarians here: https://www.linkedin.com/grp/home?gid=8338976
Many thanks.
To the layperson anxious for answers to complicated questions, the very idea of bringing together sets of disparate data and turning it into precious insights may seem like magic, a modern day alchemy, a goal placed well beyond the grasp of mere mortals. Fortunately, this is no longer the case, thanks in part to bagatelle-proportioned advances in Big Data and Big Data analytics and massive advances in imagination; we are able to look into the past, the present and the future, with absolute certainty. Continue reading
16 Sun Aug 2015
Posted in Consider this, data architecture, goodstart, goodstartegy
Tags
accountability, data archtecture, good start, goodstart, goodstrat, Martyn Jones, Martyn Richard Jones, Strategy
Stuff a really great Data Architect should know
If you enjoy this piece or find it useful then please consider joining The Big Data Contrarians:
Join The Big Data Contrarians here: https://www.linkedin.com/grp/home?gid=8338976
Many thanks. Continue reading
22 Wed Jul 2015
Posted in Big Data, Consider this, good start, Good Strat, goodstart, goodstrat
Tags
As a child, I adored the USA rock band the Eagles, especially the musical talents of Joe Walsh. This explains the inspiration behind the title of this piece.
So, what’s going down at Ashley Madison?
Never heard of them? Off your radar? Surely not?
That stretches the bounds of incredulity. As even the people in Singapore’s Media Development Authority have heard of them. They even described their business site this way “it promotes adultery and disregards family values”, and subsequently will not allow them to operate in Singapore. Well, what a turn-up for the books.
On a more serious note, and as you might know, (from Wikipedia or some other ‘sites’,) Ashley Madison is a Canadian-based online dating service and social networking service marketed to people who are married or in a committed relationship. Its slogan is “Life is short. Have an affair.” It seems, if we are to believe various reports doing the rounds, that their Big Data has been compromised, big time.
Yes, I know, how could that possibly have happened, right?
According to some reports, Adison Mashley have around 37 million clients in the Big Data pool, and large caches of it have allegedly been stolen after an apparently successful hacking attempt was carried out. According to Krebs On Security, data stolen from the web site in question “have been posted online by an individual or group that claims to have completely compromised the company’s user databases, financial records and other proprietary information.”
But, again I ask, how can this happen?
I am not an avid fan of Big Data technology for core business use, and given the level of Big Data technology maturity, it sounds like a dopey idea. But each to their own.
What I will state is that my database management experience has tended to be associated with database technologies that can only be hacked as part of an inside job i.e. where people either know user IDs, passwords, IP addresses and layers of protection etc. or know of someone who does. Either someone who is a friend, part of the family (no, not that type of ‘family’) or someone who can be blackmailed into divulging the required access paths and security check workarounds.
However, taking a broader and more permissive view of this alleged hackerisation of Big Data, do we write it up as a Big Data success, i.e. The Amazing Big Data Affair? Put it down to a technical glitch and community faux pas? Or do we take a jaundiced view of the whole thing and keep it real? I await with baited breath for the enlightened opinions of the Big Data gurus.
Mitch ‘n’ Andy are not unfamiliar with ‘issues’ related to the use of people’s data. The Daily Dot carried a piece from contributing writer S. E. Smith with the headline ‘Why Ashley Madison is cheating on its users with Big Data’ in that piece, Smith states that “Like pretty much every other website on Earth, Ashley Madison spies on its users and crunches the data in a variety of ways to increase the bottom line.”
Belinda Luscombe writing in Time confirmed these suspicions with a piece titled ‘Cheaters’ Dating Site Ashley Madison Spied on Its Users’. She wrote:
In a study to be presented at the 109th Annual Meeting of the American Sociological Association in San Francisco on Saturday Aug. 16, Eric Anderson, a professor at the University of Winchester in England claims that women who seek extra-marital affairs usually still love their husbands and are cheating instead of divorcing, because they need more passion. “It is very clear that our model of having sex and love with just one other person for life has failed— and it has failed massively,” says Anderson.
“How does he know this? Because he spied on the conversations women were having on Ashley Madison, a website created for the purpose of having an affair. Professor Anderson, who as it turns out is a the “chief science officer” at Ashley Madison, looked at more than 4,000 conversations that 100 women were having with potential paramours. “I monitored their conversation with men on the website, without their knowing that I was monitoring and analyzing their conversations,” he says. “The men did not know either.”
Elsewhere, and as reported on Wikipedia, “Trish McDermott, a consultant who helped found Match.com, accused Ashley Madison of being a “business built on the back of broken hearts, ruined marriages, and damaged families.”
Wow, wow, and triple wow! What a way to run a dance hall!
Maybe they should reconsider their slogan, making it more snappy and apposite. How about “Life is short, we pimp your Big Data” as a starter? So go ahead, make your own and post it below. Have fun.
Many thanks for reading.
Oh, and one last thing before I go… GOOD-AD: Join The Big Data Contrarianshttps://www.linkedin.com/grp/home?gid=8338976
21 Tue Jul 2015
Posted in Big Data, Consider this, good start, Good Strat, goodstart, goodstrat, Strategy
Bore da, pobl dda. A hyfryd dydd ‘Big Data’* i bawb.
When it comes to Big Data, some people accuse me of being akin to a Luddite. Nothing could be further from the truth. Not that the facts matter. In the age of superficiality and surfaces there is as much wilfully cultivated obliviousness as there is unashamed and unabashed term abuse. Add the prevailing underlying current of anti-intellectualism into the mix, and we have an explosive combination that manifests itself in the alliterative combination of bluff, bluster and banality.
— JOIN THE BIG DATA CONTRARIANS: http://www.linkedin.com/grp/home?gid=8338976
I was reticent about writing this article, because it’s a bit like arguing against the irrational, self-interested and wilfully obtuse. Or as Ben Goldacre would have it, “You cannot reason people out of a position that they did not reason themselves into.” Therefore, a lot of care needed to be exercised. Indeed, Mark Twain once stated, “Never argue with stupid people, they will drag you down to their level and then beat you with experience.” Now, I wouldn’t go that far, and I do try to be nicely diplomatic, most of the time, but I can see where he was coming from.
Anyway, without more ado let’s get a handle on what a Luddite is, in terms I hope that most will understand.
According to Wikipedia (yes, I know) The Luddites were:
“19th-century English textile workers who protested against newly developed labour-economizing technologies from 1811 to 1816. The stocking frames, spinning frames and power looms introduced during the Industrial Revolution threatened to replace the artisans with less-skilled, low-wage labourers, leaving them without work.”
So why do I get a feeling that some people think that I am a Big Data Luddite?
Here is Peter Powell of PDP Consulting Pty Ltd putting me in my place below the line on my piece titled 7 Amazing Big Data Myths:
“With all due respect – your post does sound a little like what I could envisage an exchange between a man riding a horse and a man driving one if the first automobiles….sorry.”
Although a respectable knowledge of the technology and its evolution would inform otherwise, I assume that this means that I would be the “man riding a horse”… An interesting piece of conjecture indeed, even if hat in lacks in accuracy is made up for by the inexplicable certainty of belief. Still, it’s fascinating to discover just how many ‘experts’ think that this stuff – the sort of stuff I was doing in the mid to late eighties at Sperry and later Unisys – is bleeding edge innovation,
Sassoon Kosian a Sr. Director of Data Science at AIG, had this to tell me on my piece entitled Amazing Big Data Success Stories:
“Yes, cynical indeed… here is another amazing Big Data success story. You go on your computer, type in any search phrase and get instantaneous and highly relevant results. It is so amazing that a word has been coined. Guess what that is…”
What to say? There goes a person who seems to believe that the history of search starts and ends with the Google web search engine. Something slightly less than a munificently inapposite comment, only outdone by its tragically disconnected banality.
More recently, Bernice Blaar had this to say about my take on Big Data in general and The Big Data Contrarians in particular.
“Master Jones may well be the great and ethical strategy data architecture and management guru that the chattering-class Guardian-reading wine-sipping luvvies drool over, but he is also a brazen Big Data Luddite. No, actually far worse than a Luddite, he`s a Neddite, because with his ‘facts’ and ‘logic’ (what a laugh, you can prove anything with facts, can’t you [tou}???) he is undermining the very foundation of the Big Data work, shirk and skive ethic that has been so hardily fought for by the likes of self-sacrificing champions and evangelists of the Big Data revolution, to wit, such as those bold, proud and fine upstanding members Bernard Marr, Martin Fowler and Tom Davenport, for example, and the brave sycophants that worship at their feet. Martyn is worse than Bob Hoffman, Dave Trott, Jeremy Hardy, Mark Steel, Tab C Nesbitt and Bill Inmon, all rolled into one. He may be a great strategist, but I wouldn’t hire him. Contrarian Luddite!”
And then followed it up with this broadside:
“The Big Data Contrarians group are nothing more than a bunch of over-educated clown-shoes who are trying to scupper the hard-work of decent people out to earn a crust from leveraging the promise of a bright future. In a decent society of capital and consumers, they would be banned off the face of the internets.”
How does one reciprocate such flattering flatulence? How can one possible respond to such a long concatenation of meaningless clichés? Though to be fair, I quite liked being referred to as a Neddite, whatever that is.
Anyway, to set the record straight, this is where I stand.
A contrarian is a person who takes up a contrary position, especially a position that is opposed to that of the majority, regardless of how unpopular it may be.
Like others, I am a Big Data Contrarian, not because I am contrary to the effective use of large volumes, varieties and velocities of data, but because I am contrary to the vast quantities of hype, disinformation and biased mendaciousness surrounding aspects of Big Data and some of the attendant technologies and service providers that go with the terrain. I don’t mind people guilding the lily (to use an English aphorism for exaggeration), but I do draw the line at straight out deception., which could lead to unintended consequences, such as creating false expectations, diverting scarce resources to wasteful projects or doing people out of a livelihood. That’s just not tight.
Does that make me a Luddite (or a Neddite)? I don’t think so, but do make sure that your opinion is your own and is arrived at through reason, not some other persons bullying hype. As I wrote elsewhere some moments ago “If you have to lie like an ethically challenged weasel to sell Big Data then clearly there is something amiss.”
As always I would love to hear your opinions and comments on this subject and others, and also please feel free to reach out and connect, so we can keep the conversation going, here on LinkedIn or elsewhere (such as Twitter).
Many thanks for reading.
— JOIN THE BIG DATA CONTRARIANS: http://www.linkedin.com/grp/home?gid=8338976
Photograph: Delegates at my Big Data Summer Camp in Carmarthen (Wales).
*Data mawr
20 Mon Jul 2015
Posted in Big Data, Big Data Analytics, Consider this, Data Warehousing, good start, Good Strat, goodstart, goodstrat
Okay, before we get started I have to declare the real intent for posting this piece. It is to get you to join The Big Data Contrarians professional group here on LinkedIn.
To apply to join the best Big Data community on the web simply navigate to this address http://www.linkedin.com/grp/home?gid=8338976 (or paste it into your browser) and request membership, the process is quick and painless and well worth the effort.
Now for the rest of the news…
There are many common misconceptions amongst the Big Data collective about Data Warehousing. There are common fallacies that need clearing up in order avoid unnecessary confusion, avoidable risks and the damaging perpetuation of disinformation.
In the dim and distant past of business IT, the best information that senior executives could expect from their computer systems were operational reports typically indicating what went right or wrong or somewhere in between. Applied statistical brilliance made up for what data processing lacked in processing power, up to a point, because even heavy lifting statistics requires computing horsepower, which in those days was really a question of serious capital expenditure, which not all companies were willing to commit to.
Then, and curiously coincidentally, people around the world started to posit the need for using data and information to address significant business challenges, to act as input into the processes of strategy formulation, choice and execution. Reports would no longer just be for the Financial Directors or the paper collectors, but would support serious business decision making.
Many initiatives sprang up to meet the top-level decision-making data requirements; they were invariably expensive attempts, with variable outcomes. Some approaches were quite successful, but far too many failed, until the advent of Data Warehousing.
Back then, most of the data that could potentially aid decision-making was in operational systems. Both an advantage and a problem. Data in operational systems was like having data in gaol. Getting data into operational systems was relatively easy, getting it out and moving it around was a nightmare. However, one of the advantages of operational data is that it was generally stored in a structured format, even if data quality was frequently of a dubious nature, and ideas such as subject orientation and integration were far from being widespread.
Of course, data also came in from external sources, but usually via operational databases as well. An example of such data is instrument pricing in financial services.
Therefore, briefly, a lot of Data Warehousing started as a means to provide data to support strategic decision-making. Data Warehousing ways not about counting cakes, widgets or people, which was the purview of operational reporting, or to measure sentiment, likes or mouse behaviour, but to assist senior executives, address the significant business challenges of the day.
Bill Inmon, the father of Data Warehousing, defines it as being “a subject-oriented, integrated, time-variant and non-volatile collection of data in support of management’s decision making process.”
Subject Oriented: The data in the Data Warehouse is organised conceptually (the big canvas), logically (detailing the big picture and) and physically (detailing how it is implemented) by subjects of interest to the business, such as customer and product.
The thing to remember about subject areas is that they are not created ad-hoc by IT according to the sentiments of the time, e.g. during requirements gathering, but through a deeper understanding of the business, its processes and its pertinent business subject areas.
Integrated: All data entering the data warehouse is subject to normalisation and integration rules and constraints to ensure that the data stored is consistently and contextually unambiguous.
Time Variant: Time variance gives us the ability to view and contrast data from multiple viewpoints over time. It is an essential element in the organisation of data within the data warehouse and dependent data marts.
Non-Volatile: The data warehouse represents structured and consistent snapshots of business data over time. Once a data snapshot is established, it is rarely if ever modified.
Management Decision Making: This is the principal focus of Data Warehousing, although Data Warehouses have secondary uses, such as complementing operational reporting and analysis.
In plain language, if what your business has or is planning to have does not fully satisfy the Inmon criteria then it probably is not a Data Warehouse, but another form of data-store.
The thing to remember about informed management decision making is that it needs to be as good as required but it does not need to achieve technical perfection. This observation underlies the fact that Data Warehouse is a business process, and not an obsessive search for zero defects or the application of so called ‘leading edge’ technologies – faddish, appropriate or not.
JOIN THE BIG DATA CONTRARIANS: http://www.linkedin.com/grp/home?gid=8338976
Before we delve into the meaning of Data Warehousing, there are a couple of terms that need to be understood first, so, by way of illustration:
Let’s follow the numbers in the simplification of the process.
This is a very simplified view, but it serves to convey the fundamental chain of events. The most important aspect being that we start (1) and end (7) with the user, and we fully involve them in the non-technical aspects of the process.
JOIN THE BIG DATA CONTRARIANS: http://www.linkedin.com/grp/home?gid=8338976
Essentially, a Data Warehouse is a business driven, enterprise centric and technology based solution for continual quality improvement in the sourcing, integration, packaging and delivery of data for strategic, tactical and operational modelling, reporting, visualisation and decision-making.
A data warehouse is business centric and nothing happens unless there is a business imperative for doing so. This means that there is no second-guessing the data requirements of the business users, and every piece of data in the data warehouse should be traceable to a tangible business requirement. This tangible business requirement is usually a departmental or process specific dimensional data model produced together in requirements workshops with the business. We build the Data Warehouse over time in iterative steps, based on the criteria that the requirements should be small enough to be delivered in a short timeframe and large enough to be significant.
Typically, a Data Warehouse iteration results in a new Data Mart or the revision of an existing Data Mart.
As we build up the collection of Data Marts, we are also building up the central logical store of data known as the Enterprise Data Warehouse that serves as a structured, coherent and cohesive central clearing area for data that supports enterprise decision making. Therefore, whilst we are addressing specific departmental and process requirements through Data Marts we are also building up an overall view of the enterprise data.
By technology, I mean technology in the broadest sense of techniques, methods, processes and tools, and not just a question of products, brands or badges.
Unfortunately, there is a popular misconception that Data Warehousing is primarily about competing popular and commercial available technology products. It isn’t, but they do play an important role.
The following is an example of a very high-level Data Warehouse architecture diagram.
Various methodologies support the building, expansion and maintenance of a Data Warehouse. Here is one example of a professional data integration methodology, produced, maintained and used by Cambriano Energy.
And here is an information value-chain map as used by Cambriano Energy as part of its Iter8 process management. There are alternatives, many of which do a satisfactory job.
Last but not least, this was (from memory) the way that Bill Inmon’s Prism Solutions ETL company used to view the iterative EDW building process.
JOIN THE BIG DATA CONTRARIANS: http://www.linkedin.com/grp/home?gid=8338976
At this point, I decided to cut short further explanations on aspects on Data Warehousing. However, if you have any question then please address them to me and I will do my best (or something close) to answer them.
Hold this thought for another time: If you think you can replace a Data Warehouse, that is not a Data Warehouse, with another approach to ‘Data Warehousing’ that doesn’t produce a Data Warehouse, for as fast and cheap as one can do it, then you still don’t have a Data Warehouse to show for all of your efforts. That is not a great place to be.
Therefore, you see, Data Warehousing was never about a haphazard approach to providing random structured, semi-structured and unstructured data of various qualities, provenance, volumes, varieties and velocities, to whomever was of a mind to want it.
Many thanks for reading.
If you want to connect then please send a request. I you have any questions or comments then fire them off below. Cheers :-)
Oh… and one last thing before I go… DON’T FORGET TO JOIN THE BIG DATA CONTRARIANS: http://www.linkedin.com/grp/home?gid=8338976
GOOD STRATEGY 