Category Archives: goodstrat

Big Data and Catfish

11 Wednesday Nov 2015

Posted by in Big Data, Business Intelligence, goodstrat, Inform, educate and entertain., Martyn Jones, Martyn Richard Jones

≈ Leave a comment

Tags

, , , ,


If you enjoy this piece or find it useful then please consider joining The Big Data Contrarians: https://www.linkedin.com/grp/home?gid=8338976

Many thanks, Martyn.

“Contrary to slanderous Eastern opinion, much of Iowa is not flat, but rolling hills country with a lot of timber, a handsome and imaginative landscape, crowded with constant small changes of scene and full of little creeks winding with pools where shiners, crappies and catfish hover.”

Paul Engle

Catfish are said to be named because of their passing resemblance to land-roving felines. Admittedly, it’s not like any cat I’ve seen around the house, but if you simultaneously squint your eyes – impressionist style, guzzle a quart of bourbon and smoke a stash of ganja then maybe the resemblance becomes more obvious.

Catfish come in all sizes and varieties, at times they are native and other times they are classed as an alien species, rather like this Welshman who finds himself living in the Spain of Evo Morales, Kirchner and King Mohammed. Nonetheless, you won’t find many thrilling and delightful catfish videos on YouTube nor will you see many entered for the best of breed category at the International Cat Show.

So, what have catfish got to do with Big Data?

Well, there’s loads of them, they come in many varieties, and when they aren’t eating, they can be quite swift. But that’s not what I really wanted to discuss.

Now imagine this. Given the immense geographic dispersion, varieties and volumes of catfish around the world, wouldn’t it be interesting to carry out the Ma and Pa of all Big Data experiments?

We capture – over time of course, this is not the work of one day – all the catfish in the world, and we not only electronically tag them but we also fit them with IoT (Internet of Things) devices that will tell us:

  • Where the catfish is
  • Who the catfish is with
  • What are they doing
  • What are they eating
  • How do they feel in general
  • How do they feel about certain things, like the food they just ate, the company they keep, and what they do for entertainment and distraction, etc.

We could then collect this data, in centers all around the world, and then bring it all together in a massive Catfish Big Data Processing Centre in, for example, Coney Island.

Then the data we have so carefully collected, multiplied twice, and then searched and word-counted, in parallel, can be put to revolutionary, evolutionary and amazing uses such as:

  • Analysing and forecasting the Amazon buying trends of the lost Fukawi tribe – yes, the very same tribe who used to wander around boasting about their culture and presence usually accompanied with cries such as “We’re the Fukawi” or “Where the Fukawi?”
  • Creating appealing, compelling and revenue-busting online interactive ads for Bob Hoffman
  • Predicting the outcome of the US Presidential election, the regional elections in Catalonia and the vote for Chairperson at the Hello Working Person’s Club, Hello Village, in Jolly Olde England.
  • Preventing the outbreak of a world-wide pandemic of universal proportions thanks to Big Data being used to intervene virus-bearing inter-terrestrial vehicles sent by radical-fundamentalist-Martians inhabiting the once munificent planet of Zog.
  • Providing a wealth of material success stories that can be liberally sprinkled like fairy-dust on amazing Big Data stories from the keyboards of some of the finest Big Data bullshit babbling princesses on the entire world wide webs.

Over time, the competence, repertoire and agility of Catfish of all varieties, species, volumes and velocities (did anyone mention Catfish voracity and veracity?) could be augmented, potentiated and expanded by invasive, elliptical and sublime manipulation and neuro-retraining. We could then start with in-aqua interactive stimulus, menu variation and programming and extra-sensory passivation. Later the experiments could be more complex and more all-inclusive, reaching greater and greater degrees of perfection and inclusivity and exclusivity as the Catfish Big Data bandwagon rolled on… Waterlogged, waylaid and none the wiser. Indeed, in the future, all individual decisions will also rely on Catfish input, insight and turbo-charged predictive analytics of great and lasting charm.

Diet manipulation, an habituation test, and chemical analysis of urinary free amino acids were used to demonstrate that bullhead catfish (Ictalurus nebulosus) naturally detect the body odors of conspecifics and respond to them in a predictable fashion. These signals are used in dominance and territorial relationships and lead to increased aggression toward chemical “strangers.” The results support the general notion that nonspecific metabolites, as well as specific pheromones, are important in chemical mediation of social behavior.

There is also one very important thing about catfish that not many people know – apart from Michael Caine, who of course is a leading authority on catfish – and not many people know that either. But, anyway… Catfish are also bottom feeders, this is because of some complex physiological configuration that I won’t go into here – for fear of hurting the sensibilities of the puerilely prudish and wasting valuable drinking time – so in terms of data, the Catfish are able to plumb the depths of the most obtuse, dark and murky data, gobble it up, transform it and… err… load it into Hadoop, to be analyzed with Spark and presented in Excel… or something like that.

So, you’re not convinced by this story? Okay, I didn’t want to tell you this, but here it goes…

Many of us worry about leveraging all data, and mainly we worry because we don’t really have a clue about what we are bullshitting about. We see Big Data, and we believe that is good, whether we know this to be true or not. We are grasping at straws like so many bottom feeders, so many feces-eating walking-catfish, motivated by ideas of maximizing the sale of useless and outdated crap to ignorant people who don’t need it and can’t derive any tangible benefit from it in the first place. This is the biggest takeaway from this current schizophrenic Big Data BS Kulturkampf. Beyond a limited set of interest stories and an even more limited set of peripheral benefits accruable in very specific circumstances, there is nothing tangible that really grabs the attention, apart from the razzle-dazzle, smoke and mirrors of vacuous cant dressed up as showmanship.

The biggest problem with Big Data isn’t so much the plethora of technology (which is more and more reminding me of box of half-eaten chocolates,) nor even the niche applications – for as miraculous and mysterious as most of them are. It’s more about Big Data being turned into a seriously creepy religion, where belief is paramount, and where there is little or no questioning of the tenets, the fables, the dogma and the liturgy, and where one person’s willful ignorance is just as valid as another person’s aspiration to gain knowledge and experience.

Make no mistake, Big Data can be useful for certain businesses and for certain situations. But for many of us in practice it’s either a peripheral player or doesn’t even make it to the bench.

A final thought. Treating Big Data as a religion is foolish, unhelpful and ultimately doomed to failure and ignominy. You have been warned!

For what it’s worth, I am currently writing the Ma and Pa of all Big Data parallel-analytics languages (details to follow), and I might even call it catfish (it’s sorta catchy) and I will have it represented by a muddy-looking open-source cartoon catfish, one worthy of a spot on YouTube.

Many thanks for reading.

If you enjoy this piece or find it useful then please consider joining The Big Data Contrarians: https://www.linkedin.com/grp/home?gid=8338976

Many thanks, Martyn.

The Amazing Big Data Challenge – 2015

11 Wednesday Nov 2015

Posted by in Analytics, Big Data, goodstrat, Inform, educate and entertain., Martyn Jones, Martyn Richard Jones, Strategy, The Amazing Big Data Challenge

≈ Leave a comment

Tags

, , ,


For those of you who are familiar with the world of Big Data you will also be aware of the vanguard data community known as The Big Data Contrarians (the most fabulous Big Data community online).

Launched today (23 September 2015), the Big Data Contrarian’s Challenge is destined to fast become the most prestigious, enviable and prized challenge on the entire global world-wide-web. Continue reading →

Whither Big Data bullshit?

11 Wednesday Nov 2015

Posted by in All Data, Big Data, Good Strategy, goodstrat, Martyn Jones

≈ Leave a comment

Tags

, , ,


If you enjoy this piece or find it useful then please consider joining The Big Data Contrarians: https://www.linkedin.com/grp/home?gid=8338976

Many thanks, Martyn.

Pundits far and wide are hailing the end of the period of big data babble, hyperbole and bullshit and are looking forward to an epoch of practical, tangible and verifiable Big Data success stories.

Gartner themselves came out some time ago and declared that Big Data was no longer in the hype cycle. Some took this as a sign that the Big Data bullshit bonanza was over, others were more cynical and suspected a highly orchestrated ruse, a move to the next level in the game plan.

But does this new attitude towards Big Data really ring true?

Accompanying this apparent bold openness, frankness and humility in the ranks of the rehabilitated Big Data bullshit babblers there is an awful lot of what appears to be ‘more of the same’. Or as the people of Thailand might say, “same, same, but different”.

As some of you might know, I am the administrative owner of The Big Data Contrarians community group on LinkedIn, and even I was somewhat taken aback by a recent piece by Bernard Marr entitled 20 Stupid Claims About Big Data. So much so that I wrote a fairly complimentary comment on LinkedIn about it. The thing is, even as a posted it I was thinking to myself “you’ll be sorry”.

Today I read yet another Big Data ‘reformation’ piece on LinkedIn Pulse, this time from Matthew Reaney and with the compelling title of The 5 Myths of Big Data.

Call me naïve, call me illusory, and a believer in humankinds need for basic decency, but I frequently have the idea that praising moderately acceptable behaviour leads to even more good behaviour. But it was not to be, and as fast as one could say ‘what the hell is going on here?’ back came a surfeit of astroturfed Big Data bananas – from all directions – bigger, brasher and more bogus than ever before.

Make no mistake, Big Data hype hasn’t gone away, it has become more subtle, more cunning and even more misleading.

Leading the charge is the initiative to discredit Data Warehousing by all means possible, and the amount of bullshit, disinformation and blatant lies doing the rounds is beginning to look like Big Data hype reflecting Big Data itself, if only in terms of the vast volumes, varieties and velocities that this Big Data babbling bullshit comes in.

But seriously, we are simply getting more of the same, as the end of the Big Data hype war is declared, we are subject to a bombardment of Big Data boloney via Cloud, IoT, the Hadoop ecosphere (as if using Hadoop was someone linked to ecology and saving the planet), and especially this incredibly obnoxious and dopey vehicle for Big Data tripe known widely as the Data Lake – more on that stupidity at some other time. But onwards and upwards…

This all reminds me of a joke from many decades ago, retold in part from memory.

A teacher was looking for a subject about which her class pupils could write, to set as a homework exercise.

After much deliberation she decided to as ask the children to write about what they thought of the police?

Sure, not a good question, I know, and as I stated, this was many decades ago, when even grown-ups could be innocent and naïve and hopeful.

Anyway, when the children had handed in all their essays, the teacher read the essays and was disappointed to find that most of them were very wishy-washy and that the children were almost all unanimously indifferent or grudgingly respectful of the police, except for one. One of the children, let’s call him Dave, was very critical and had written “I don’t think much of the police.” When the teacher asked Dave why he had written that, he replied “All police is bastards, Miss”. The teacher was vexed by the reply, but being a good and caring teacher she considered how she could change this obviously hostile view of the bobby on the beat and the police detective taking evil doers out of circulation, so she decided to do something about it.

She had a bright idea and took her problem to the police and discussed what could be done to give the children a much more positive view of the police and the work they did, so they would see the police as a necessary part of society, to be respected but not feared.

As a result, the teacher and the police organised a police day at the school. It was a big party, with lots of free goodies, badges and posters, rides in patrol cars, sirens, interesting stories and a movie, and a big discussion with the police dog handler and his faithful and brave police-dog, Ajax. The police took special interest in Dave, he was the one they wanted to convince the most, and he was the one they made the most fuss of.

At the end of the day, the teacher again asked the children to write about what they got from the school police day that she had organised.

The following Monday, after all the essays had been handed in by the children, she sought out and read Dave’s essay, eager with anticipation.

This time it contained the surprising phrase of “I really, really don’t think much of the police.”

Again, the teacher asked Dave why he had written what he had wrote, especially considering all the effort the police had gone to in order to leave a good and lasting impression with the children in general, and Dave in particular.

He simply replied “the Police is cunning bastards, Miss.”

Personally, I have respect for the professionalism, courage and hard work of many officers in our police forces, but when it comes to my view of certain Big Data pundits – and naming no names, just watch my eyes – the feeling is not the same.

Make of that what you will.

Many thanks for reading.

If you enjoyed this piece or found it useful then please consider joining The Big Data Contrarians: https://www.linkedin.com/grp/home?gid=8338976

Many thanks,

Martyn.

Big Data, ESP and Transubstantiation

19 Wednesday Aug 2015

Posted by in Big Data, good start, goodstart, goodstrat

≈ Leave a comment

Tags

, , , , ,


vocationIf you enjoy this piece or find it useful then please consider joining The Big Data Contrarians:

Join The Big Data Contrarians here: https://www.linkedin.com/grp/home?gid=8338976

Many thanks.

To the layperson anxious for answers to complicated questions, the very idea of bringing together sets of disparate data and turning it into precious insights may seem like magic, a modern day alchemy, a goal placed well beyond the grasp of mere mortals. Fortunately, this is no longer the case, thanks in part to bagatelle-proportioned advances in Big Data and Big Data analytics and massive advances in imagination; we are able to look into the past, the present and the future, with absolute certainty. Continue reading →

You can’t hide your lyin’ Big Data

22 Wednesday Jul 2015

Posted by in Big Data, Consider this, good start, Good Strat, goodstart, goodstrat

≈ Leave a comment

Tags

, , , ,


As a child, I adored the USA rock band the Eagles, especially the musical talents of Joe Walsh. This explains the inspiration behind the title of this piece.

So, what’s going down at Ashley Madison?

Never heard of them? Off your radar? Surely not?

That stretches the bounds of incredulity. As even the people in Singapore’s Media Development Authority have heard of them. They even described their business site this way “it promotes adultery and disregards family values”, and subsequently will not allow them to operate in Singapore. Well, what a turn-up for the books.

On a more serious note, and as you might know, (from Wikipedia or some other ‘sites’,) Ashley Madison is a Canadian-based online dating service and social networking service marketed to people who are married or in a committed relationship. Its slogan is “Life is short. Have an affair.” It seems, if we are to believe various reports doing the rounds, that their Big Data has been compromised, big time.

Yes, I know, how could that possibly have happened, right?

According to some reports, Adison Mashley have around 37 million clients in the Big Data pool, and large caches of it have allegedly been stolen after an apparently successful hacking attempt was carried out. According to Krebs On Security, data stolen from the web site in question “have been posted online by an individual or group that claims to have completely compromised the company’s user databases, financial records and other proprietary information.”

But, again I ask, how can this happen?

I am not an avid fan of Big Data technology for core business use, and given the level of Big Data technology maturity, it sounds like a dopey idea. But each to their own.

What I will state is that my database management experience has tended to be associated with database technologies that can only be hacked as part of an inside job i.e. where people either know user IDs, passwords, IP addresses and layers of protection etc. or know of someone who does. Either someone who is a friend, part of the family (no, not that type of ‘family’) or someone who can be blackmailed into divulging the required access paths and security check workarounds.

However, taking a broader and more permissive view of this alleged hackerisation of Big Data, do we write it up as a Big Data success, i.e. The Amazing Big Data Affair? Put it down to a technical glitch and community faux pas? Or do we take a jaundiced view of the whole thing and keep it real? I await with baited breath for the enlightened opinions of the Big Data gurus.

Mitch ‘n’ Andy are not unfamiliar with ‘issues’ related to the use of people’s data. The Daily Dot carried a piece from contributing writer S. E. Smith with the headline ‘Why Ashley Madison is cheating on its users with Big Data’ in that piece, Smith states that “Like pretty much every other website on Earth, Ashley Madison spies on its users and crunches the data in a variety of ways to increase the bottom line.”

Belinda Luscombe writing in Time confirmed these suspicions with a piece titled ‘Cheaters’ Dating Site Ashley Madison Spied on Its Users’. She wrote:

In a study to be presented at the 109th Annual Meeting of the American Sociological Association in San Francisco on Saturday Aug. 16, Eric Anderson, a professor at the University of Winchester in England claims that women who seek extra-marital affairs usually still love their husbands and are cheating instead of divorcing, because they need more passion. “It is very clear that our model of having sex and love with just one other person for life has failed— and it has failed massively,” says Anderson.

“How does he know this? Because he spied on the conversations women were having on Ashley Madison, a website created for the purpose of having an affair. Professor Anderson, who as it turns out is a the “chief science officer” at Ashley Madison, looked at more than 4,000 conversations that 100 women were having with potential paramours. “I monitored their conversation with men on the website, without their knowing that I was monitoring and analyzing their conversations,” he says. “The men did not know either.”

Elsewhere, and as reported on Wikipedia, “Trish McDermott, a consultant who helped found Match.com, accused Ashley Madison of being a “business built on the back of broken hearts, ruined marriages, and damaged families.”

Wow, wow, and triple wow! What a way to run a dance hall!

Maybe they should reconsider their slogan, making it more snappy and apposite. How about “Life is short, we pimp your Big Data” as a starter? So go ahead, make your own and post it below. Have fun.

Many thanks for reading.

Oh, and one last thing before I go… GOOD-AD: Join The Big Data Contrarianshttps://www.linkedin.com/grp/home?gid=8338976

Consider this: Big Data Luddites

21 Tuesday Jul 2015

Posted by in Big Data, Consider this, good start, Good Strat, goodstart, goodstrat, Strategy

≈ Leave a comment

Tags

, , , , ,


Bore da, pobl dda. A hyfryd dydd ‘Big Data’* i bawb.

When it comes to Big Data, some people accuse me of being akin to a Luddite. Nothing could be further from the truth. Not that the facts matter. In the age of superficiality and surfaces there is as much wilfully cultivated obliviousness as there is unashamed and unabashed term abuse. Add the prevailing underlying current of anti-intellectualism into the mix, and we have an explosive combination that manifests itself in the alliterative combination of bluff, bluster and banality.

— JOIN THE BIG DATA CONTRARIANS: http://www.linkedin.com/grp/home?gid=8338976

I was reticent about writing this article, because it’s a bit like arguing against the irrational, self-interested and wilfully obtuse. Or as Ben Goldacre would have it, “You cannot reason people out of a position that they did not reason themselves into.” Therefore, a lot of care needed to be exercised. Indeed, Mark Twain once stated, “Never argue with stupid people, they will drag you down to their level and then beat you with experience.” Now, I wouldn’t go that far, and I do try to be nicely diplomatic, most of the time, but I can see where he was coming from.

Anyway, without more ado let’s get a handle on what a Luddite is, in terms I hope that most will understand.

According to Wikipedia (yes, I know) The Luddites were:

“19th-century English textile workers who protested against newly developed labour-economizing technologies from 1811 to 1816. The stocking frames, spinning frames and power looms introduced during the Industrial Revolution threatened to replace the artisans with less-skilled, low-wage labourers, leaving them without work.”

So why do I get a feeling that some people think that I am a Big Data Luddite?

Here is Peter Powell of PDP Consulting Pty Ltd putting me in my place below the line on my piece titled 7 Amazing Big Data Myths:

“With all due respect – your post does sound a little like what I could envisage an exchange between a man riding a horse and a man driving one if the first automobiles….sorry.”

Although a respectable knowledge of the technology and its evolution would inform otherwise, I assume that this means that I would be the “man riding a horse”…  An interesting piece of conjecture indeed, even if hat in lacks in accuracy is made up for by the inexplicable certainty of belief. Still, it’s fascinating to discover just how many ‘experts’ think that this stuff – the sort of stuff I was doing in the mid to late eighties at Sperry and later Unisys – is bleeding edge innovation,

Sassoon Kosian a Sr. Director of Data Science at AIG, had this to tell me on my piece entitled Amazing Big Data Success Stories:

“Yes, cynical indeed… here is another amazing Big Data success story. You go on your computer, type in any search phrase and get instantaneous and highly relevant results. It is so amazing that a word has been coined. Guess what that is…”

What to say? There goes a person who seems to believe that the history of search starts and ends with the Google web search engine. Something slightly less than a munificently inapposite comment, only outdone by its tragically disconnected banality.

More recently, Bernice Blaar had this to say about my take on Big Data in general and The Big Data Contrarians in particular.

“Master Jones may well be the great and ethical strategy data architecture and management guru that the chattering-class Guardian-reading wine-sipping luvvies drool over, but he is also a brazen Big Data Luddite. No, actually far worse than a Luddite, he`s a Neddite, because with his ‘facts’ and ‘logic’ (what a laugh, you can prove anything with facts, can’t you [tou}???) he is undermining the very foundation of the Big Data work, shirk and skive ethic that has been so hardily fought for by the likes of self-sacrificing champions and evangelists of the Big Data revolution, to wit, such as those bold, proud and fine upstanding members Bernard Marr, Martin Fowler and Tom Davenport, for example, and the brave sycophants that worship at their feet. Martyn is worse than Bob Hoffman, Dave Trott, Jeremy Hardy, Mark Steel, Tab C Nesbitt and Bill Inmon, all rolled into one. He may be a great strategist, but I wouldn’t hire him. Contrarian Luddite!”

And then followed it up with this broadside:

“The Big Data Contrarians group are nothing more than a bunch of over-educated clown-shoes who are trying to scupper the hard-work of decent people out to earn a crust from leveraging the promise of a bright future. In a decent society of capital and consumers, they would be banned off the face of the internets.”

How does one reciprocate such flattering flatulence? How can one possible respond to such a long concatenation of meaningless clichés? Though to be fair, I quite liked being referred to as a Neddite, whatever that is.

Anyway, to set the record straight, this is where I stand.

A contrarian is a person who takes up a contrary position, especially a position that is opposed to that of the majority, regardless of how unpopular it may be.

Like others, I am a Big Data Contrarian, not because I am contrary to the effective use of large volumes, varieties and velocities of data, but because I am contrary to the vast quantities of hype, disinformation and biased mendaciousness surrounding aspects of Big Data and some of the attendant technologies and service providers that go with the terrain. I don’t mind people guilding the lily (to use an English aphorism for exaggeration), but I do draw the line at straight out deception., which could lead to unintended consequences, such as creating false expectations, diverting scarce resources to wasteful projects or doing people out of a livelihood. That’s just not tight.

Does that make me a Luddite (or a Neddite)? I don’t think so, but do make sure that your opinion is your own and is arrived at through reason, not some other persons bullying hype. As I wrote elsewhere some moments ago “If you have to lie like an ethically challenged weasel to sell Big Data then clearly there is something amiss.”

As always I would love to hear your opinions and comments on this subject and others, and also please feel free to reach out and connect, so we can keep the conversation going, here on LinkedIn or elsewhere (such as Twitter).

Many thanks for reading.

 

— JOIN THE BIG DATA CONTRARIANS: http://www.linkedin.com/grp/home?gid=8338976

Photograph: Delegates at my Big Data Summer Camp in Carmarthen (Wales).

*Data mawr

Data Warehousing Explained to Big Data Friends

20 Monday Jul 2015

Posted by in Big Data, Big Data Analytics, Consider this, Data Warehousing, good start, Good Strat, goodstart, goodstrat

≈ Leave a comment

Tags

, , , , ,


Okay, before we get started I have to declare the real intent for posting this piece. It is to get you to join The Big Data Contrarians professional group here on LinkedIn.

To apply to join the best Big Data community on the web simply navigate to this address http://www.linkedin.com/grp/home?gid=8338976 (or paste it into your browser) and request membership, the process is quick and painless and well worth the effort.

Now for the rest of the news…

There are many common misconceptions amongst the Big Data collective about Data Warehousing. There are common fallacies that need clearing up in order avoid unnecessary confusion, avoidable risks and the damaging perpetuation of disinformation.

Big Picture

In the dim and distant past of business IT, the best information that senior executives could expect from their computer systems were operational reports typically indicating what went right or wrong or somewhere in between.  Applied statistical brilliance made up for what data processing lacked in processing power, up to a point, because even heavy lifting statistics requires computing horsepower, which in those days was really a question of serious capital expenditure, which not all companies were willing to commit to.

Then, and curiously coincidentally, people around the world started to posit the need for using data and information to address significant business challenges, to act as input into the processes of strategy formulation, choice and execution. Reports would no longer just be for the Financial Directors or the paper collectors, but would support serious business decision making.

Many initiatives sprang up to meet the top-level decision-making data requirements; they were invariably expensive attempts, with variable outcomes. Some approaches were quite successful, but far too many failed, until the advent of Data Warehousing.

Back then, most of the data that could potentially aid decision-making was in operational systems. Both an advantage and a problem. Data in operational systems was like having data in gaol. Getting data into operational systems was relatively easy, getting it out and moving it around was a nightmare. However, one of the advantages of operational data is that it was generally stored in a structured format, even if data quality was frequently of a dubious nature, and ideas such as subject orientation and integration were far from being widespread.

Of course, data also came in from external sources, but usually via operational databases as well. An example of such data is instrument pricing in financial services.

Therefore, briefly, a lot of Data Warehousing started as a means to provide data to support strategic decision-making. Data Warehousing ways not about counting cakes, widgets or people, which was the purview of operational reporting, or to measure sentiment, likes or mouse behaviour, but to assist senior executives, address the significant business challenges of the day.

Who’s your Daddy?

Bill Inmon, the father of Data Warehousing, defines it as being “a subject-oriented, integrated, time-variant and non-volatile collection of data in support of management’s decision making process.”

Subject Oriented: The data in the Data Warehouse is organised conceptually (the big canvas), logically (detailing the big picture and) and physically (detailing how it is implemented) by subjects of interest to the business, such as customer and product.

The thing to remember about subject areas is that they are not created ad-hoc by IT according to the sentiments of the time, e.g. during requirements gathering, but through a deeper understanding of the business, its processes and its pertinent business subject areas.

Integrated: All data entering the data warehouse is subject to normalisation and integration rules and constraints to ensure that the data stored is consistently and contextually unambiguous.

Time Variant:  Time variance gives us the ability to view and contrast data from multiple viewpoints over time. It is an essential element in the organisation of data within the data warehouse and dependent data marts.

Non-Volatile:  The data warehouse represents structured and consistent snapshots of business data over time. Once a data snapshot is established, it is rarely if ever modified.

Management Decision Making: This is the principal focus of Data Warehousing, although Data Warehouses have secondary uses, such as complementing operational reporting and analysis.

In plain language, if what your business has or is planning to have does not fully satisfy the Inmon criteria then it probably is not a Data Warehouse, but another form of data-store.

The thing to remember about informed management decision making is that it needs to be as good as required but it does not need to achieve technical perfection. This observation underlies the fact that Data Warehouse is a business process, and not an obsessive search for zero defects or the application of so called ‘leading edge’ technologies – faddish, appropriate or not.

JOIN THE BIG DATA CONTRARIANS: http://www.linkedin.com/grp/home?gid=8338976

Some Basic Terms

Before we delve into the meaning of Data Warehousing, there are a couple of terms that need to be understood first, so, by way of illustration:

Let’s follow the numbers in the simplification of the process.

  1. We gather specific and well-bound data requirements from a specific business area. These are requirements by talking to business people and in understanding their requirements from a business as well as a data sourcing and data logistics perspective. Here we must remember at all times not to over-promise or to set expectations too high. Be modest.
  2. These business requirements are typically captured in a dimensional data model and supporting documentation. Remember that all requirements are subject to revision at a later data, usually in a subsequent iteration of a requirements gathering to implementation cycle.
  3. We identify the best source(s) for the required data and we record basic technical, management and quality details. We ensure that we can provide data to the quality required. Note that data quality does not mean perfection but data to the required quality tolerance levels.
  4. Data Warehouse data models modified as required to accommodate any new data at the atomic level.
  5. We define, document and produce the means (ETL) for getting data from the source and into the target Data Warehouse. Here we also pay especial attention to the four characteristics of Data Warehousing. ETL is an acronym for Extract (the data from source / staging), Transform (the data, making it subject oriented, integrated, and time-variant) and Load (the data into the Data Warehouse and Data Mart).
  6. We define, document and produce the means for getting data from the Data Warehouse into the Data Mart. In short, a bit more ETL.
  7. User acceptance testing. NB Users must ideally be involved in all parts of the end-to-end process that involves business requirements, participation and validation.

This is a very simplified view, but it serves to convey the fundamental chain of events. The most important aspect being that we start (1) and end (7) with the user, and we fully involve them in the non-technical aspects of the process.

JOIN THE BIG DATA CONTRARIANS: http://www.linkedin.com/grp/home?gid=8338976

Business, Enterprise and Technology

Essentially, a Data Warehouse is a business driven, enterprise centric and technology based solution for continual quality improvement in the sourcing, integration, packaging and delivery of data for strategic, tactical and operational modelling, reporting, visualisation and decision-making.

Business Driven

A data warehouse is business centric and nothing happens unless there is a business imperative for doing so. This means that there is no second-guessing the data requirements of the business users, and every piece of data in the data warehouse should be traceable to a tangible business requirement. This tangible business requirement is usually a departmental or process specific dimensional data model produced together in requirements workshops with the business. We build the Data Warehouse over time in iterative steps, based on the criteria that the requirements should be small enough to be delivered in a short timeframe and large enough to be significant.

Typically, a Data Warehouse iteration results in a new Data Mart or the revision of an existing Data Mart.

Enterprise Centric

As we build up the collection of Data Marts, we are also building up the central logical store of data known as the Enterprise Data Warehouse that serves as a structured, coherent and cohesive central clearing area for data that supports enterprise decision making. Therefore, whilst we are addressing specific departmental and process requirements through Data Marts we are also building up an overall view of the enterprise data.

Technology Based

By technology, I mean technology in the broadest sense of techniques, methods, processes and tools, and not just a question of products, brands or badges.

Unfortunately, there is a popular misconception that Data Warehousing is primarily about competing popular and commercial available technology products. It isn’t, but they do play an important role.

Architecture

The following is an example of a very high-level Data Warehouse architecture diagram.

Methodologies

Various methodologies support the building, expansion and maintenance of a Data Warehouse. Here is one example of a professional data integration methodology, produced, maintained and used by Cambriano Energy.

And here is an information value-chain map as used by Cambriano Energy as part of its Iter8 process management. There are alternatives, many of which do a satisfactory job.

Last but not least, this was (from memory) the way that Bill Inmon’s Prism Solutions ETL company used to view the iterative EDW building process.

JOIN THE BIG DATA CONTRARIANS: http://www.linkedin.com/grp/home?gid=8338976

Keeping it Shortish

At this point, I decided to cut short further explanations on aspects on Data Warehousing. However, if you have any question then please address them to me and I will do my best (or something close) to answer them.

That’s all folks

Hold this thought for another time: If you think you can replace a Data Warehouse, that is not a Data Warehouse, with another approach to ‘Data Warehousing’ that doesn’t produce a Data Warehouse, for as fast and cheap as one can do it, then you still don’t have a Data Warehouse to show for all of your efforts. That is not a great place to be.

Therefore, you see, Data Warehousing was never about a haphazard approach to providing random structured, semi-structured and unstructured data of various qualities, provenance, volumes, varieties and velocities, to whomever was of a mind to want it.

Many thanks for reading.

 If you want to connect then please send a request. I you have any questions or comments then fire them off below. Cheers :-)

Oh… and one last thing before I go… DON’T FORGET TO JOIN THE BIG DATA CONTRARIANS: http://www.linkedin.com/grp/home?gid=8338976

 

The Big Data Contrarians – New Big Data Community

03 Friday Jul 2015

Posted by in Big Data, community, Good Strat, goodstart, goodstrat

≈ Leave a comment

Tags

, , , ,


Friends, peers and colleagues, lend me your bandwidth and 10 minutes of your time.  Gather around and let me tell you about the greatest, most interesting and fantastically diverse Big Data and Data community right here in our very midst on this amazing LinkedIn community.

We have a new Big Data/Data group, and the group is aptly named The Big Data Contrarians, and yet it is neither a ‘me too’ group, of which there are too many to mention, or a ‘belief circle’, of which the less said, the better. Not, The Big Data Contrarians group is a place for cool opinion pieces, creative abrasion, practical insight and (within the realms of the possible) BS free comment.

However, before going into more detail about the group, I would like to digress for a moment.

Like many people, I take a lot of inspiration from outside my own professional spheres of practice, principles and technologies, and this is no less true when it comes to advertising.

Two of my real influencers – the real kind not the LinkedIn kind – are advertising legends Dave Trott (also author of Predatory Thinking) and Bob Hoffman (the Ad Contrarian), who are exceptionally experienced, talented and creative people, of the NoBS (no flim-flam) kind. Indeed, it was after reading some of Bob’s and Dave’s recent articles that I decided to get this group registered on LinkedIn, which, love it or loath it, is where many of us connect.

So, I hear you ask “What’s The Big Data Contrarians, Mart?”

Okay, to be fair, The Big Data Contrarians group is about far more than just being contrarian and a legitimate means of inciting discussion, for as reasonable as that is. It’s also about arguing against or openly rejecting mistakenly cherished and contrived Big Data beliefs and ‘institutions’ and established Big Data hype, speculation and opinion. It’s about separating Big Data fads, fantasises and folk-tales from Big Data reality.

What we seek to understand and convey is where, when, how and for what ends data (including Big Data) can be used to derive legitimate benefits. Moreover, stated from a position of reason and facts, and not simply projected as an issue of Big Data faith, speculation and clairvoyance.

On the other side, we can call out the Big Data hype for what it is, and just as Bob Hoffman calls out the social media and advertising BS babblers in his trade, this too lends a platform for people to do the same with the disreputable and dubious practices of Data gurus, courtesans and ‘influencers’.

“So, Mart, is being a Big Data Contrarian a bit like being a Big Data Luddite?”

Well, not really, but the problem with having so many people who are new to IT is that the past is a mystery top them, so anything that is new to them is actually taken as new, whether it is new or not.

Those who know will know that technologies of distributed file stores and search over unstructured data has been around for quite some time, and some of the “new” technologies that we big-up today, are actually simple developments of data technologies that go back to the seventies and eighties, or maybe even before.

However, this is not essentially about being anti-technology or even in advances in the application of technology, but of understanding that it isn’t helpful for the media, the big industry players and their indentured acolytes, to railroad, cajole and bully businesses into buying Big Data technology they don’t need, to solve Big Data problems and opportunities they don’t have.

That said, it’s up to the members of The Big Data Contrarians to decide on what shape the community should take, and as it is an open forum in democratic terms, the members have equal rights in presenting their own opinions, lessons learned and other insights.

So, if you haven’t yet drunk the Big Data kool-aid, come on down to The Big Data Contrarians, the place for everyone interested in Big Data/Data and its many potential uses.

Many thanks for reading.

Of course, this piece will also not feature on LinkedIn’s Big Data channel, because apparently that channel editor (naming no names) doesn’t like anyone raining on their particular Big Data flim-flam parade.

#BigDataChannel

Let’s talk strat! Business Strategy and IT

22 Friday May 2015

Posted by in business strategy, Consider this, Good Strat, goodstrat, IT strategy, Strategy

≈ Leave a comment

Tags

, , ,


I used to work for an affable person from Chicago. His two favourite phrases were “Let’s talk strat” and “Brought your cheque book with you?”

There are many misconceptions about strategy. But, I particularly want to address two things:

  • What is business strategy?
  • What is IT (information technology) strategy?

So, without more ado, let’s get the baby off the ground.

Continue reading →