Simply stated, the best application of Big Data is in systems and methods that will significantly reduce the data footprint.
Why would we want to reduce the data footprint?
- Years of knowledge and experience in information management strongly suggests that more data does not necessarily lead to better data.
- The more data there is to generate, move and manage, the greater the development and administrative overheads.
- The more data we generate, store, replicate, move and transform, the bigger the data, energy and carbon footprints will become.
“Anger is the enemy of nonviolence and pride is a monster that swallows it up.”
The predominance of strength and innocence, better known as ignorance and arrogance, is undermining Information Management, and in turn is ensuring that many Data Warehousing and Decision Support initiatives are disappointments.
2015 will again give IM professionals the opportunity to regain some dignity and professional integrity.
First, by recognizing that there are grave problems within IM; then slowing down and halting the toxic trends, carelessness and bad practices; and then in subsequently, reversing, through intelligence, perseverance and integrity, the ingenuous and decrepit habits that still trouble the profession.
In the rush to the bottom we throw excellence in analysis, architecture, engineering and business understanding, under the bus. In IM as well as in many other branches of IT (Information Technology), mediocrity has become the new excellent, regular the new exceptional, and shoddiness the new normal.
Whether it is in Data Warehousing, Big Data, Business Intelligence, Analytics, Decision Support or Data Integration, we see that professional integrity and ethical behaviour – already enough of a rarity in IT – is being repeatedly trumped by short-term expediency, wilful witlessness, and the cultivation and perpetuation of dogmas, dysfunctional behaviour and dubious doings.
The Information Management sector is rife with elaborate charlatanry, partisan expediency and wilful self-deception. There is not a day that goes by in which we are not submitted to an avalanche of contemptible claims from rogue IM evangelists, DW neophytes and unsophisticated opportunists, who chose to simply make things up as they go along.
It is in the best interests of IM to raise the profession out of the ditch; to reform the profession from the inside; to drive sea-change improvements in knowledge, quality and professional integrity; to ensure a drastic reduction in destructive hype, deception and dogma, and, to show the artless charlatans, chancers and snake-oil merchants the door.
Data Warehousing and Decision Support – if done right, and for the right reasons – can deliver tangible benefits to many organisations. Simply stated, if business information has a value in the realm of business and strategy then it should be treated as an asset, if it is an asset then it should be managed and nurtured as such, which means aiming to do the right thing right, first time, every time, whilst focusing on maximising confidence, availability and agility.
File under: Good Strat, Good Strategy, Martyn Richard Jones, Martyn Jones, Cambriano Energy, Iniciativa Consulting, Iniciativa para Data Warehouse, Tiki Taka Pro
The matching takes place through the practice of pre-strategy analysis.
That stated, it is very easy to fall into the trap of simplifying the high level concepts and overstating the intricacies and interdependence of strategic-fit factors. Continue reading
My sister Liz was part of a group that offered support to the striking miners of Wales, Scotland and England.
They organized a public fund raiser and invited the politician Tony Benn to speak.
The trouble was that none of the support group were Labour people, and they weren’t the greatest admirers of British parliamentary democracy and the Labour party.
So they sort of moved the problem up-stream.
They asked me if I would be Tony’s minder for the night.
They didn’t actually use the word minder, but that what it was mainly about.
Because they probably reckoned that as a long time Labour member myself with an unquestioning belief in Westminster democracy, we might actually be able to talk the same language.
I had dinner with Tony that day, just before he was due to speak.
The conversation came around to Tony’s book, Arguments for Democracy.
Well, actually I had pushed the conversation in that direction.
I mentioned that I had read it at least three times, and that I used some of the examples from the book in my work.
In particular the part dealing with the questions that an elected politician and Minister of State must ask any technologist who is proposing a new projects or programme.
I told him that I had applied these principles in a large US multi-national corporation called Sperry, notorious for its Republican hue, its affinity to the Department of Defense, intelligence agencies and Federal Government, and its alleged hire and fire culture – which somehow I managed to evade for almost thirteen years.
He found that quite funny, in a surreal way.
I said “over the last eighteen years I have often used the following questions, which you designed to indicate that the role of the elected representative and minister is not to seek to reproduce the expertise, which he or she could not do, but to see that the expert is subjected to rigorous cross-examination on behalf of the people”.
Anyway, to cut a long story short, I shall now move on to the crux of the matter.
But before then, a final comment.
Because I have been using these lines of enquiry primarily in business I have replaced the role of the government minister with that of the Project Board and project stakeholders, and the role of the “people” with the role of the organisational stakeholders and the business community.
So here you have it. The leader must ask the technologist:
First, would your project, if carried through, promise benefits to the organisation, and if so, what are the benefits, how will they be distributed and to whom and when will they accrue?
Second, what disadvantages would you expect might flow from your work? Who would experience them? What, if any, remedies would correct them? Is the technology for correcting them sufficiently advanced for the remedies to be available when the disadvantages begin to accrue?
Third, what demands would the development of your project make upon our resources of skilled manpower [I would include demands on all organizational resources in this context, and would also ensure to enquire about the availability of those resources]
Fourth, is there a cheaper, a simpler, and a less sophisticated way of achieving at least part of your objective and if so, what would it be, and what proportion of your total objective would have to be sacrificed if we adopted it?
Fifth, what new skills would have to be acquired by people who would be called upon to use the product or project which you are recommending, and how could these skills be created?
Sixth, what skills would be rendered obsolete by the development that your propose, and how serious a problem would the obsolescence of these skills create for the people who have them?
Seventh, is the work upon which you are engaged being done, or has it been done, or has it been started and stopped in other parts of the world, and what experience is available from abroad [elsewhere] that might help us to assess your own proposal?
Eighth, if what you propose is not done, what disadvantages or penalties do you believe will accrue to the organisation, and what alternative projects might be considered
Ninth, if your proposition is accepted, what other work in the form of supporting systems should be set in hand simultaneously, either to cope with the consequences or to prepare for the next stage and what would the next stage be?
Tenth, a final and very important question. If an initial decision to proceed is made, how long will the option to stop remain open, and how reversible will this decision be at progressive stages beyond there?
Later that evening I had to drive Tony to the station to catch his train to Oxford.
We were late, it looked like we would miss the train.
In the car I asked Tony if he would care to sign my copy of Arguments for Democracy.
Trouble was, in the rush he didn’t pick up anything to read on the train and he hadn’t brought anything with him.
So I gave him The Chomsky Reader. Which just happened to be on the back seat of the car.
As one would.
Anyway, off we rushed. Hell for leather through the empty streets of Worcester.
We arrived at the station in time to catch the train.