Category Archives: goodstart

Data Supply Framework 3.0 – ETL Patterns

26 Thursday Jan 2017

Posted by in 4th generation Data Warehousing, All Data, Big Data, Big Data 7s, Big Data Analytics, business strategy, dark data, data architecture, Data governance, Data Lake, Data Supply Framework, Extract, Good Strategy, goodstart, governance, Information Management, Information Supply Frameowrk, Information Supply Framework, IT strategy, Strategy, The Big Data Contrarians

≈ Leave a comment

Martyn Richard Jones

Mountain View, 22nd January 2015

image3This article is the first in a series of articles that discuss aspects of the use of architectural patterns in the Cambriano Information Supply Framework 3.0

The term architectural pattern may sound grand, misleading or daunting, but it’s really quite a simple concept. It’s like writing a function in a programming language to log in to a database, check that the connection is alive and working and report back the success of the connection request. If that function can be reused either in the same application development, in the same IT shop or in IT in general (e.g. Java code to connect and test the connection to SQL Server) then it’s well on its way to becoming an architectural pattern. Of course, there are much more sophisticated architectural patterns. But generally a pattern is a simplified and generic template for address a generally occurring problem. But as with much in architecture, less usually turns out to be more.

Continue reading →

Seven Magnificent Big Data Success Stories

31 Wednesday Aug 2016

Posted by in 4th generation Data Warehousing, All Data, Ask Martyn, Big Data, Big Data 7s, Big Data Analytics, Cambriano, dark data, data architecture, Data governance, Data Lake, data management, data science, Data Supply Framework, Data Warehouse, Data Warehousing, good start, Good Strat, Good Strategy, goodstart, goodstartegy, goodstrat, Martyn does, Martyn Jones, Martyn Richard Jones, pig data, The Amazing Big Data Challenge, The Big Data Contrarians

≈ Leave a comment

Mount_Everest_as_seen_from_Drukair2_PLW_editMartyn Richard Jones

Lora del Rio, 31st August 2016

Big data has arrived. Big Data is here for keeps. Big Data is the future.

Despite some of the malicious, mendacious and malodorous words of naysayers, sceptics and contrarians, the world of big data and big data analytics is replete with totally amazing and fabulous success stories.

Big Data gurus are often accused of not delivering coherent, cohesive and verifiable accounts of Big Data successes. Which is understandable but at the same time a pity. So here, to illustrate this miraculous and remarkable turnaround, I give you not three but seven of the many Big Data success stories that I could have casually grabbed out of the ether.

First, we take a trip to Glasgow to discover the leveraging of Big Data in alternative investments. Then we pass over to Boston to explore the magic of Big Data at Universal Legal. We venture through Switzerland and innovative marketing. Explore the heights of Dongalong Creek. Have a word with the good folks at Heisenberg Labs. Then round it off with a quick in-depth summary of Big Data at Choppers. So, here we go…

Continue reading →

Big Data on the Roof of the World

29 Monday Aug 2016

Posted by in 4th generation Data Warehousing, All Data, Ask Martyn, Big Data, Big Data 7s, Big Data Analytics, business, Business Intelligence, business strategy, dark data, data architecture, Data governance, Data Lake, data management, good start, Good Strat, Good Strategy, goodstart, goodstartegy, goodstrat, Martyn does, Martyn Jones, Martyn Richard Jones

≈ Leave a comment

Mount_Everest_as_seen_from_Drukair2_PLW_editOnce upon a time, there was a mountain known as Peak 15. Very little was known about it. Then in 1852, surveyors found it was the highest in the world, and they named it Everest.

As with other significant challenges that we can identify in life, many people have been driven by a passionate desire to conquer peaks all around the world. This is just one illustration of those of us who can identify their significant challenges and rise to them. This sharp focus, determination and courage turns ordinary citizens into people who are invariably on a mission. People who know what they want. Continue reading →

Stories from the Data Warehousing front-line

21 Sunday Feb 2016

Posted by in 4th generation Data Warehousing, Big Data, Big Data 7s, Data Warehouse, Data Warehousing, Good Strat, Good Strategy, goodstart, goodstrat, Information Management, Information Supply Frameowrk, Information Supply Framework, Inmon, Martyn Jones, Martyn Richard Jones, Strategy, The Big Data Contrarians

≈ Leave a comment

NB THIS IS FICTION

All characters appearing in this work are fictitious. Any resemblance to real persons, living or dead, is purely coincidental.

Data warehousing, what is she like?

Although the answers are probably obvious, and to be honest, compared to the Big Data hype-circus this is a walk in the park, I have often wondered why Data Warehousing attracts such a surfeit of lazy, socially inept and shallow-thinking chancers.

I could go on about this at length, about how I convened a meeting recently (held on the outskirts of Bornheim, a small town in Germany ) to discuss how to move rapidly forward with a new strategic data-warehousing project, and how, whilst putting aside the crass impertinence and barely-disguised arrogance of my guests, I was still amazed by the unabashed and brazen snow-job that I was subjected to.

I imagined that this was a deliberate tactic used in the craven hope that I would be overcome by the depth and breadth of their ‘inside knowledge’, and would consent to having my workshop hijacked and reframed

But, I wasn’t having any of that. I know bluff and bluster when I see it, and as a reformed bullshitter I will not willingly accept bullshit from anyone else.

So, as their bullshit came in fast and furious I started making notes, and thinking of the most adequate response that a Project Manager could make in the circumstances, but I soon tired of note taking and was rapidly becoming irritated by a total lack of empathy and an utter lack of engagement.

Irritated as I was, I still tried putting things back on the rails. Therefore, I continued to be as engaging and constructive as one should, whilst internally suppressing the urge to ask ‘what the feck is going on here?’ So, I talked about lifting, shifting and dropping a legacy data warehouse and marts from one box to another – thinking that this would be the minimum that Data Warehouse experts could engage with – and the need to get estimates of the effort required to do so (you know, things like roles and number of days, Big ballpark estimate stuff). The ‘Data Warehouse Architect’ and I use that term loosely, went off on a tangent. Vague, fuzzy and disjointed. The architect threw in some nonsensical vagaries about the need for Master Data Management to be an integral component of any future data warehouse. I half-managed to avoid the incredulous Jeremy Paxman look of ‘what on earth are you talking about?’ just as the gathered augmented the assault on MDM with a call to Information Lifecycle arms. Therefore, when things were becoming even weirder, the weird turned pro, with the train kicked off the tracks, rolled down the hill, and then set on fire, by so-called professionals, passing themselves off as supine yobs, and reciting, in close harmony, “Proof of Concept! Proof of Concept! What about the f****** proof of Concept! Uh? Uh? Uh?”

Well, well, well… what a way to run a dance hall!

We were opening up all technological fronts, apart from the ones that would actually be relevant. I felt like a PG Tips chimp getting bananas and cups of tea thrown in their general direction. I was Martyn, the Project Chimp, plaintively calling out “‘ere mate, do you know this is not the way to do Data Warehousing”, and half-expecting a response along the lines of “you plan it, Son, we’ll muddle along “. I didn’t get a response, all I got was what looked like the human equivalent (if there is such a thing) of a page-fault, a glazing-over of the eyes and a rapid reboot into full-on bullshit mode.

I could go on and on about this all day, but I would rather not. Just the day in the life of a PM tasked with getting sense, sensibility, and work out of profane variations on the theme of Blackadder’s stupid Prince George. “I don’t need Inmon or Kimball, I know data! And… I have been to Ikea!” Sorry, that was just an example of how utterly obtuse things can get on the front-line of Data Warehousing.

So, to close, I would like to pose a question, one that goes beyond Data Warehousing and Big Data. Do people have the same or similar issues in other parts of IT or indeed in other businesses and technical related activities?

Bamberg

22nd September 2012

 

The Big Data Contrarians at 1000

07 Monday Dec 2015

Posted by in Big Data, Cambriano, Consider this, Good Strategy, goodstart, goodstrat, Martyn Jones, The Big Data Contrarians, Uncategorized

≈ Leave a comment

Tags

,

First things first. The Big Data Contrarians (“a hype free Agora for Big Data dialogue”) is now a community of over one thousand professionals.

Since its LinkedIn group registration on the 1st of July 2015, the Big Data Contrarians has grown to become, without a shadow of doubt, the nicest, friendliest and most well informed Big Data group that you will ever come across in your entire life.

The Big Data Contrarians is a community of professionals who enjoy talking about data, statistics, analytics, data-centric applications, ideas, opinions and insights.

The Big Data Contrarians is a great place to contrast ideas about data. It is a group that passively encourages discourse. Especially discourse that comes with a touch of humour, a hint of disbelief and a delightful bouquet of subtle cynicism.

Also, no data, analytics or visualisation related subject, for as tenuous as the relationship might be, is out of bounds. This is a forum by professional adults for professional adults, with all its attendant facets and all that this implies. Indeed, who knows what the next topic of conversation will be on The Big Data Contrarians forum. But here’s some ideas:

  • We may call it Big Data, Smart Data or Small data, but in reality isn’t the only intrinsic quality of data is in its being and in its symbolism, if indeed it has any?
  • If data were a religion would Big Data be a craven image, a sect or a schism?
  • Does ascribing qualities to data, such as Big, Small or Smart, places us at risk of outdoing the degrees of anthropomorphism of some pet lovers?
  • To be a Big Data guru, is it necessary to know the difference between Hadoop and Spark?
  • Did Big Data hype fall off the radar because it’s gone, or did Big Data hype turn ‘pro’?
  • How do we measure the qualitative and quantitative value of data?
  • Do we really need The Big Data Contrarians community?

After four weeks of the group’s existence, I wrote a piece for Data Science Central (July 23rd, 2015), in which I itemised some of the reasons why I believed that The Big Data Contrarians groups was necessary. Those reasons were:

  1. To alert people to interesting but ultimately dubious Big Data claims
  2. To share lessons learned, good sense and practical data and Big Data principles
  3. To connect professionals in overlapping disciplines, for example, in statistics, data architecture and data management, project management, solutions architecture, database administration, data science, risk management, technical, management and executive management roles, and a long list of etceteras.
  4. To educate, inform and entertain each other about the practical world of data and Big Data
  5. To weigh up the pros and cons of data technologies and techniques applied to solving business challenges
  6. To minimise the hype surrounding Big Data, covered in part by the first item, but more so
  7. To engender critical thinking, healthy scepticism and reasoned contrarianism

I genuinely believe that those reasons are as valid now as they were then. Perhaps even more so.

So, to get back to basics, I will leave you where I started.

The Big Data Contrarians. Is quite simply the best Big Data community on the whole wild-wild-wild internet, anywhere? Yes, anywhere. The Big Data Contrarians is an amazingly great data community that promotes inclusivity, interaction and coherence in data, statistics, analytics and Big Data discourse.

Anyone who is anyone in Big Data and data is a member of THE BIG DATA CONTRARIANS, either now or in the near future.

So, don´t be left behind. Join today. Be the Big Data smarty amongst your Big Data party.

Many thanks for reading.

 

Join The Big Data Contrarians: https://www.linkedin.com/groups/8338976

Big Data, ESP and Transubstantiation

19 Wednesday Aug 2015

Posted by in Big Data, good start, goodstart, goodstrat

≈ Leave a comment

Tags

, , , , ,

vocationIf you enjoy this piece or find it useful then please consider joining The Big Data Contrarians:

Join The Big Data Contrarians here: https://www.linkedin.com/grp/home?gid=8338976

Many thanks.

To the layperson anxious for answers to complicated questions, the very idea of bringing together sets of disparate data and turning it into precious insights may seem like magic, a modern day alchemy, a goal placed well beyond the grasp of mere mortals. Fortunately, this is no longer the case, thanks in part to bagatelle-proportioned advances in Big Data and Big Data analytics and massive advances in imagination; we are able to look into the past, the present and the future, with absolute certainty. Continue reading →

Stuff a great data architect should know

16 Sunday Aug 2015

Posted by in Consider this, data architecture, goodstart, goodstartegy

≈ 2 Comments

Tags

, , , , , , ,

Stuff a really great Data Architect should know

If you enjoy this piece or find it useful then please consider joining The Big Data Contrarians:

Join The Big Data Contrarians here: https://www.linkedin.com/grp/home?gid=8338976

Many thanks. Continue reading →

Why so many ‘fake’ Big Data Gurus?

16 Sunday Aug 2015

Posted by in Big Data, Consider this, good start, goodstart, Strategy

≈ Leave a comment

Tags

, , , , , , , , ,

Why so many ‘fake’ Big Data Gurus?

Where do you all come from?

Where do you all come from?

All your integrity’s gone

Now tell me, where do you all come from?

From ‘Where Do You All Come From‘ by Mott the Hoople Continue reading →

Big Data, the promised land where ‘smart’ is the new doh!

03 Monday Aug 2015

Posted by in Big Data, Consider this, good start, goodstart, Martyn Jones, Strategy

≈ Leave a comment

Tags

, , , ,

If you enjoy this piece or find it useful then please consider joining The Big Data Contrarians:

Join The Big Data Contrarians here: https://www.linkedin.com/grp/home?gid=8338976

Many thanks.

So you want to ‘do’ Big Data

Now everyone is doing Big Data you don’t want to be the odd one out, right? Of course not.

Now, if you are serious about looking at Big Data from a business perspective then I will try and lend you some advice. If you are doing it from an IT or technology perspective, then I wish you good luck, and I hope that your Big Data initiative doesn’t turn into another tech crash-and-burn show.

Now some Big Data pros are telling us that the place to start with Big Data is with strategy. Now, I’m too polite to call this out as abject bullshit, even though it is, and will instead content myself by offering an alternative and simple approach to approaching and addressing Big Data.

My first piece of advice is this. DON’T START WITH STRATEGY!

Don’t start with Strategy

Strategy is a coherent, cohesive and executable response to a significant challenge.

Strategy is not a definition of objective, a wish list of what you are trying to achieve or aspirational goals of a nebulous nature. No, strategy is not the objective but a means of reaching that objective. Strategy is real, tangible and executable. Strategy is doing.

So what is a Big Data strategy?

If a company is looking at the Big Data options, the last place they should want to start out from is from strategy. That is as silly idea as they come. Starting with strategy on the road to formulating viable responses to significant challenges and opportunities is like saying that before we choose strategic options and a realisable strategy, then you must have a strategy in place.

Strategy is not working out what you want to achieve. That sort of thing should happen prior to any strategic work. Neither is strategy an exercise in establishing starting points, nor formulating questions nor understanding the challenges. All of this should come well before the major strategy aspects even kicks-in.

Big Data strategy is a realisable, tangible and manageable response to a significant challenge, one that depends heavily on the availability, usability and credibility of Big Data (or Very Large Data Bases) and the business value of processing that Big Data.

So, a word of advice. If you are thinking of embarking on a Big Data initiative, do not start with strategy. That is a really daft place to start.

Start with business imperatives

Start here instead. With real business imperatives. This is where you are thinking about the big and significant challenges to the business, and how, at a high level of abstraction, you could go about meeting those challenges. Here you identify your challenges and your responses, aligned to your objectives.

If you can identify business imperatives that make it absolutely necessary to include elements of Big Data, then go forward with that mandatory requirement in mind. If not, then don’t try to shoe-horn Big Data into a place where it really isn’t needed or wanted. Because if you go against the grain in this way it may well hurt you and your business, in more ways than you bargained for.

Know what you are looking for

In order to go out looking for data requirements driven by business imperatives, we really need to know what we are looking for.

What we are looking for maybe highly tangible or less so. We may have to derive the data we are looking for by refining, aggregating, enriching, filtering and cleansing. Therefore, with those and other aspects in mind, we can go out and find what we need.

How to find what you are looking for

From looking at the data requirements, you should have a good idea of potential sources of that data. Agility in this aspect is predicated on the premise that one knows the systems on the IT landscape, the business processes and all the potential sources of data – at a high level at least. So, this is not the sort of work you can do remotely with little or no knowledge of the clients business, IT setup, processes or culture.

But anyway, after you identify the sources you move on to the next step.

Check data availability

Here you discuss aspects of the data you require with the database / application platform owners to ensure that:

  1. they have the data you are looking for
  2. that quality of the data is known and data quality can be addressed
  3. that the data is relevant for what is needed
  4. that the cost of providing this data is not prohibitive
  5. that this data can be made available to you
  6. that service levels could be put in place, if and when required

So far so good. Once passed these hurdles (and don’t forget this is a super-simplification) we move in to the next.

Make proof of concepts

So, now we know:

  1. What data we need
  2. Where we can get it from
  3. How we get it
  4. What we need to do to make it usable
  5. How we need to analyse it

Therefore, we go ahead and create a proof of concept or three. Simples!

However, make sure that all prototypes are governed by these simple timeless guidelines:

  1. The proof of concept should be small enough to be doable in a reasonable time-frame. I would be rather generous for the very first pilot of its type in a company, but would set that limit at 90 days, tops.
  2. Make sure that the proof of concept is big enough to be significant. Again, ‘simple enough to be realisable’ and ‘large enough to be significant’, should go hand in hand.
  3. Arrange your proof of concept execution into sprints. So your 90 days may be made up of nine 10 day sprints.
  4. Don’t try and shoe-horn infrastructure aspects of your initiative into sprints, it just doesn’t work, and simply pisses people off.
  5. If a proof of concept looks like it will fail, then make sure it fails early. There’s nothing worse than having people insist on pushing a dead project to live the full length of its planned term. Failing early means that business doesn’t take a dim view of the pilot, and will be more open to new proof of concept initiatives.

Analyse the outcomes

You run your proof of concept. You analyse, assess and represent your outcomes. You socialise, present and interpret.

Revise your strategic outlook accordingly

When you’ve done that you are in now in a good position to estimate the usefulness of the exercise, from both a qualitative and quantitative perspective.

Did I mention technology?

I did not want to touch in specific aspects of technology in this piece, in part, because I did not consider it a central issue in the theme of things. Of course, as part of creating proofs of concepts and pilot schemes you may want to experiment with the swatch (swaith? oh for auto-correction) of technologies out there. So go ahead and evaluate ‘Big Data’ technologies, and don’t forget, the answer to every Big Data technology question isn’t an automatic ‘Hadoop’. There are other valid Big Data technology options around, such as Lustre and GPFS, or even Oracle, Teradata or EXASol. Also, remember this, if all you are working on is a prototype, a proof of concept or a pilot then you can try and negotiate a free license with any of the major DBMS vendors for that initiative. So negotiate, bargain and get the most appropriate technologies with the best deals.

That’s all folks

Finally I will leave you with three guidelines to consider:

  1. Don’t ask ‘how can I do Big Data?’ but ‘what data do we need?’
  2. You don’t need to seek out Big Data. If you really need it, and it’s available, and it’s adequate and appropriate, then you’ll be getting it soon enough.
  3. Avoid searching for a Big Data problem you don’t have, which can only be solved by Big Data technology you don’t need.

Many thanks for reading.

In subsequent blog pieces I will be sharing my views on the evolution of information management in general, and the incorporation novel and innovative techniques, technologies and methods into well architected mainstream information supply frameworks, for primarily strategic and tactical objectives.

As always, please reach out and share your questions, views and criticisms on this piece using the comment box below. I frequently write about strategy, organisational, leadership and information technology topics, trends and tendencies. You are more than welcome to keep up with my posts by clicking the ‘Follow’ link and perhaps you will even consider sending me a LinkedIn invite if you feel our data interests coincide. Also feel free to connect via Twitter, Facebook and the Cambriano Energy website.

For more on this and other topics, check out some of my other posts:

Absolutely Fabulous Big Data Roles – https://www.linkedin.com/pulse/absolutely-fabulous-big-data-roles-martyn-jones?trk=prof-post

Not banking on Big Data? – https://www.linkedin.com/pulse/banking-big-data-martyn-jones?trk=prof-post

10 amazing reasons to join The Big Data Contrarians –https://www.linkedin.com/pulse/10-amazing-reasons-join-big-data-contrarians-martyn-jones?trk=prof-post

Amazing Data Warehousing with Hadoop and Big Data –https://www.linkedin.com/pulse/cloudera-kimball-dw-building-disinformation-factory-martyn-jones?trk=prof-post

The Big Data Contrarians: The Agora for Big Data dialogue –https://www.linkedin.com/pulse/big-data-contrarians-agora-dialogue-martyn-jones?trk=mp-reader-card

The Big Data Shell Game – https://www.linkedin.com/pulse/big-data-shell-game-martyn-jones?trk=mp-reader-card

Aligning Data Warehousing and Big Data –https://www.linkedin.com/pulse/aligning-data-warehousing-big-martyn-jones?trk=mp-reader-card

Big Data Luddites – https://www.linkedin.com/pulse/big-data-luddites-martyn-jones?trk=mp-reader-card

Data Warehousing Explained to Big Data Friends –https://www.linkedin.com/pulse/data-warehousing-explained-big-friends-martyn-jones?trk=mp-reader-card

Big Data, a promised land where the Big Bucks grow –https://www.linkedin.com/pulse/big-data-promised-land-where-bucks-grow-martyn-jones-6023459994031177728?trk=mp-reader-card

The Big Data Contrarians – https://www.linkedin.com/pulse/big-data-contrarians-martyn-jones?trk=mp-reader-card

Is big data really for you? Things to consider before diving in –https://www.linkedin.com/pulse/big-data-really-you-things-consider-before-diving-martyn-jones?trk=mp-reader-card

Big Data Explained to My Grandchildren – https://www.linkedin.com/pulse/big-data-explained-my-grandchildren-martyn-jones?trk=mp-reader-card

If you enjoy this piece or find it useful then please consider joining The Big Data Contrarians:

Join The Big Data Contrarians here: https://www.linkedin.com/grp/home?gid=8338976

Many thanks.

Absolutely Fabulous Big Data Roles

03 Monday Aug 2015

Posted by in Big Data, Consider this, good start, goodstart, Martyn Jones, Strategy

≈ 1 Comment

Tags

, , , ,

Plus ça change, plus c’est la même chose.

Jean-Baptiste Alphonse Karr

Prologue

I wrote a piece called ‘7 New Big Data Roles for 2015′. I published it on LinkedIn. Many people read it. Some people made suggestions. Others politely ignored it.

I listened to the suggestions, comment and criticisms, and revised the piece as a result.

So here, it is… I hope you like it. And if not, I might try again in six months’ time. Continue reading →