Monthly Archives: May 2015

Let’s talk strat! Business Strategy and IT

22 Friday May 2015

Posted by in business strategy, Consider this, Good Strat, goodstrat, IT strategy, Strategy

≈ Leave a comment

Tags

, , ,

I used to work for an affable person from Chicago. His two favourite phrases were “Let’s talk strat” and “Brought your cheque book with you?”

There are many misconceptions about strategy. But, I particularly want to address two things:

  • What is business strategy?
  • What is IT (information technology) strategy?

So, without more ado, let’s get the baby off the ground.

Business Strategy

So what is business strategy? Here is a very brief description that I cooked up earlier for the Twitter page of goodstrat.com:

“Good strategy is a cohesive and coherent response to a significant challenge.”

Simples!

Strategy is about designing and, more importantly, executing a deliberate plan to respond to a substantial and manifest opportunity. Good strategy is not a PowerPoint slide deck, Big Data or the casting of sortes.  It should not be complicated, obtuse, or impractical.  It’s not a wish list, or a warm and fuzzy expression of faith or a muddled concoction of conviction, discrimination and bias. That is what people come up with when they do not know what they are doing, when they want to dazzle you with side-lining flim-flam or are simply winging it.

Good strategy is practical, effective and doable.

IT at the Business Strategy table

Should IT be at the strategy table?

Yes, always. Almost everything of significance that happens in modern business includes some element of ICT (information and communications technology). So, yes. Make sure that IT is in the ‘strat’ meetings, all of them. But, don’t use it as a ploy to get an army of IT related CXOs into the C-Suite. That attitude makes IT look dopey, detached and otherworldly. So get one or two people – preferably the CIO and the CTO – into that setting and make sure they can listen, understand and be proactive.

When putting together a cohesive and coherent response to a significant challenge the business must understand what the options are, and what these options mean in terms of risk, reality and responsibility, as well as cost, quality and time and place utilities.

Not only is IT part of the business strategy formulation but IT in turn treats the business strategy as their own “significant challenge”, which may drive the need for a separate IT strategy.

Do we need an IT Strategy?

Of course, if IT is a big part of your business, then you will need an IT strategy. In fact, have more than one. It’s always good to have a Plan B and it’s always good to have an alternative strategy, just in case.

However, do we need to come up with a new IT strategy every time the business adopts a new business strategy?

  • Well, it depends. If the new business strategy does not represent a significant challenge to IT, and is accommodated within the present ‘business as usual’ strategy, then it does not require a new IT strategy.
  • A business strategy that requires some small changes to operational systems also does not call for a new IT strategy.
  • Conversely, anything that requires a significant response from IT, must also be accompanied by a coherent and cohesive IT strategy.

Strategy is about action, if you are not planning to do anything significantly different then you do not need a strategy.

I know that some people will see this as overstating the obviously obvious, but believe me, not many people get this right, even with the ‘help’ of strategy consulting houses and other syndicated poster-makers. Especially in businesses where conformity to preconceived ideas and cracker-barrel business philosophy, whether correct or not, is encouraged.

Difficulties

There are times when it is difficult to come up with an adequate IT strategy:

  • The business might not have a formalised strategy.
  • The business might not have a clue about strategy formulation.
  • There is a strategy, but it is rubbish, unusable or meaningless.
  • Business leaders can become uncomfortable when asked about their business strategy.
  • IT has a lousy reputation of understanding, or even wanting to know anything about the business strategy, or indeed anything about the business itself, beyond that required to analyse, design, code, test and implement applications and databases.

However, that is no excuse for not knowing how to come up with a good strategy and then doing ‘great strategizing’.

Be prepared

Ideally, IT is business driven, market focussed and technology based, and what IT does, especially in terms of business strategy, should be adequate, appropriate and timely. But, that requires continuous preparation.

For IT to be strategic effective, it must provide:

  • Business, technology and resource alignment and advantage
  • Enabling technology – not just products, but tech stack mixes, and who is doing what and with what
  • Business knowledge – knowledge not just of The Business but also of similar business
  • Business development skills – IT got this partly right when it started to appoint customer relationship managers, but they got it very wrong by hiring the wrong people for the task

For example, let’s take a look at an idealistic view of the ‘marketing driven’ Information Management organisation (based on Iniciativa’s Knowledge Asset Management Organisation) as a separate entity but adjunct to IT. Such an organisation could be used for empowering business competitiveness through the provision of adequate, appropriate and timely information … driven by the needs of the business… through continual improvements in knowledge and experience… by focusing on delivery of service excellence in Information Management… and by continually negotiating business needs and priorities.

That’s all folks

Bottom line. IT should be helping to shape strategy, tactics and operational awareness, every step of the way; it should be proactive in identifying opportunities, designing options and selling (yes, the real selling of) coherent, cohesive and doable solutions. Solutions that offer tangible business benefits.

To reiterate, a good IT strategy is “a cohesive and coherent response to a significant challenge.”

So let’s talk strat!

Thanks very much for reading. As always, if you have any questions, criticisms or tips then please post it below and I will try my best to answer.

#Strategy #BusinessStrategy #ITStrategy #GoodStrategy #InformationTechnology

Advertisements

The Hadoop Honeymoon is Over

16 Saturday May 2015

Posted by in Big Data, Consider this, Good Strat, Good Strategy, Martyn Richard Jones, Strategy

≈ 5 Comments

Tags

, , ,

Listen up Big Data playmates! The ubiquitous Big Data gurus, tied up in their regular chores of astroturfing mega-volumes, velocities and varieties of superficial flim flam, may not have noticed this, but, Hadoop is getting set up for one mighty fall – or a fast-tracked and vertiginous black run descent. Why do I say that? Well, let’s check the market.

According to Gartner there is “continuing enthusiasm for the big data phenomenon”. However, “demand for Hadoop specifically is not accelerating.” So what’s up doc?

The Hadoop companies have been spending big-time on marketing, and I mean big, and lo and behold, the sales are dipping as fast as the prospects evaporate. You might even have noticed this. Of course, this can’t go on forever. You cannot maximise your marketing spend when your revenues are heading south, and not even the most naïve of angels or other types of investors will keep up with misguided corporate-welfare for too long. Even if you’re flogging a moribund yellow elephant, you make bank or you die.

So what went wrong?

First of all the Hadoop people got well ahead of themselves. Their own enthusiasm for the toys they helped to ‘create’ got the better of them. But first, in order to understand the commercial problems it’s necessary to delve into some of the techie stuff.

Hadoop is a child of Agile and Agile is no way to design a solid, coherent and cohesive data management architecture, never mind build it.

Hadoop grown, feature by feature, as a response to specific technical challenges in specific and somewhat peculiar businesses. When it all kicked off, the developers weren’t thinking about creating a new generic data management architecture, one for handling massive amounts of data. They were thinking of how to solve specific problems. Then it rather got out of hand, and the piecemeal scope grew like topsy as did the multifarious ways to address the product backlog. At least that’s the impression I get.

At Sperry and then Unisys, I worked in database technology research and development for a number of years, and learned from the best and mentored some of the best. To be frank, if I had to design an architecture for handling massive data volumes, data velocities and data varieties, it would not look like Hadoop nor would it have the ‘features’ of Hadoop that are there because of a lack of a coherent up-front architecture or simply naiveté. Hadoop people, who thought they knew data architecture, actually did not, and they were therefore simply winging-it. I mean, what sort of eejit would deliberately design a comprehensive parallel data management system with only one master controller (namenode)? Duh!

What’s the take-away from that? Hadoop clearly demonstrates that Agile is not a good approach for all types of software development projects.

You don’t need to read pieces and comments on LinkedIn’s Big Data channel to know that some people, many people, think that Hadoop is the brand new shiny gizmo that will cure all existential threats, bring about world peace and fix the negative effects of climate change. Read items on the Big Data channel if you must – this piece probably won’t get featured there, but anyway. So there they are, the marketers, Big Data gurus, the chancers, PR people, vendors, management consultants, the wacky and waylaid, all of them, ‘bigging it up for Big Data’, up the wazoo. Clueless, almost to a man. They all proclaim the Emperor’s new distributed file system as the greatest thing since time-slicing my little pony, but they are so dumb, so enamoured with their new-best-fad, and so wilfully superficial that they don’t see or care that they are somewhat wrong.

Hadoop? It’s almost all about old technology, which has been repackaged, and frequently not very well. The only thing truly innovative about it is how a repackaging of Unix primitives on parallel platforms can be marketed in such imaginative, exaggerated and inaccurate ways.

Then things moved over to Skid Row when Big Data and Hadoop people started eating their own marketing dog food.

So, the sorry story goes… if Hadoop can be used for managing all data, all data volumes, all data velocities and all data varieties, it would be a small step to displace existing technologies in the Enterprise Data Warehousing and Business Intelligence space. It’s so cool, so powerful and so remarkable that it can even replace IMS, Teradata, DB2, SQL Server and Oracle, without skipping a beat. Right? Wrong!

The idea must have looked very attractive at the time, but the reality is that Hadoop is far too much like using parallel cat, grep, awk and cut (apologies for the techie talk). These Unix (and Linux) primitives have their uses, but Data Warehousing data-management is definitely not one of them. But, no one mentioned this mistaken conclusion – Oy! Mate! The Emperor has no clothes – and it helped considerably in inflating sales projections and the massive hype surrounding Hadoop and Big Data.

It also turns out that, in spite of the babbling of the usual suspects, Big Data is not for everyone, not everyone needs it, and even if some businesses benefit from analysing their data, they can do smaller Big Data using conventional rock-solid, high-performance and proven database technologies, well-architected and packaged technologies that are in wide use.

Put it this way, when I hear descriptions of how this revolutionary new Hadoop works, it’s a bit like a sophomore who has read up the previous night on the early architectures of Oracle, and is now recounting what they have read, badly, with the concepts and architecture turned into a potpourri of buzz words, incongruences and baloney.

And, you wonder why the corporate uptake of Hadoop is stalling?

Many thanks for reading.

lang: en_US

On Not Knowing Sentiment Analysis

12 Tuesday May 2015

Posted by in Big Data, Big Data Analytics, Consider this, good start, goodstart, sentiment analysis

≈ Leave a comment

Tags

, , , , , , , ,

If you know all about Sentiment Analysis, you’ve come to the right place. Because I don’t have a clue if what I know about it is accurate or not.

I started to do a bit research into this Sentiment Analysis lark, in particular with the theoretical idea of using it to analyse and draw conclusions from comments on Pulse – assuming that this is what it can be used for.

To begin at the beginning, which is good place to start, I read the piece on Wikipedia, and this was how it began:

“Sentiment analysis (also known as opinion mining) refers to the use of natural language processing, text analysis and computational linguistics to identify and extract subjective information in source materials.

Generally speaking, sentiment analysis aims to determine the attitude of a speaker or a writer with respect to some topic or the overall contextual polarity of a document. The attitude may be his or her judgment or evaluation (see appraisal theory), affective state (that is to say, the emotional state of the author when writing), or the intended emotional communication (that is to say, the emotional effect the author wishes to have on the reader).” Source: Wikipedia Link:http://en.wikipedia.org/wiki/Sentiment_analysis

Well, that’s a fairly intuitive description. I could have almost have guessed as much.

But, back to the aim of analysing sentiment in Pulse comments, where to start and what to do.

What would sentiment analysis make of these:

On the death of an IT-business celebrity. What would sentiment analysis make of the very emotive comments of desolation, sadness and poignancy of people who didn’t personally know the departed, even remotely, or maybe didn’t even know of them until after they had ‘shuffled off life’s mortal coil’? How would that work? What would sentiment analysis make of the maudlin aphorisms, surrogate grief and bizarre sorrow of people separated by more degrees than Kofi Anan and Mork from Ork.  What additional insight does sentiment analysis tell us when these comments are analysed along with the body of the text and other comments that triggers these comments?

In a similar vein, how does sentiment analysis catch instances of sycophancy? Especially considering the fact that some of it is so ‘in your face’ and blatant that it often times seems to be a bad parody of a bad parody. “Oh, Ricky, why are you such a sexy brainbox?” How does it work in those situations?

Worse than that is the preening, gushing and obtuse texts of massive, errm… fabulators[i]. If it wasn’t about Big Data or Strategy or IT, it would be about something else, usually about the writer themselves. “I give Rafa and Rodge tips on tennis! I went to the University of the Universe and got a first! I challenged Superman to a race, and won! I have read the entire works of Dan Brown, 25 times…Neeeh!” What would sentiment analysis do with that sort of gold?

Also, what does sentiment analysis do with texts so ambiguously daft that they could mean anything? Okay, it might be able to pick up a few trigger words here or there, “rubbish”, “of”, “load”, “a”, “what”, etc. However, how does it know when “excellent” is being used in a way that means anything but excellent? For example, “Excellent Big Data job there”, with the silent “if you want a job doing properly then do it yourself”.

Finally, for the purpose of this little piece, what would sentiment analysis do with term abuse, if it could actually identify it? Going back to the use of the terms such as Big Data or Strategy, how can sentiment analysis discern between the dopey and wrong-headed use of the term, and when it is actually being used in a coherent, cohesive and consistent way, in line more or less with its formal definition? I suppose we can always write a mountain of rules to help us out:

If topic in focus of piece is strategy

And context of topic is business

And author of piece is Richard Rumelt

Then the credibility of text is good (with a certainty of 100%)

But you and try and maintain a rule base with isntances like that. It soon becomes a management nightmare.

Alternatively, maybe it could be used to analyse this text. It’ll have its work cut out, that’s for sure. Does sentiment analysis do sarcasm and cynicsm?

Anyway! I bet you might know how this sentiment analysis works, don’t you? On the other hand, if not, then it will be someone else who ‘knows’. But of course, all will not be revealed, because it’s a secret so powerful, that in the wrong hands it could be used to dominate the entire galaxy.

Only joking; and many thanks for reading.

[i]To engage in the composition of fables or stories, especially those featuring a strong element of fantasy: “a land which … had given itself up to dreaming, to fabulating, to tale-telling” (Lawrence Durrell).

lang: en_US

Big Data Tales: Bernice and the Martians

12 Tuesday May 2015

Posted by in Big Data, Consider this, good start, goodstart, goostart, Martyn Jones

≈ Leave a comment

Tags

, , ,

Bernice and the Martians, BATM for short, were an incredibly popular progressive-rock band.

Their first big commercial success came with the release of their first album and their planned promotional tour, which took in all continents.

The manager of the band was none other than effable polymath, Renaissance man and good all-round rogue, Ricky Jonesy – an obsessive control freak, lover of fine wines and darling of predictive analytics. He really loved his numbers, his social media and his sentiment analysis.

In fact, much of the early success BATM came about due to Ricky’s unparalleled passion for the ‘Big Data’.

Ricky was the band’s architect. He had major input into their material: what they composed; how they composed; their stage sets and lighting; where they performed; the way they played; how they dressed; were photographed; spoke; walked; and, ate and drank. In short, he controlled the whole BATM enchilada. It was like being in data-driven heaven.

As I said, their first album, a progressive-rock masterpiece called ‘Your Hole’, achieved major critical acclaim even before it was bolting out of the stalls and across the interwebs. Overnight the band became big property, and their notional market value ran higher than Twitter on steroids.

The band members were really please. The presses interviewed Bernice right, left and centre and he made no bones about the fact that a major part of their success was due to Ricky and his Big Data mojo.

Articles about the phenomenon appeared in all the major social media sites. Facebook, LinkedIn and BubbaToons. Ricky was named Supreme Data Scientist of the year by the Gardener Group, hailed as a messiah by the Big Data Front and lauded by all and sundry.

Then the band went on tour. Blazing a trail of ones and zeros across the face of the planet.

They were 5 gigs into their tour and Ricky decided to call a band meeting.

“Hi, guys” said Ricky “I’ve been analysing the stats, and I see that those yokes Big Blokes in Tights are trending strongly on the social media, coinciding with the release of their new single Never Stick A Banger In Your Ear”.

“Oh, whoa” chimes in Bernice, “tell us what we gotta do then, Ricky”.

Back comes Ricky. “Well, this is what I thought we might do”

“We take the old Fester and Ailin song Tropical Diseases, we practice it as much as can, and then we play it at the next gig in Birmingham, this weekend”

“But, Ricky!” pipes up Marty Smarty, “it’s an Irish country and western song. It doesn’t fit in with what we do, does it? And, anyway, we only have three days to get it prepared.”

Ricky responds. “Ah, you don’t want to be worrying your little head over that. Trust me. Learn the song. It’ll be great. The public will love it.”

So, BATM learn the song. It’s perfect. At the Saturday gig, they play it as the encore. The fans love it to bits and there’s not a cold cigarette lighter in the place.

Then they fly off to Palma de Mallorca for a bit of a rest before their next gig in Madrid.

The guys and gals are lounging at the poolside at the legendary Don Pimpón Espinete Plaza complex. The weather is glorious, the food is glorious, the scenery is glorious, and even the orchestra is glorious.

Then along comes Ricky, calling yet another band meeting.

“Hi, guys” said Ricky “I’ve been analysing the stats again, and I see that those yokes Spanky’s Magic Piano are trending strongly on the social media with their cover version of Engel Humpadink’s The Monkey Song”

“Oh yeah, what’s that mean for us, Ricky” chimes in Halo Popette, the bands keyboardist.

Back comes Ricky. “Well, this is what I thought we might do”

“We take the old Fester and Ailin song There’s A Dead Man Up The Chimney, and we rewrite it in the style of Tom Jones when he made that album of his, Little Fockers, was it? Then we practice it as much as can, until it’s perfect, and then we play it at the next gig in Madrid, this weekend”

“But, Ricky!” pipes up Brian McGarsical, “It’s a bit of an odd one isn’t it? I mean to say, it doesn’t fit in with what we do, does it? And, anyway, we only have four days to get it prepared.”

Ricky responds as fast as a chalked-up cat going down a drainpipe. “Ah, you don’t want to be worrying your little head over that. Trust me. Learn the song. It’ll be great. The public will love it. And anyways, it will fit nicely on the playlist, up there with Tropical Diseases.”

The band rewrite the song, and practice the Bedejaysus out of it. Ricky likes it so much that he gets the stats to confirm that this has to be number one on the next gig playlist.

Come the day of the gig, and BATM kick off, not with a progressive-rock anthem, but with There’s A Dead Man Up The Chimney. A group of young people at the front clearly are loving this new sound, but quite a few people are starting at the stage in fright, and it’s not from skunk induced paranoia either.

Two guys are having a conversation at the back of the hall.

“Yo, lunchbox, hurry this gig up, I thought this band was all progressive-rock and stuff, not this wiener schnitzel stuff.”

“No comment.”

Having divided the crowd with their first song, they play songs from their album. Again, they encore with Tropical Diseases. The crowd at the front go wild. The progressive rockers look on, bemused.

“Well, that was a mixed bag” says Bernice.

“Take it from your man Ricky. It all went fine lads. Just needs some fine-tuning of the songs and the analytics need to be a bit more real time. Take me word for it.”

Back comes a unison of “Okay, Ricky. We believe yas!”

So, off they go to Bonn, to prepare for the following weeks gig at the Live Music Hall in Cologne.

The band goes out visiting the museums, they have lunch at Brauhaus Bonnsch, and after a leisurely walk along the banks of Rhine they are taking a beer or three in a lovely little beer garden close to the United Nations campus.

Then out of the blue, a familiar voice can be heard.

“Hi, guys! We’re all goin’ on a summer ‘oliday”. It’s the voice of Ricky. “Anyway, Good news guys. I’ve been analysing the amazin’ Big Data stats again, and I see that those mensch Die Zahnarzt are trending strongly on the social media, especially on Swotter and Titter, with their amazon’ cover version of Podge and Rodge’s chillout mix of Currywurst and Microchips.”

Silence. No one says a word for the best part of infinity.

Ricky continues… “As you’re not going to ask, lads, I’ll tell you. We take the old song A Great Day for the Washing, and we rewrite it in the style of techno-Buddah-bar-chill. Then we practice it as much as can, until it’s perfect, and then we bang it out at the next gig in Cologne, this Friday. Innit. Come on lads, it’s 20 minutes of stage magic, and it’s a breeze.”

Come the day of the gig, and the band arrive early at the hall. Ricky is already there. He’s changed the stage set completely and has a new wardrobe for the lads – Bavarian romantic. They’ll soon be all Princed and Smiley Virused up to the eyeballs, wrecking ball included.

and BATM kick off, not with a progressive-rock anthem or chill, but again with There’s A Dead Man Up The Chimney. Again, a group of young people at the front clearly are loving this new sound, but quite a few people are starting at the stage in drug induced awe. Then they follow that up with A Great Day for Washing. By the time they get to the encore of There’s A Dead Man Up The Chimney, boisterous arguments are breaking out everywhere and empty crisp packets and used sticks of chalk are being thrown at the stage. It’s a disaster.

Four guys are having a conversation at the back of the hall.

“I liked the first song”

“No! The first was terrible. Minging! I want my prog rock back.”

“It’s like the choice of leprosy or the plague.”

“Down with this sort of thing.”

Next day Bernice calls an urgent meeting of the band.

Ricky kicks off.

“Well, lads bit of mid-week game yesterday wasn’t it?”

Bernice comes back with a “You can say that again, Rick”

“Don’t worry, I have analysed the social-media Big Data from all of the concerts, and we’re doing good guys. It’s in the analytics”

“We have to go back to our roots and drop all the changes we made”

A stranger in the lounge where they are having the meeting walks up to them and in simple language explains to them what has happened.

“You created a great product, a great brand, with some interesting progressive music”

“Your music was acclaimed and your world tour was eagerly anticipated by all your fans”

“But then you went wrong”

“You became data driven, dopey and data driven”

“You chased fads, tendencies and styles, and it became a mish-mash”

“People don’t want mish-mashes. Not your base. They wanted good progressive music”

“You’ve lost all credibility. No, you’re just an eccentric band of brothers and sisters that no one will really want to see more than once, if at all”

“Your former fan base is acutely embarrassed by you. That’s your bread, butter, vodka and caviar… in your terms”

“Data drive, Big Data, Big Data analytics in real time?”

“You people have no idea the damage you can do, and so easily”

To be continued…

Many thanks for reading.

Consider this: The Big Data Workout

01 Friday May 2015

Posted by in Big Data, Consider this, good start, goodstart

≈ Leave a comment

Tags

, , , , , ,

To begin at the beginning

Miss Piggy said, “Never eat more than you can lift”. That statement is no less true today, especially when it comes to Big Data.

The biggest disadvantage of Big Data is that there is so much of it, and one of the biggest problems with Big Data is that few people can agree on what it is. Overcoming the disadvantage of size is possible; overcoming the problem of understanding may take some time.

As I mentioned in my piece Taming Big Data, “the best application of Big Data is in systems and methods that will significantly reduce the data footprint.” In that piece I also outlined three conclusions:

  • Taming Big Data is a business, management and technical imperative.
  • The best approach to taming the data avalanche is to ensure there is no data avalanche – this approach is moving the problem upstream.
  • The use of smart ‘data governors’ will provide a practical way to control the flow of high volumes of data.

“Data Governors”, I hear you ask, “What are Data Governors?”

Let me address that question.

Simply stated, the Data Governor approach to Big Data obtuseness is this:

  • The Big Data Governor’s role is to help in the purposeful and meaningful reduction of the ever-expanding data footprint, especially as it relates to data volumes and velocity (see Gartner’s 3Vs).
  • The reduction techniques are exclusion, inclusion and exception.
  • It’s implementation is made through a development environment that can target hardware, firmware, middleware and software forms of hosting and continuously monitored execution.

In Short, it is a comprehensive approach to reducing the Big Data footprint whilst simultaneously maintaining data fidelity.

Here are some examples:

Integrated Circuit Wafer Testing

What’s this all about? Here’s an answer the good folk at Wikipedia cooked up earlier:

“Wafer testing is a step performed during semiconductor device fabrication. During this step, performed before a wafer is sent to die preparation, all individual integrated circuits that are present on the wafer are tested for functional defects by applying special test patterns to them. The wafer testing is performed by a piece of test equipment called a wafer prober. The process of wafer testing can be referred to in several ways: Wafer Final Test (WFT), Electronic Die Sort (EDS) and Circuit Probe (CP) are probably the most common.” (Link / Wikipedia)

Fig.1 – IC Fab Testing and the CE Data Governor

This exhibit shows where the Data Governor is placed in the Integration Circuit fabrication and testing/probing chain.

In large plants, the IC probing process generates very large volumes of data at high velocity rates.

Based on exception rules the Data Governor reduces the flow of data to the centralised data store.

It also speeds up velocity and time to analysis.

Greater speed and less volume mean that production showstoppers are spotted earlier, thereby potentially leading to significant savings in production and recuperation costs.

Let’s look at some of the technical details:

  • Taking our example of the IC Fab test/probe chain, a Data Governor should be able to handle a hierarchy or matrix of designation and exception.
  • For example, a top-level Data Governor actor could be the Production Run actor.
  • The Production Run actor could designate and assign exception rules to a Batch Analysis actor.
  • In turn, the Batch Analysis actor could designate and assign exception rules to a Wafer Instance Analysis actor.

The Internet of Things – IoT

Intrinsically linked to Big Data and Big Data Analytics, the Internet of Things (IoT) is described as follows:

“The Internet of Things (IoT) is the network of physical objects or “things” embedded with electronics, software, sensors and connectivity to enable it to achieve greater value and service by exchanging data with the manufacturer, operator and/or other connected devices. Each thing is uniquely identifiable through its embedded computing system but is able to interoperate within the existing Internet infrastructure.” (Link / Wikipedia)

Fig.2 – The Internet of Things and the CE Data Governor

This exhibit shows where the Data Governor is placed in the Internet of Things data flow.

The Data Governor is embedded into an IoT device, and functions as a data exception engine.

Based on exception rules and triggers the Data Governor reduces the flow of data to the centralised / regionalised data store.

It also speeds up velocity and time to analysis.

Greater speed and less volume means that important signals are spotted earlier, thereby quite possibly leading to more effective analysis and quicker time to action.

Net Activity

Much play is made of the possibility that we will all be extracting golden nuggets from web server logs sometime in the near future. I don’t want to get into the business value argument here, but would like to describe a way of getting Big Data to shed the excess web-server-log bloat.

Fig.3 – Web Server Activity Logging and the CE Data Governor

This exhibit shows where the Data Governor is placed in the capture and logging of interactive internet activity.

The Data Governor acts as a virtual device written to by standard and customised log writers, and functions as a data exception engine.

Based on exception rules and triggers the Data Governor reduces the flow of data from internet activity logging.

It also speeds up velocity and time to analysis.

Greater speed and significantly reduced data volumes may lead to more effective and focused analysis and quicker time to action.

Signal Data

Signal data can be a continuous stream of data originating from devices such as temperature and proximity sensors, by its nature, it can generate high-volumes of data and at high velocity –  it can add lots of data, and very quickly.

Fig.4 – Signal Data and the CE Data Governor

This exhibit shows where the Data Governor is placed in the stream of continuous signal data.

The Data Governor acts as an in-line data-exception engine.

Based on exception rules and triggers the Data Governor reduces the flow of signal data.

It also speeds up velocity and time to analysis.

Greater speed and significantly reduced data volumes may lead to more effective and focused analysis and quicker time to action.

Machine Data

“Machine-generated data is information which was automatically created from a computer process, application, or other machine without the intervention of a human.” (Link / Wikipedia)

Fig.5 – Machine Data and the CE Data Governor

This exhibit shows where the Data Governor is placed in the stream of continuous machine generated data.

The Data Governor acts as an in-line data analysis and exception engine.

Exception data is stored locally and periodically transferred to an analysis centre.

Analysis of the totality of the same class and origins of data can be used to drive ANN* and statistical analysis which can be used to support (for example) the automatic and semi-automatic generation of preventive maintenance rules.

Greater speed and significantly reduced data volumes may lead to more effective and focused analysis and quicker time to proactivity.

Other Applications of the Data Governor

The options are not endless and the prizes are not rich beyond the dreams of avarice, but there are some exciting possibilities out there. Including applications in the trading; plant monitoring; sport; and, climate change ‘spaces’.

Fig.6 – Other Applications in the Big Data ‘space’ and the CE Data Governor

Summary

To wrap up, this is what the CE Data Governor approach looks like at a high level of abstraction:

  • Data is generated, captured, created or invented.
  • It is stored to a real device or virtual device.
  • The Data Governor (in all its configurations) acts as a data discrimination and data exception manager and ensures that significant data is passed on.
  • Significant data is used for ‘business purposes’ and to potentially refine the rules of the CE Data Governor.

To summarise the drivers:

  • We should only generate data that is required, that has value, and that has a business purpose – whether management oriented, business oriented or technical in nature.
  • We should filter Big Data, early and often.
  • We should store, transmit and analyse Big Data only when there is a real business imperative that prompts us to do so.

Moreover, we have a set of clear and justifiable objectives:

  • Making data smaller reduces the data footprint – lower cost, less operational complexity and greater focus.
  • The earlier you filter data the smaller the data footprint is – lower costs, less operational complexity and greater focus.
  • A smaller data footprint accelerates the processing of the data that does have potential business value – lower cost, higher value, less complexity and best focus.

Many thanks for reading.

Can we help? Leave a comment below. contact me via LinkedIn or write to martyn.jones@cambriano.es

Additional graphics:

Consider this: Taming Big Data

01 Friday May 2015

Posted by in Big Data, Big Data Analytics, Consider this, Good Strat, goodstart

≈ 2 Comments

Tags

, , , ,

Simply stated, the best application of Big Data is in systems and methods that will significantly reduce the data footprint.

Why would we want to reduce the data footprint?

  • Years of knowledge and experience in information management strongly suggests that more data does not necessarily lead to better data.
  • The more data there is to generate, move and manage, the greater the development and administrative overheads.
  • The more data we generate, store, replicate, move and transform, the bigger the data, energy and carbon footprints will become.

Continue reading →