Tag Archives: Analytics

The Big Data ‘Wow Wow’ Factor

08 Sunday Feb 2015

Posted by in Big Data, Consider this, Good Strat, Good Strategy, Martyn Jones

≈ Leave a comment

Tags

, , , , , , ,

wFactor

The Wow Wow Factor! Trading, Big Data and 7 HabitsHi, I’m Ricky Jones, boss and co-founder of Becci Boo International Investments. Last week we said goodbye to our best ever Big Data energy commodity trader. He’d been with us for years.

Sadly, Coco Jones was determined to retire to the countryside, to his birthplace, to his real home, a snug little village in the hills of Montseny, and there was absolutely nothing we could do to convince him to stay. The thing about Coco is that he is not like you or me, he’s a highly intelligent Catalan sheepdog.

So, you might ask, how did Coco get to be a star trader at the Becci Boo Hedge Fund? Was it the tools he used? Was it the techniques he adopted? Was it the food he ate? What was so special about him and his amazing abilities? It’s a long story that I will relate as briefly as I can.

Back in time, there was one particularly disastrous week of trading at Becci Boo. Something had gone really wrong with our once reliable Big Data Trade Analytics platform, and wrong bets were being placed right, left and centre – and against trader’s better judgement. The CFO was livid. Out he comes onto the trading floor, swearing and blinding. “God! You guys are the damn pits! What the hell do you think you are doing? Can’t you get anything right? A Catalan sheepdog could trade more effectively than you feckless lot of feckless things.”

I try and diffuse the situation. “Come on, Jordi, don’t be like that, we’re only human and this is a tough business.” “You don’t believe me” he replies. “Sure, but you’re not going to convince me that a Catalan sheepdog could be a substitute for a highly experienced human trader are you?” “How much do you want to bet? These superior four legged beings would never place any faith in things that they don’t understand or can’t control, and they certainly wouldn’t need your Big Data gizmo to make a success of things.”

So, in the following weeks we arrange to run an experiment. We ring around all the owners of registered Catalan sheepdogs and ask them if they would like their dog to take part in our simple trading experiment. Food, lodging and generous expenses all included. We easily manage to get together 64 Catalan sheepdogs and their owners from all over Europe. The experiment we design is quite simple.

We give each dog a gadget (and actually for the more sophisticated dogs it was a smart phone and specially designed app) with two big buttons on it, a red button and a yellow button.

Every Monday morning we ask the assembled dogs to press one of two buttons depending on whether they think that the chosen energy commodities market will close higher or lower at the end of the week.

Red button for a higher closing price, yellow button for a lower closing price. As you might imagine dear reader it’s a walk in the park for these cunning canines. To make sure that all of the participating dogs have equal and fair chances in the experiment, we provide guardians with access to all our big data analytics, and we have multiple screens set up in each studio apartment so that each dog can follow all of the financial news that they need to take in and then view the results of machine learning, data mining and predictive analytics.

At the end of week one, 32 dogs remain in the experiment, and the other 32 are sadly sent home. Every week the exercise is repeated, and every week less dogs move on to the next round. By the end of week 5 only two dogs remain. By the end of week 6 we have a sole winner, Afi Bastò, who has accurately predicted the market movement of our energy commodities market for a record six weeks in succession. It’s an amazing success that proves Jordi right. Unfortunately, this is where things start to go wrong. Our marketing department tweets the news of the amazing Afi and it immediately goes viral.

Articles appear in the Wall Street Journal, Financial Times, Economist, Cinco Dias and Les Échos, hailing the amazing brilliance of the newly discovered Catalan commodity trader.

Hundreds of interviews are held and millions of photos make the rounds of the social and professional networks. Over the following year Afi is interviewed, researched, studied and investigated. Academic papers are written about him.

Amazing claims are made about his canine knowledge, wisdom and experience. This is shortly followed by the publication of a plethora of bestselling business books, with titles such as: The 7 Habits of the Highly Effective Canine – Personal Lessons in Trading; Be More Afi and Grow Rich; The Gos d’Atura That Conquered Chicago; Learn to Trade like Afi; Good Sheepdog, Great Afi; Who Moved My Dog Food; How Afi outperformed Big Data Analytics; etc.

Which unfortunately all goes to Afi’s head, and he begins to seriously lose his commodity trading mojo. So much so, that by the end of the year, we let Afi go, and we bring back “the incredible” Coco Jones, the dog that managed to get the market movements right for five weeks in a row, and who was only pipped at the post by “the amazing” Afi.

Since then, Coco has helped us to win far more than we lose, he has had an innate knack of being able to combine market knowledge with statistical analysis and a certain indescribable ‘insight’ that no one quite manages to understand never mind emulate, and all combined with an amazing caring character.

So, as you might imagine, we were really sad to see him go. After his epic leaving party, where he is acclaimed by all and sundry, but especially by Jordi, all the staff assembled outside the office entrance to see him off, and as he was slowly chauffeured away down the driveway he turned and looked back, and his face said it all. “I was lucky — I found what I loved to do early in life”.

Many thanks for reading.  

Here’s Coco paddling in his Big Data Lake

OLYMPUS DIGITAL CAMERA

File under: Good Strat, Good Strategy, Martyn Richard Jones, Cambriano Energy, Iniciativa Consulting, Iniciativa para Data Warehouse, Tiki Taka Pro

Consider this: Big Data and the Analytics Data Store

19 Monday Jan 2015

Posted by in Analytics, Big Data, Consider this, statistics

≈ Leave a comment

Tags

, , , ,

To begin at the beginning

Hold this thought: If Data Warehousing was Tesco then Big Data would be the “try something different”.

Since the publication of the article Aligning Big Data, which basically laid out a draft view of DW 3.0 Information Supply Framework and placed Big Data within a larger framework, I have been asked on a number of occasions recently to go into a little more detail with regards to the Analytics Data Store (ADS) component. This is an initial response to those requests. Continue reading →

The World’s Best Data Quotes… Including Big Data quotes

17 Saturday Jan 2015

Posted by in Analytics, Architecture, Big Data, Business Intelligence, Consider this, Data Warehousing, statistics

≈ 4 Comments

Tags

, , , , , ,

OLYMPUS DIGITAL CAMERA

A Random walk down Data Street

If you enjoy, abhor or are simply bored with the massive surfeit of hype surrounding Big Data, Data Warehousing and Analytics, then you might just hate these less than faithful quotes as well.

If you enjoy one or two of the quotes, well, then that’s an acceptable bonus too.

So, to begin at the beginning…xHound

Data Sources

“My data sources are unreliable, but their information is fascinating.” – Ashleigh Brilliant

“I give no data sources, because it is indifferent to me whether what data I have sourced has already been sourced before me by another.” – Ludwig Wittgenstein

“In the kitchen of a great Data Warehouse, the data source chef is a soloist.” – Fernand Point

“It is better to be hated for what data sources you have than to be loved for what data sources you do not have.” – André Gide

“In England, there are sixty different types of Data Warehouse and only one data source.” – Attributed to Voltaire

“It is a capital mistake to theorize before one has data sources. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.” – Arthur Conan Doyle, Sherlock Holmes

“From such a gentle thing, from such a source of all data, my every pain is born.” –Michelangelo

“Noise free data is a source of great strength.” – Lao Tzu

“In three words I can sum up everything I’ve learned about data: it goes on.” – Robert Frost

“Data enrichment improves a mighty fine data source” – Anonymous

xButcherBig Data

“Junk food, empty calories and carbs are the Big Data of the masses” – Karl Marx

“We live, I regret to say, in an age of Big Data hype.” – Oscar Wilde

“We are not rich by the Big Data we possess but by what Big Data we can do without.” – Immanuel Kant

“He who has Big Data hype on his side has no need of proof.” – Theodor Adorno

“The religion of Big Data sets itself the goal of fulfilling man’s unattainable desires, but for that very reason ignores her attainable needs.” – Ludwig Feuerbach

“The flesh endures the storms of the present alone; the mind in our social network interactions, those of the past and future as well as the present. Big Data is a covetousness of the mind.” – Thomas Hobbes

“Big Data is negative and dialectical, because it resolves the determinations of the understanding of things into nothings.” – Georg Wilhelm Friedrich Hegel

“I am trapped in this Big Data, and there is nothing I can do about it.” – Dudley Moore

“And remember, never take the ruby case off your iPad for a moment, or you will be at the mercy of the Big Data Witch of the West.” – The Wizard of Oz

“Imagine there’s no Big Data…” – John Lennon

Abacus3Data Transformation

“Analysis does not transform data.” – Jiddu Krishnamurtu

“I live in a data landscape, which every single day of my life is enriching data.” – Daniel Day-Lewis

“Data opportunities multiply as the data is transformed” – Sun Tzu

“He who integrates data badly is lost.” – Theodor Adorno

“Today we transform the data; tomorrow, the whole enchilada” – Leon Trotsky

“Well, it’s all about the ETL law of the transformation of data quantity into data quality, and vice versa. Innit!” – Friedrich Engels

“The management consultants have only interpreted the business data, in various ways. The point, however, is to transform it.” – Karl Marx

“Hey! What’s going down here in the Hollyweird of data?” – Joe McCarthy

“The Big Data alchemists in their transformational search for gold discovered much data of greater value.” – Arthur Schopenhauer

“That Schopenhauer yolk was a bit of an old Big Data ‘procurer’ wasn’t he now Rodge?” – Pádraig Judas O’Leprosy

IMGQBusiness Intelligence

“The trouble with the world is that the cocksure have Big Data and that Data Science and Business Intelligence are all sexed up.” – Bertrand Russell

If people never did silly things no Business Intelligence would ever get done.” – Ludwig Wittgenstein

“The best Business Intelligence user is intelligent, well-educated and a little drunk.” –Alben W Barkley

“The Master said, “If your conduct is determined solely by considerations of Business Intelligence and profit you will arouse great resentment.” ― Confucius

“That’s cricket, Harry, you get these sort of things in Business Intelligence” – Frank Bruno

“Business Intelligence without ambition is a bird without wings.” – Salvador Dali

“I would prefer a Business Intelligence hell to a Big Data paradise.” – Blaise Pascal

“Many much-learned business men have no Business Intelligence.” – Democritus

“We should not only use the brains we have, but all that we can borrow.” – Woodrow Wilson

“The reason we have Business Intelligence is so we don’t have to think all the time” –Homer Simpson

P3160034Data Warehousing

“The study of Data Warehousing, like the Nile, begins in Inmon and ends in magnificence.” – Charles Caleb Colton

“Big Data wins games, but Data Warehousing wins championships.” – Michael Jordan

“Big Data is no substitute for Data Warehousing.” – Frank Herbert

“It’s in me blood, Clive, without Data Warehousing I’d be nothing,” – Alan Latchley

“The trouble with the world is that the cocksure have Big Data and that Data Science is all sexed up.” – Bertrand Russell

If people never did silly things no Business Intelligence would ever get done.” – Ludwig Wittgenstein

“The best Business Intelligence user is intelligent, well-educated and a little drunk.” –Alben W Barkley

“You can catch all the whales in the ocean and stack them together and they still do not make a minnow.” – Ralph Wiggum

“Well, the smarter I practice Inmon Data Warehousing, the luckier I get.” – Gary Player

“Well, I’ve cleaned up facts and dimensions in a star-schema ‘data warehouse’. That was pretty terrible. But I can’t complain because I’m sure other people have done worse.” – Cee Lo Green

“You can give a person a bowl of Big Data Gruel and feed them for a day, or teach them Inmon Data Warehousing and feed them for a lifetime.” – Proverb

“A Data Warehouse is like a tea bag; you never know how strong it is until you are in hot water.” – Eleanor Roosevelt

” οἶδα δ᾽ ἐγὼ ψάμμου τ᾽ ἀριθμὸν καὶ μέτρα θαλάσσης, καὶ κωφοῦ συνίημι, καὶ οὐ φωνεῦντος ἀκούω. ὀδμή μ᾽ ἐς φρένας ἦλθε κραταιρίνοιο χελώνης ἑψομένης ἐν χαλκῷ ἅμ᾽ ἀρνείοισι κρέεσσιν, ᾗ χαλκὸς μὲν ὑπέστρωται, χαλκὸν δ᾽ ἐπιέσται.” – An Oracle to Croesus of Lydia

IMGThat’s all folks!

Well, now that that’s done I can always ask for forgiveness. Not that I will of course.

 Many thanks for reading.

abfab111

Founder and CEO, Cambriano Energy

File under: Good Strat, Good Strategy, Martyn Richard Jones, Martyn Jones, Cambriano Energy, Iniciativa Consulting, Iniciativa para Data Warehouse, Tiki Taka Pro

Continue reading →

Marty does… Big Data and the Vs

09 Tuesday Dec 2014

Posted by in Big Data, Marty does

≈ 11 Comments

Tags

, , , , ,

Clive: Yeah, well, you had to, didn’t you? You had to stand up for what you stood for, didn’t you? I mean, the only time I remember a similar occasion was, I was in, errm… I was at Spurs, Tottenham Hotspurs.

Derek: Yeah.

Clive: I was watching a game against Arsenal, and this bloke come up to me and said, “Hello”.

Derek: Oh no…

Derek and Clive – This Bloke Came Up to Me Continue reading →

The Awkward Squad – Big data informs

11 Saturday Oct 2014

Posted by in Big Data, BS, Data Warehouse, disinformation, information, Knowledge, wisdom

≈ Leave a comment

Tags

, , , , ,

I am a sceptic. Part of the awkward squad of troublemakers.

People who ask questions and who won’t stop asking.

People who won’t take bullshit for an answer.

People who are not preprogramed to follow certain paths, unquestioningly.

But to question everything.

Such as, “What the feck’s that all about then?”

I’m in good company. Many people who have made a difference have been fully paid up members of the awkward squad.

People in the awkward squad might sell dog food, but we know that we shouldn’t eat it ourselves.

I used to tell people.

“If you must exaggerate, try and remember this one thing”

“What’s that then, Marty?”

“Never, ever, believe your own bullshit or you’ll ending up having to eat it”.

I remember when Cloud first appeared on the horizon, a marketing idea that was to popularise the expression “put it on the cloud”.

I vaguely remember Larry Ellison being asked about Cloud.

If I recall rightly his reply was along the lines of ‘Cloud? Oh, you mean connected mainframes and data centres?”

He saw it, others saw it, and I saw it. It had been done many times before.

But cloud was new, exciting, vibrant and well, vague enough for the market.

Only that it wasn’t and isn’t new.

The only thing is, a handful of stylists and hacks were let loose on what already existed, and they came up with a new idea, that wasn’t new, creative or innovative.

It was just repackaged. Old wine in new bottles.

I have the same issues with marketing terms such as business intelligence, virtualisation and big data.

Can you imagine Steve Jobs peddling such rebranded and rebadged crap?

I can’t.

Big data brings all the promise of being better informed by having access to far more data.

But for most things in the commercial world quantity of data has never been the issue.

If anything, we’ve had too much of it and for far too long.

We’ve been doing big data for years.

Previously we called it Very Large Data Bases.

We have been handling some forms of highly structured data for years.

We used to call it things like text management, document management and knowledge management.

Not that it matters too much.

We are still looking for real insight, but most of us are overwhelmed by countless gigabytes, terabytes and petabytes of data, and much of what we get is recycled, repackaged and ultimately repetitive.

We are drowned in data, low-utility information and marketing hype.

For all the good that the information we receive does us, we may as well be more dog.

As Ad superman Dave Trott asked, what the feck does ‘be more dog’ mean?

Exactly…

It doesn’t mean anything.

We could write a whole litany of the endless succession of IT snake-oil merchants that have passed through techy-tinsel-town flogging yet another dead-horse as the latest and greatest Kentucky Derby favourite.

But that’s just hokie. Even the media and the presses are in on it, up-close and intimate collaborators in keeping reality from us, by burying us in shit.

Remember the joke about the mushrooms?

That’s right.

“Keep them in the dark, feed them bullshit, and watch them grow”

Well, it came true.

Like life reflecting comic art.

Actually, most of us are still starved of knowledge and insight.

And, as for wisdom?

What’s that then?

Operational awareness isn’t for wimps! So, get it right

08 Wednesday Oct 2014

Posted by in awareness, Big Data, business, Business Intelligence, Data Warehouse, operational, pig data, Strategy

≈ Leave a comment

Tags

, , , , , , ,

I frequently include the term Operational Awareness in talks.

I think it’s important for strategy.

So I wrote a piece that tried to convey what I mean by the term.

But first, a diagram:

Image1

Figure 1 – Operational awareness

This is a simplified high-level example of business data objects found in certain organisations. In the above diagram I have reused an industry example of nine business data objects to represent operational data[1].

For completeness and to maintain rationality in this section here follows a summary list of the nine key groups of business data needed to have a coherent and cohesive operational awareness (these data groups are also frequently referred to as business data objects):

  • A: Party embodies all of the participants that may have contact with the organisation or that are of interest to the organisation and about which the organisation maintains data. This includes data about the organisation itself; data about external organisations; data about external and internal individuals; and, data about the roles of involved parties.
  • B: Arrangement represents a prospective or existing agreement, between two or more individuals, organizations or organizational units that provides and affirms the rights, rules and obligations associated with a transaction between parties.
  • C: Condition describes the specific requirements that pertain to how the business is conducted and includes information such as prerequisite or qualification criteria and restrictions or limits associated with the requirements. Conditions can apply to various aspects of an enterprise’s operations, such as the operational parameters of a resource item, the sale and servicing of products, the determination of eligibility to purchase a product, the authority to perform business transactions, the assignment of specific general ledger accounts appropriate for different business transactions, the required file retention periods for various types of information kept by an enterprise and the selection criteria for a market segment.
  • D: Product/Service describes the services, merchandise or facilities that can be offered, sold or purchased by the enterprise, its competitors and other Involved Parties during the normal course of its business. This concept also includes goods and services that are of interest to the enterprise such as supplies for manufacture.
  • E: Location covers a place where something can be found, a destination of information or a bounded area, such as a country or state, about which the enterprise wishes to keep information.
  • F: Classification is used to organize and manage specific business information by defining structures that represent classification categories. Classification also organizes and manages groups of business concepts that apply to multiple concepts.
  • G: Business Direction/Organisation Direction refers to and records expressions of a party’s intent with regard to the manner and environments in which it wishes to carry out its business. Business direction items contains, keeps data about, and is used to support the enterprise’s business and financial plans, policies, procedures and schedules.
  • H: Events describe a happening about which the organisation wishes to keep information as a part of carrying out its mission and conducting its mission.
  • I: Resource object includes and describes any value item, either tangible or intangible, that is owned, managed or used by, or of specific interest to, the organisation in the course of accomplishing its mission.

The key facets of operational awareness detailed above constitute a potential of fundamental importance in the formulation of organizational strategy.

Timely, accurate and appropriate data at this level can temper ambition with the facts on the ground, with operation insight, and with the effectiveness of time and place utilisation.

But take care. In most organisations there will be a spread of attention to the key facets illustrated here, and they will be treated with varying degrees of intensity relative to their overall contribution to strategy formulation. In addition, organizational specific facets may also be introduced where needed in order to complement the overall set of operational awareness facets described here.

[1] IBM’s IFW/BDW.