• Home
  • About
  • Strategy
    • Data Warehousing
    • Ask Martyn
  • Opinion
  • MARTYN
    • MARTYN’S MUSIC
    • Must-Read Books from Martyn
    • PODCASTS
    • MARTYN.ES

GOOD STRATEGY

~ DATA WITH DIRECTION

GOOD STRATEGY

Tag Archives: Statistical Analysis

The Big Data ‘Wow Wow’ Factor

08 Sunday Feb 2015

Posted by Martyn Rhisiart Jones in Big Data, Consider this, Good Strat, Good Strategy, Martyn Jones

≈ Leave a comment

Tags

Big Data, big data analytics, business intelligence, Commodity Trading, data science, Data-Driven Decision Making, Predictive Analytics, Statistical Analysis, The Sheepdog School of Big Data, Trading Strategies

wFactor

The Wow Wow Factor! Trading, Big Data and 7 HabitsHi, I’m Ricky Jones, boss and co-founder of Becci Boo International Investments. Last week we said goodbye to our best ever Big Data energy commodity trader. He’d been with us for years.

Sadly, Coco Jones was determined to retire to the countryside, to his birthplace, to his real home, a snug little village in the hills of Montseny, and there was absolutely nothing we could do to convince him to stay. The thing about Coco is that he is not like you or me, he’s a highly intelligent Catalan sheepdog.

So, you might ask, how did Coco get to be a star trader at the Becci Boo Hedge Fund? Was it the tools he used? Was it the techniques he adopted? Was it the food he ate? What was so special about him and his amazing abilities? It’s a long story that I will relate as briefly as I can.

Back in time, there was one particularly disastrous week of trading at Becci Boo. Something had gone really wrong with our once reliable Big Data Trade Analytics platform, and wrong bets were being placed right, left and centre – and against trader’s better judgement. The CFO was livid. Out he comes onto the trading floor, swearing and blinding. “God! You guys are the damn pits! What the hell do you think you are doing? Can’t you get anything right? A Catalan sheepdog could trade more effectively than you feckless lot of feckless things.”

I try and diffuse the situation. “Come on, Jordi, don’t be like that, we’re only human and this is a tough business.” “You don’t believe me” he replies. “Sure, but you’re not going to convince me that a Catalan sheepdog could be a substitute for a highly experienced human trader are you?” “How much do you want to bet? These superior four legged beings would never place any faith in things that they don’t understand or can’t control, and they certainly wouldn’t need your Big Data gizmo to make a success of things.”

So, in the following weeks we arrange to run an experiment. We ring around all the owners of registered Catalan sheepdogs and ask them if they would like their dog to take part in our simple trading experiment. Food, lodging and generous expenses all included. We easily manage to get together 64 Catalan sheepdogs and their owners from all over Europe. The experiment we design is quite simple.

We give each dog a gadget (and actually for the more sophisticated dogs it was a smart phone and specially designed app) with two big buttons on it, a red button and a yellow button.

Every Monday morning we ask the assembled dogs to press one of two buttons depending on whether they think that the chosen energy commodities market will close higher or lower at the end of the week.

Red button for a higher closing price, yellow button for a lower closing price. As you might imagine dear reader it’s a walk in the park for these cunning canines. To make sure that all of the participating dogs have equal and fair chances in the experiment, we provide guardians with access to all our big data analytics, and we have multiple screens set up in each studio apartment so that each dog can follow all of the financial news that they need to take in and then view the results of machine learning, data mining and predictive analytics.

At the end of week one, 32 dogs remain in the experiment, and the other 32 are sadly sent home. Every week the exercise is repeated, and every week less dogs move on to the next round. By the end of week 5 only two dogs remain. By the end of week 6 we have a sole winner, Afi Bastò, who has accurately predicted the market movement of our energy commodities market for a record six weeks in succession. It’s an amazing success that proves Jordi right. Unfortunately, this is where things start to go wrong. Our marketing department tweets the news of the amazing Afi and it immediately goes viral.

Articles appear in the Wall Street Journal, Financial Times, Economist, Cinco Dias and Les Échos, hailing the amazing brilliance of the newly discovered Catalan commodity trader.

Hundreds of interviews are held and millions of photos make the rounds of the social and professional networks. Over the following year Afi is interviewed, researched, studied and investigated. Academic papers are written about him.

Amazing claims are made about his canine knowledge, wisdom and experience. This is shortly followed by the publication of a plethora of bestselling business books, with titles such as: The 7 Habits of the Highly Effective Canine – Personal Lessons in Trading; Be More Afi and Grow Rich; The Gos d’Atura That Conquered Chicago; Learn to Trade like Afi; Good Sheepdog, Great Afi; Who Moved My Dog Food; How Afi outperformed Big Data Analytics; etc.

Which unfortunately all goes to Afi’s head, and he begins to seriously lose his commodity trading mojo. So much so, that by the end of the year, we let Afi go, and we bring back “the incredible” Coco Jones, the dog that managed to get the market movements right for five weeks in a row, and who was only pipped at the post by “the amazing” Afi.

Since then, Coco has helped us to win far more than we lose, he has had an innate knack of being able to combine market knowledge with statistical analysis and a certain indescribable ‘insight’ that no one quite manages to understand never mind emulate, and all combined with an amazing caring character.

So, as you might imagine, we were really sad to see him go. After his epic leaving party, where he is acclaimed by all and sundry, but especially by Jordi, all the staff assembled outside the office entrance to see him off, and as he was slowly chauffeured away down the driveway he turned and looked back, and his face said it all. “I was lucky — I found what I loved to do early in life”.

Many thanks for reading.  

Here’s Coco paddling in his Big Data Lake

OLYMPUS DIGITAL CAMERA

File under: Good Strat, Good Strategy, Martyn Richard Jones, Cambriano Energy, Iniciativa Consulting, Iniciativa para Data Warehouse, Tiki Taka Pro

All Data: It’s about statistics

30 Friday Jan 2015

Posted by Martyn Rhisiart Jones in All Data, Consider this, DW 3.0, Good Strat, Good Strategy, Information Supply Frameowrk, Martyn Jones, Martyn Richard Jones, statistics

≈ Leave a comment

Tags

Big Data, business intelligence, data analytics, data science, Data-Driven Decision Making, It's All About the Numbers, Predictive Analytics, Probability, Statistical Analysis, statistics

LinkedInHeader1

A big computer, a complex algorithm and a long time does not equal science.

Robert Gentleman

To begin at the beginning

Fueled by the new fashions on the block, principally Big Data, the Internet of Things, and to a lesser extent Cloud computing, there’s a debate quietly taking please over what statistics is and is not, and where it fits in the whole new brave world of data architecture and management. For this piece I would like to put aspects of this discussion into context, by asking what ‘Core Statistics’ means in the context of the DW 3.0 Information Supply Framework.

Core Statistics on the DW 3.0 Landscape

The following diagram illustrates the overall DW 3.0 framework:

There are three main concepts in this diagram: Data Sources; Core Data Warehousing; and, Core Statistics.

Data Sources: All current sources, varieties, velocities and volumes of data available.

Core Data Warehousing: All required content, including data, information and outcomes derived from statistical analysis.

Core Statistics: This is the body of statistical competence, and the data used by that competence. A key data component of Core Statistics is the Analytics Data Store, which is designed to support the requirements of statisticians.

The focus of this piece is on Core Statistics. It briefly looks at the aspect of demand driven data provisioning for statistical analysis and what ‘statistics’ means in the context of the DW 3.0 framework.

Demand Driven Data Provisioning

The DW 3.0 Information Supply Framework isn’t primarily about statistics it’s about data supply. However, the provision of adequate, appropriate and timely demand-driven data to statisticians for statistical analysis is very much an integral part of the DW 3.0 philosophy, framework and architecture.

Within DW 3.0 there are a number of key activities and artifacts that support the effective functioning of all associated processes. Here are some examples:

All Data Investigation: An activity centre that carries out research into potential new sources of data and analyses the effectiveness of existing sources of data and its usage. It is also responsible for identifying markets for data owned by the organization.

All Data Brokerage: An activity that focuses on all aspects of matching data demand to data supply, including negotiating supply, service levels and quality agreements with data suppliers and data users. It also deals with contractual and technical arrangements to supply data to corporate subsidiaries and external data customers.

All Data Quality: Much of the requirements for clean and useable data, regardless of data volumes, variety and velocity, have been addressed by methods, tools and techniques developed over the last four decades. Data migration, data conversion, data integration, and data warehousing have all brought about advances in the field of data quality. The All Data Quality function focuses on providing quality in all aspects of information supply, including data quality, data suitability, quality and appropriateness of data structures, and data use.

All Data Catalogue: The creation and maintenance of a catalogue of internal and external sources of data, its provenance, quality, format, etc. It is compiled based on explicit demand and implicit anticipation of demand, and is the result of an active scanning of the ‘data markets’, ‘potential new sources’ of data and existing and emerging data suppliers.

All Data Inventory: This is a subset of the All Data Catalogue. It identifies, describes and quantifies the data in terms of a full range of metadata elements, including provenance, quality, and transformation rules. It encompasses business, management and technical metadata; usage data; and, qualitative and quantitative contribution data.

Of course there are many more activities and artifacts involved in the overall DW 3.0 framework.

Yes, but is it all statistics?

Statistics, it is said, is the study of the collection, organization, analysis, interpretation and presentation of data. It deals with all aspects of data, including the planning of data collection in terms of the design of surveys and experiments; learning from data, and of measuring, controlling, and communicating uncertainty; and it provides the navigation essential for controlling the course of scientific and societal advances[i]. It is also about applying statistical thinking and methods to a wide variety of scientific, social, and business endeavors in such areas as astronomy, biology, education, economics, engineering, genetics, marketing, medicine, psychology, public health, sports, among many.

Core Statistics supports micro and macro oriented statistical data, and metadata for syntactical projection (representation-orientation); semantic projection (content-orientation); and, pragmatic projection (purpose-orientation).

The Core Statistics approach provides a full range of data artifacts, logistics and controls to meet an ever growing and varied demand for data to support the statistician, including the areas of data mining and predictive analytics. Moreover, and this is going to be tough for some people to accept, the focus of Core Statistics is on professional statistical analysis of all relevant data of all varieties, volumes and velocities, and not, for example, on the fanciful and unsubstantiated data requirements of amateur ‘analysts’ and ‘scientists’ dedicated to finding causation free correlations and interesting shapes in clouds.

That’s all folks

This has been a brief look at the role of DW 3.0 in supplying data to statisticians.

One key aspect of the Core Statistics element of the DW 3.0 framework is that it renders irrelevant the hyperbolic claims that statisticians are not equipped to deal with data variety, volumes and velocity.

Even with the advent of Big Data alchemy is still alchemy, and data analysis is still about statistics.

If you have any questions about this aspect of the framework then please feel free to contact me, or to leave a comment below.

Many thanks for reading.

Catalogue under: #bigdata #technology

[i] Davidian, M. and Louis, T. A., 10.1126/science.1218685


File under: Good Strat, Good Strategy, Martyn Richard Jones, Martyn Jones, Cambriano Energy, Iniciativa Consulting, Iniciativa para Data Warehouse, Tiki Taka Pro

Consider this: Big Data in Context

21 Wednesday Jan 2015

Posted by Martyn Rhisiart Jones in Big Data, Consider this, Data Warehouse, Data Warehousing

≈ Leave a comment

Tags

Big Data, business intelligence, data architecture, data science, Data Warehousing, data-strategy, DW 3.0, information management, Making Big Data Make Sense, Statistical Analysis

Big Data, together with Cloud computing and the Internet of Things, are topics that are very much to the fore in contemporary trends in Information Management. Continue reading →

CONSIDER THIS: Did Big Data Kill The Statistician? 2026

03 Wednesday Dec 2014

Posted by Martyn Rhisiart Jones in consider, Consider this, data science, Polemic, statistics

≈ 20 Comments

Tags

Big Data, data analytics, Data Literacy, data science, Data Scientists, Predictive Analytics, Quantitative Analysis, Statistical Analysis, statistics, The Big Data Statistics Circus

OLYMPUS DIGITAL CAMERA
Blue sky data

Martyn Rhisiart Jones

Hold this thought: ‘There are big lies, damn big lies and big data science’.

Statistics is a science. Some argue that it is the oldest of sciences. It can be traced back in history to the days of Augustus Caesar, and before.

In 1998, Lynn Billard wrote a paper that laid out the role of the Statistician and Statistics. She stated that “no science began until man mastered the concepts and arts of counting, measuring, and weighting”.[1]

Continue reading →

Protesters? Actually, most people are happy with things as they stand – Remaster 2026

01 Wednesday Oct 2014

Posted by Martyn Rhisiart Jones in Big Data, Business Intelligence, Data Warehouse

≈ Leave a comment

Tags

Behavioural Economics, Big Data, business intelligence, Data-Driven Decision Making, Protest Movements, Public Opinion, Social Analytics, Social Behaviour, Statistical Analysis, The Great Global Moan-Off.

If you judged everything by what the media tells us, you would get the impression that the world is in continuous revolt. Violence, destruction and protest is everywhere. It’s like as if every hour of the day there is one reason or another to go marching, standing or sitting somewhere about something.

Well, that’s the impression one may get.

Continue reading →

Want the same partner in old age? Never marry tall! – Remaster 2026

30 Tuesday Sep 2014

Posted by Martyn Rhisiart Jones in Big Data

≈ Leave a comment

Tags

Big Data, Correlation vs Causation, data analytics, Data Misinterpretation, Data-Driven Decision Making, Demographic Analysis, Population Statistics, Social Trends, Statistical Analysis, The Tall-People Conspiracy

You walk down the street, maybe in London, Paris and Rome. You look out of your spacious apartment in Manhattan, Unter den Linden or Mahim. You are cycling slowly along the streets of Amsterdam, Bonn or Zurich.

Got the picture?

Continue reading →

Follow Good Strategy

Enter your email address to follow this strategy platform and receive free notifications of new content by email.

Top posts

  • Automatic identification and data capture (AIDC)
  • TEAM 2.0 Total Eminence Analytical Mapper – The highest level architecture
  • Big Data, a promised land where the Big Bucks grow
  • Origins of Relational Databases: Transaction Processing Misconceptions
  • CEO BRIEFING: The Truth Behind AI Hype in 2026 - 2026/01/22
  • Consider this: The ten key dimensions of Applied Business Knowledge and AI
  • BERNIE AND THE SLICKEST HEDGE FUND IN THE WEST
  • Mario Benedetti, 1920 to 2009
  • The World's Best Data Quotes... Including Big Data quotes
  • Postmodern Digital Stories: We've never seen anything like this before

Recent Comments

Unlock Business Valu… on Building the Data Logistics Hu…
Unlock Business Valu… on DATA WORLD: Big Data With Bigg…
Unlock Business Valu… on Revealing Hidden Wealth throug…
Unlocking Business V… on Building the Data Logistics Hu…
Unlocking Business V… on DATA WORLD: Understanding the…

Meta

  • Register
  • Log in
  • Entries feed
  • Comments feed
  • WordPress.org

Recent articles

  • Strategic Data Warehousing: A Smart Approach – REMASTER 2026
  • NATIONAL RECONSTRUCTION: THE GOSPEL ACCORDING TO THE SANHEDRINS OF THE CONTRACTOR
  • RECONSTRUCCIÓN NACIONAL DEL PARTIDO PATO, NO PATÓ
  • LA RECONSTRUCCIÓN NACIONAL DE LA RE-POSTURA
  • Censoring Mister Jones
  • The Top 7 Vapourware Hallucinations For Next Year
  • Tokenmaxxing for Jesus

The Stats

  • 131,542 hits

Meta

  • Register
  • Log in
  • Entries feed
  • Comments feed
  • WordPress.org
Log in
  • Strategic Data Warehousing: A Smart Approach – REMASTER 2026

Create a website or blog at WordPress.com

Loading Comments...