Category: Big Data

Posts that contain a reference to Big Data

  • Big Data in Question – Again

    Big Data in Question – Again

    Big Data is now an inhospitable and unhealthy land inhabited by those who, through accident or design, deceive naïve and sentimental bystanders and those who are willingly mislead.

    When all of this Big Data malarkey started it was sort of funny, humorous and occasional witty, especially in the affected, bizarre and the frequently uninhibited ways that freshly-minted self-appointed gurus and experts would “big it up”

    Doctor Freud would have had a field day with all of that, being as it was, and for that matter still is, a postmodern mishmash of Riefenstahl, Freddy Mercury and Monty Python on steroids. However, after that extended, operatic and high-camp hiatus it all went downhill.

    The Big Data scene is fast becoming an outrageous and brash festival of deception, disinformation and obliviousness. Which is a pity, because it does the industry no good whatsoever.

    It is telling that Big Data evangelists, gurus and assorted sycophants cannot even define Big Data adequately, never mind discuss (or for that matter, point at) tangible success stories, without falling into contradictions on all of the key defining characteristics of volume, variety and velocity, and resorting to crude debating devices to avoid or finesse the concerns and the questions.

    Almost every morning I check out the industry news, and almost invariably, it comes with new mind-boggling examples of Big Data nonsense.

    However, it isn’t always nonsense for nonsense’s sake, there are agendas, there are rational explanations why Big Data has become at the same time, one of the most hyped up fads in the history of IT, and one that its supporters find so difficult to actually explain and justify, in any reasonable sort of way.

    Therefore, when it comes to Big Data, beyond the surfeit of platitudes, clichés, bluff and bluster, the only thing in play are the interests of industry, the patrons, the courtesans and their entourage of the innocent and the beguiled.

    One of the biggest deceptions in Big Data is in the misleadingly named ‘success stories’. The thing is that most of these success stories that I have ever read have been:

    • So vague that it’s difficult to know how success is being defined never mind reached.
    • So secretive and obtuse is the avoidance of naming names, locations and other relevant Big Data references that it’s impossible to corroborate if these claims are actually true or not. Disclaimer: I have worked for some of the biggest IT vendors, and in senior roles, and I know what is behind comments such as “the Big Data project is a success, although the client name and project are confidential” and “it’s delivering such major competitive advantages that we are obliged to keep it under wraps”.
    • Stories stolen from elsewhere, such as from Data Warehousing, Business Intelligence, VLDB or Business Application projects.
    • Borderline fantasies and badly contrived technology fan fiction.

    However, it doesn’t stop there.

    One of the clearest examples of the questionable nature of Big Data evangelism is when it is used to piggyback Big Data hype on simple, tangible and immediately recognisable artefacts or applications that have little in common with Big Data.

    This is an extreme illustration, but it works like this: “iPhones are commercially successful, iPhones are part of Big Data, and therefore Big Data is commercially successful.”

    As if the mere conjuring up of association, affinity and proximity will convince people of the great and growing value of Big Data.

    What I am also referring to are publicity pieces that may as well have been titled:

    • Smith, Galbraith, Mies, Keynes, Homer SImpson and the economic justification of Big Data
    • Lovelace, Babbage, von Neumann, Eckert, Davies, Codd, Knuth, Naur and the technological underpinnings of Big Data
    • Einstein, Freud, Edison, Faraday, Recorde and the intellectual structure of Big Data
    • Socrates, Kant, Hegel, Marx , Adorno and the philosophical correctness of Big Data
    • Great quotes about Big Data, from the Cambrian era to the postmodern époque
    • Great jokes about Big Data, from Mel Brooks to Steve Martin
    • Sportspeople and Big Data, from Lottie Dodd and Babe Ruth to Rafa Nadal and CR7
    • Industry support of Big Data, from Henry Ford to Neutron Jack

    Do you recognise similarities?

    It’s no big deal, just the use of unreliable, misleading and inappropriate fallacies, dressed up as cute, plausible and accessible collateral. People may think that such things are clever and witty, but they aren’t, it’s just misleading.

    Let’s continue with something simple.

    Evasion is, in ethics, an act that deceives by stating a true statement that is immaterial or leads to a false deduction. For example, citing events, persons or anecdotes from the history of IT to justify the supposed or imaginary value of Big Data. This is close to the notion of a non sequitur, which of course is an argument, the conclusions from which do not follow from its premise. It falls short of being full-on sophistry, purely because the simplistic, puerile and superficial arguments put forward in favour of Big Data do not match those of the true sophist who seeks to reason with clever but fallacious and deceptive arguments. Too many of the Big Data arguments are fallacious and deceptive, but no one, equipped with a reasonable capacity for critical thinking, should take such ‘arguments’ as valid.

    Hold this thought: Big Data hype is a viper’s nest of logical fallacies, white lies and disinformation.

    Just when I think things could not get any weirder, they do, and Big Data ceiling of hyperbole rises even higher, up to the rarer atmosphere of extreme tendentiousness.

    There is a growing mass of Big Data hoop-la, hyperbole and flim flam that exceeds all previously bounds of overstatement, solecism and confabulation. This is where the real volumes, varieties and velocities are in Big Data; in hokie.

    We live, as Oscar Wilde said in his day, in and age of surfaces. Yes, superficiality, puerility and short-termism are the competing orders of the day. However, I am still amazed – and maybe wrongly so – by what ostensibly professional, experienced and knowledgeable people are willing, able and prepared to accept, especially when it comes to Big Data flim flam sauce.

    Here are some examples of the nonsense about Big Data that is taken as gospel by ‘adults’:

    Data Warehousing is part of Big Data: No comment.

    Big Data will replace Enterprise Data Warehousing: People can’t even explain the features and benefits of Big Data. I try it make it as easy as possible, ‘if you can’t say it, point to it’. But, seriously, people can’t even relate tangible and credible Big Data success stories, never mind show how it will replace Enterprise Data Warehousing, whether that’s the Inmon or Kimball flavour, take your pick.

    Everyone and every organisation can benefit from Big Data: If people can’t explain this, and they don’t in terms of tangible benefits, then the claim should remain questionable.

    Data Scientists will replace Statisticians: Why is that so? It is claimed that Data Scientists are uniquely equipped to handle massive volumes, varieties and velocities of data – well, as it turns out, this isn’t certain either.

    Big Data is in its infancy: I think we may be confusing infancy with lack of real traction, and of time and place utility.

    You cannot be serious: Just what are people talking about here? I have read vague, naïve and ill-informed pieces about data management, data architecture, data warehousing, reporting, business intelligence and a plethora of etcetera that have been passed off as observations and commentary on Big Data. So, what makes people recycle hackneyed, misleading and badly conceptualised ‘content’?

    In the commentary on one of Bernard Marr’s pieces on LinkedIn (a professional networking site) I observed that no one can adequately explain what Big Data is without falling into contradictions and fancies, and no one seems to be capable or willing to provide tangible success stories.

    Bernard responded to this comment by pointing out “the reason for that is that Big Data means different things to different people.”

    Fair enough. It’s an explanation.

    That said, I have always had more than a tenuous dislike of postmodern thinking, in fact most things ‘postmodern’. Call me old fashioned, jaded or cynical, but to me, the idea that everything can mean anything is an aberration that I prefer to leave to others.

    I am at a loss to explain why so many reasonable people are willing to embrace the hype surrounding Big Data and Big Data Analytics, including the attendant surfeit of nonsense, incongruences and contradictions, and from my perspective, it defies reason and good sense.

    Therefore, I will just end again with a fabulous quote from Ben Goldacre:

    “You cannot reason people out of a position that they did not reason themselves into”.

    Many thanks for reading.

  • Contradictions of Big Data – Short

    Contradictions of Big Data – Short

    Please note: This is an edited version of a previous piece with a similar name, but focusing solely on the three main Vs of Big Data.

    What we’ve been told

    We’ve been told that business Big Data is the greatest thing since sliced bread, and that its major characteristics are:

    • massive volumes – so great are they that mainstream relational products and technologies such as Oracle, DB2 and Teradata just can’t hack it, and
    • high variety – not only structured data, but also the whole range of digital data, and
    • high velocity – the speed at which data is generated, transmitted and received

    Which is a simple and straightforward means of classification. Big Data is about massive volumes, high variety and high velocity. Right?

    It’s not about big

    I have never bought into the idea that more data is necessarily better data, or that it provides better focus or leads to increased insight, in fact I have been quite vocal with my contrarian opinion, but now this view is getting some additional support, and from some surprising corners.

    In a recent blog piece on IBM’s Big Data and Analytics Hub (Big data: Think Smarter, not bigger), Bernard Marr wrote that “the truth is, it isn’t how big your data is, it’s what you do with it that matters!”

    Over at Fierce Big Data it was Pam Baker who stated that “the term big data is unfortunate because it’s really not about the size of the data”. (Big data is not about petabytes, but complex computing).

    Elsewhere, SAS echoed similar sentiments on their web site: “The real issue is not that you are acquiring large amounts of data. It’s what you do with the data that counts.”

    Well, apparently Big Data isn’t about “massive volumes” of data.

    Strike 1!

    It’s not about variety

    It is claimed that 20% of digital data is structured, it is based on the problematic suggestion that structured data is uniquely relational.

    It is also said that unstructured data includes CSV files and XML data, and this makes up far more than the 20% of the data generated. But this definition is wrong.

    If anything, CSV data is structured, and XML data is highly structured, and it’s typically regular ASCII data. So there it does not add variety, even though it is not structured in the ways that some someone might expect, especially if that someone lacks the required knowledge and experience. Simply stated, CSV data is structured, it’s just that it lacks rich metadata, but that doesn’t make it unstructured.

    “But”, I hear you say “what about all the non-textual data such as multi-media, and what about the masses of unstructured textual data?”

    Take it from me, most businesses will not be basing their business strategies on the analysis of a glut of selfies, juvenile twittering, home videos of cute kittens, or the complete works of William Shakespeare. Almost all business analysis (whether done by a professional statistician or a data scientist) will continue to be carried out using structured data obtained primarily from internal operational systems and external structured data providers.

    Variety, Sir? No problem.

    Strike two!

    It’s not even about velocity

    So, if we accept that Big Data isn’t really about the massive data volumes or high data variety then that leaves us with velocity. Because if it isn’t about record breaking VLDB or significant data variety, then for most commercial businesses the management of data velocity becomes either less of an issue or just is no issue.

    Even in some extreme circumstances, one can explore the suggestion that data sampling can remove issues with data volume as well as velocity.

    However, the fact that some software vendors and IT service suppliers set up this‘straw man’ velocity argument and then knock it down with the ‘amazing powers’ of their products and services, is quite another matter.

    So, is it really about velocity?

    Strike three!

    So what is it really about?

    Big Data is a dopey term, applied necessarily ambiguously to a surfeit of tenuously connected vagaries, and its time has come and gone. Let’s dump the Big Data moniker, and the 3 Vs along with it, and embrace the fact that data is data, there will always be more of it.

    So, let’s consider ‘all data’ and principally for its time and place utility.

    If there is something that you are not sure about or have questions with then please leave a comment below or email me.

    Thanks very much for reading.

  • Consider this: Big Data and the Pot of Tea

    Consider this: Big Data and the Pot of Tea

    To begin at the beginning

    Hold this thought: Big Data is King.

    Is there just nothing that Big Data isn’t capable of fixing? From terrorism, world hunger, Ebola, HIV, fraud, money laundering and hiring the ‘right’ people through to winning the lottery, curing hangovers, arranging entrapment and finding the love of your life. Big Data is King. (more…)

  • The amazing world of Fred’s Big Data

    The amazing world of Fred’s Big Data

    Hold this thought: There are real golden nuggets of data that many organisations are oblivious to. But first let’s look at business process management. (more…)

  • A brief introduction to Knowledge Management

    A brief introduction to Knowledge Management

    A helpful slideset that is used to explain the purposes, positions and roles of Knowledge Management.

    http://www.slideshare.net/MartynInEurope/a-brief-introduction-to-knowledge-management

    Enjoy! Please tell me what you think about this slide deck. Many thanks for viewing. 

  • Big Data, a promised land where the Big Bucks grow

    Big Data, a promised land where the Big Bucks grow

    Consider this. Many people come up to me in the street, and, apropos of nothing, they ask me how they can make money from Big Data.

    Normally I would send such people to see a specialist – no, not a guru, but a sort of health specialist, but because this has happened to me so many times now, I eventually decided to put pen to paper, push the envelope, open up the kimono, and to record my advice for posterity and the great grandchildren.

    So, here are my top seven tips for cashing in quick on the new big thing on the block.

    1 – A business opportunity for faith

    Like every new religion, trend or fad, Big Data has its own founding myths, theology and liturgy, and there is money to be made in it; loadsa lovely jubbly money. By predicating and evangelising Big Data you will be welcomed with open arms into the Big Data faith, and will receive all the attendant benefits that will miraculously and mysteriously fall upon you and your devout friends. Go on, I dare you. Be a Big Data guru, a shepherd to a flock of sheep, and enjoy the wealth, health and happiness that most surely will come your way. You too can look cool in red Prada slippers, a flattering and flowing gown and matching accessories.

    2 – Acquire it, multiply it, weigh it, mark it up and sell it on

    Simply stated, this is about acquiring other people’s data, by sacred means or profane, marking it up and then selling it on. The value you add is that you act as a trusted conduit, a conduit for good. You may care to enrich the data, swop the order of data, replicate and embellish data, make stuff up, etc. which all serves to ‘add value’ to the data. You may even consider adding nuggets of value to the data, just for kicks and giggles. My best friend’s favourite is injecting the good old ‘diaper and beer’ and ‘friends and family’ clichés into every Big Data collection, as it never fails to thrill, please and delight.

    3 – Anything can be anything

    The good thing about making money from Big Data is that it doesn’t need to be anything to do with Big Data. Make a 20GB Enterprise Data Warehouse? Call it a Big Data success. Sell 20 boxes of dodgy doughnuts down the alternative market? Proclaim a Big Data triumph. Sell your digital porn stash to your best mate? Point to the incredible invisible hand of the Big Data market at work. See what I’m doing there. Anything can be anything, and you too can cash in on that opportunity, big time.

    4 – Big Data Patronage

    Tense, nervous headaches? Do you like making up stories about Big Data, or for that matter anything else? Are you a natural born fibber but are strapped for cash? Then worry no longer. If you get a Big Data patron you will be sorted for ‘life’; get two and you’ll be sorted for the afterlife as well. With a Big Data patron you can get the most tenuous, crappiest and superficial of pieces published, promoted and vaunted – globally. Can’t make it up yourself, then outsource and offshore it, after all, just get the keywords right for SEO ranking and the gullible will flock to you in droves. The down side of this profession is that you will be targeted for writing half-truths, quarter-truths and downright lies, and you will be pilloried as a purveyor of rank hyperbole. But don’t worry, take heart and never lose the faith, you will be in good company. As one Big Data guru was want to say ” If you repeat a lie often enough, people will believe it, and you will even come to believe it yourself.” Amen! brother.

    5 – Big Data Certification

    By 2016 there will be global demand for 30 billion Big Data professionals. Are you prepared to cash in on that inevitability? No? Then consider this.

    One of my best friends makes his living as a completely phony Big Data Scientist. For two hundred bucks he can make you a Data Scientist or a Big Data guru. Some guys give you an education but this guy gives you immediate access to high paying jobs, sex and a life in the city. Moreover, for an extra 250 bucks you can also become a certified Big Data Trainer, which will allow you to do unto others what has been done unto you.

    6 – Creative Technology Reuse

    Big Data has heralded in the biggest innovations known in the history of computing, and arguably in the entire history of humankind. One of those new inventions has been the now widely acclaimed and revolutionary ‘flat file data base’ (FFDB), and this has been accompanied with developments in low level operating system primitives that allow for the processing of these collections and hierarchies of FFDBs. So, if one has a mind to do so, one can get some real business leverage off of these new tendencies by borrowing 21st century technology found in old operating system hacks from the sixties and seventies and eighties and nineties and… Well, the point is that in order to get serious funding it is no longer good enough to have a half page business plan, it is also necessary to eke out ‘stuff’ that works within the new paradigms of Big Data and Big Data Analytics. For my next venture I will be looking for serious funding for my ‘Arbitrary Dawdle Down Data Street’ (AD3S) Big Data Analytics platform, a platform designed to support virtual 1k bit processing and the massively parallel provision of global regular expression search and match (S&M), concatenation and listing, and cooperative data-driven and streamed data extraction and reporting. I’m hoping to attract the attention of governments, the EU, the Manic Street Preachers, the UN, China, Vladimir Putin, the DOD, HP, Oracle, Gartner, Lana Del Rey, Deloitte and IBM. So, this is going to be absolutely massive. Word!

    7 – Big Data Brokerage

    According to leading management consultants and industry watchers Gartner, McKinsey and Deloitte, data needs to be managed and accounted like any other asset, such as money. To get into a similar view-point requires a massive leap of faith, but it is a conversion that might drive dividends. One avenue to be explored in eking out value from the apparently massively valuable Big Data lakes, silos and pools is through the operation of a Big Data Brokerage. A Big Data Brokerage is a business whose main responsibility is to be an intermediary that puts Big Data buyers and Big Data sellers together in order to facilitate a transaction. Big Data Brokerage companies are compensated via commission after the Big Data transaction has been successfully completed. They may also charge introductory fees. Just imagine the wealth of business opportunities in that. You could become the Goldman Sachs of data.

    That’s it folks!

    I hope you enjoyed this piece and would be pleased to hear your views on this and other subjects.

    Whilst I understand the attraction and even the need of creating a new and significant growth industry, I would also advise a degree of restraint, and whilst I see that “Big Data” (the consideration of the potential value of All Data) has its allure, I also think that some good sense and informed caution should also prevail.

    Thank you so much for reading.

    Martyn Richard Jones

  • How to position Big Data

    How to position Big Data

    To begin at the beginning

    Fueled by the new fashions on the block, principally Big Data, the Internet of Things, and to a lesser extent Cloud computing, there’s a debate quietly taking please over what statistics is and is not, and where it fits in the whole new brave world of data architecture and management. For this piece I would like to put aspects of this discussion into context, by asking what ‘Core Statistics’ means in the context of the DW 3.0 Information Supply Framework.

    Core Statistics on the DW 3.0 Landscape

    The following diagram illustrates the overall DW 3.0 framework:

    There are three main concepts in this diagram: Data Sources; Core Data Warehousing; and, Core Statistics.

    Data Sources: All current sources, varieties, velocities and volumes of data available.

    Core Data Warehousing: All required content, including data, information and outcomes derived from statistical analysis.

    Core Statistics: This is the body of statistical competence, and the data used by that competence. A key data component of Core Statistics is the Analytics Data Store, which is designed to support the requirements of statisticians.

    The focus of this piece is on Core Statistics. It briefly looks at the aspect of demand driven data provisioning for statistical analysis and what ‘statistics’ means in the context of the DW 3.0 framework.

    Demand Driven Data Provisioning

    The DW 3.0 Information Supply Framework isn’t primarily about statistics it’s about data supply. However, the provision of adequate, appropriate and timely demand-driven data to statisticians for statistical analysis is very much an integral part of the DW 3.0 philosophy, framework and architecture.

    Within DW 3.0 there are a number of key activities and artifacts that support the effective functioning of all associated processes. Here are some examples:

    All Data Investigation: An activity centre that carries out research into potential new sources of data and analyses the effectiveness of existing sources of data and its usage. It is also responsible for identifying markets for data owned by the organization.

    All Data Brokerage: An activity that focuses on all aspects of matching data demand to data supply, including negotiating supply, service levels and quality agreements with data suppliers and data users. It also deals with contractual and technical arrangements to supply data to corporate subsidiaries and external data customers.

    All Data Quality: Much of the requirements for clean and useable data, regardless of data volumes, variety and velocity, have been addressed by methods, tools and techniques developed over the last four decades. Data migration, data conversion, data integration, and data warehousing have all brought about advances in the field of data quality. The All Data Quality function focuses on providing quality in all aspects of information supply, including data quality, data suitability, quality and appropriateness of data structures, and data use.

    All Data Catalogue: The creation and maintenance of a catalogue of internal and external sources of data, its provenance, quality, format, etc. It is compiled based on explicit demand and implicit anticipation of demand, and is the result of an active scanning of the ‘data markets’, ‘potential new sources’ of data and existing and emerging data suppliers.

    All Data Inventory: This is a subset of the All Data Catalogue. It identifies, describes and quantifies the data in terms of a full range of metadata elements, including provenance, quality, and transformation rules. It encompasses business, management and technical metadata; usage data; and, qualitative and quantitative contribution data.

    Of course there are many more activities and artifacts involved in the overall DW 3.0 framework.

    Yes, but is it all statistics?

    Statistics, it is said, is the study of the collection, organization, analysis, interpretation and presentation of data. It deals with all aspects of data, including the planning of data collection in terms of the design of surveys and experiments; learning from data, and of measuring, controlling, and communicating uncertainty; and it provides the navigation essential for controlling the course of scientific and societal advances[i]. It is also about applying statistical thinking and methods to a wide variety of scientific, social, and business endeavors in such areas as astronomy, biology, education, economics, engineering, genetics, marketing, medicine, psychology, public health, sports, among many.

    Core Statistics supports micro and macro oriented statistical data, and metadata for syntactical projection (representation-orientation); semantic projection (content-orientation); and, pragmatic projection (purpose-orientation).

    The Core Statistics approach provides a full range of data artifacts, logistics and controls to meet an ever growing and varied demand for data to support the statistician, including the areas of data mining and predictive analytics. Moreover, and this is going to be tough for some people to accept, the focus of Core Statistics is on professional statistical analysis of all relevant data of all varieties, volumes and velocities, and not, for example, on the fanciful and unsubstantiated data requirements of amateur ‘analysts’ and ‘scientists’ dedicated to finding causation free correlations and interesting shapes in clouds.

    That’s all folks

    This has been a brief look at the role of DW 3.0 in supplying data to statisticians.

    One key aspect of the Core Statistics element of the DW 3.0 framework is that it renders irrelevant the hyperbolic claims that statisticians are not equipped to deal with data variety, volumes and velocity.

    Even with the advent of Big Data alchemy is still alchemy, and data analysis is still about statistics.

    If you have any questions about this aspect of the framework then please feel free to contact me, or to leave a comment below.

    Many thanks for reading.

    Catalogue under: #bigdata #technology big data, predictive analytics

    [i] Davidian, M. and Louis, T. A., 10.1126/science.1218685

  • Big Data Will Save the World

    Big Data Will Save the World

    Good morning fellow consumers; here’s a pop quiz question: What does Big Data have in common with Robitussin? Think about, take your time.

    Okay, times up!

    Robitussin is a legal pharmaceutical product commonly associated with coughs, colds and flu combinations. (more…)

  • The Big Data ‘Wow Wow’ Factor

    The Big Data ‘Wow Wow’ Factor

    wFactor

    The Wow Wow Factor! Trading, Big Data and 7 HabitsHi, I’m Ricky Jones, boss and co-founder of Becci Boo International Investments. Last week we said goodbye to our best ever Big Data energy commodity trader. He’d been with us for years.

    Sadly, Coco Jones was determined to retire to the countryside, to his birthplace, to his real home, a snug little village in the hills of Montseny, and there was absolutely nothing we could do to convince him to stay. The thing about Coco is that he is not like you or me, he’s a highly intelligent Catalan sheepdog.

    So, you might ask, how did Coco get to be a star trader at the Becci Boo Hedge Fund? Was it the tools he used? Was it the techniques he adopted? Was it the food he ate? What was so special about him and his amazing abilities? It’s a long story that I will relate as briefly as I can.

    Back in time, there was one particularly disastrous week of trading at Becci Boo. Something had gone really wrong with our once reliable Big Data Trade Analytics platform, and wrong bets were being placed right, left and centre – and against trader’s better judgement. The CFO was livid. Out he comes onto the trading floor, swearing and blinding. “God! You guys are the damn pits! What the hell do you think you are doing? Can’t you get anything right? A Catalan sheepdog could trade more effectively than you feckless lot of feckless things.”

    I try and diffuse the situation. “Come on, Jordi, don’t be like that, we’re only human and this is a tough business.” “You don’t believe me” he replies. “Sure, but you’re not going to convince me that a Catalan sheepdog could be a substitute for a highly experienced human trader are you?” “How much do you want to bet? These superior four legged beings would never place any faith in things that they don’t understand or can’t control, and they certainly wouldn’t need your Big Data gizmo to make a success of things.”

    So, in the following weeks we arrange to run an experiment. We ring around all the owners of registered Catalan sheepdogs and ask them if they would like their dog to take part in our simple trading experiment. Food, lodging and generous expenses all included. We easily manage to get together 64 Catalan sheepdogs and their owners from all over Europe. The experiment we design is quite simple.

    We give each dog a gadget (and actually for the more sophisticated dogs it was a smart phone and specially designed app) with two big buttons on it, a red button and a yellow button.

    Every Monday morning we ask the assembled dogs to press one of two buttons depending on whether they think that the chosen energy commodities market will close higher or lower at the end of the week.

    Red button for a higher closing price, yellow button for a lower closing price. As you might imagine dear reader it’s a walk in the park for these cunning canines. To make sure that all of the participating dogs have equal and fair chances in the experiment, we provide guardians with access to all our big data analytics, and we have multiple screens set up in each studio apartment so that each dog can follow all of the financial news that they need to take in and then view the results of machine learning, data mining and predictive analytics.

    At the end of week one, 32 dogs remain in the experiment, and the other 32 are sadly sent home. Every week the exercise is repeated, and every week less dogs move on to the next round. By the end of week 5 only two dogs remain. By the end of week 6 we have a sole winner, Afi Bastò, who has accurately predicted the market movement of our energy commodities market for a record six weeks in succession. It’s an amazing success that proves Jordi right. Unfortunately, this is where things start to go wrong. Our marketing department tweets the news of the amazing Afi and it immediately goes viral.

    Articles appear in the Wall Street Journal, Financial Times, Economist, Cinco Dias and Les Échos, hailing the amazing brilliance of the newly discovered Catalan commodity trader.

    Hundreds of interviews are held and millions of photos make the rounds of the social and professional networks. Over the following year Afi is interviewed, researched, studied and investigated. Academic papers are written about him.

    Amazing claims are made about his canine knowledge, wisdom and experience. This is shortly followed by the publication of a plethora of bestselling business books, with titles such as: The 7 Habits of the Highly Effective Canine – Personal Lessons in Trading; Be More Afi and Grow Rich; The Gos d’Atura That Conquered Chicago; Learn to Trade like Afi; Good Sheepdog, Great Afi; Who Moved My Dog Food; How Afi outperformed Big Data Analytics; etc.

    Which unfortunately all goes to Afi’s head, and he begins to seriously lose his commodity trading mojo. So much so, that by the end of the year, we let Afi go, and we bring back “the incredible” Coco Jones, the dog that managed to get the market movements right for five weeks in a row, and who was only pipped at the post by “the amazing” Afi.

    Since then, Coco has helped us to win far more than we lose, he has had an innate knack of being able to combine market knowledge with statistical analysis and a certain indescribable ‘insight’ that no one quite manages to understand never mind emulate, and all combined with an amazing caring character.

    So, as you might imagine, we were really sad to see him go. After his epic leaving party, where he is acclaimed by all and sundry, but especially by Jordi, all the staff assembled outside the office entrance to see him off, and as he was slowly chauffeured away down the driveway he turned and looked back, and his face said it all. “I was lucky — I found what I loved to do early in life”.

    Many thanks for reading.  

    Here’s Coco paddling in his Big Data Lake

    OLYMPUS DIGITAL CAMERA

    File under: Good Strat, Good Strategy, Martyn Richard Jones, Cambriano Energy, Iniciativa Consulting, Iniciativa para Data Warehouse, Tiki Taka Pro

  • Big Data in Social Studies

    Big Data in Social Studies

    You walk down the street, maybe in London, Paris and Rome – well, of course, not all at the same time.

    You look out of your spacious and luxurious apartment in Manhattan, Unter den Linden or Mahim.

    You are cycling slowly along the streets of Amsterdam, Bonn or Zurich.

    Got the picture? (more…)