Martyn Richard Jones
Dublin 10th May 2017
Throw AI, Big Data and Data Science into a pot, and what have you got?
Yes. A pig’s breakfast, not fit for a pig.
Martyn Richard Jones
I first became involved in commercial analytics in the eighties. First, through my involvement in customer segmentation and data visualisation, principally in banking but also in energy, manufacturing and the chemical industry. It also emerged later, in conjunction with my activities at the Sperry European Centre for AI, and was centred on pricing and yield management applications developed using a combination of statistical techniques, expert system technology and data centre architectures all tightly integrated within a 4GL development and delivery environment. This provided a comprehensive and seamless scenario building, hypothesis testing and reporting capability. Continue reading
In the beginning was the Big Iron, the Big Data, and the Big Data Plan.
And then came the Big Data Assumptions.
And the Big Data Assumptions were without form.
And the Big Data Plan was without substance.
And the Big Iron was without movement.
And the Big Data was without velocity, variety and volume.
And darkness was upon the face of the data workers.
And they spoke amongst themselves, saying: “Big Data, is a crock of shit, and it stinketh mucho”.
And the data workers went unto their Data Supervisors and said: “This here Big Data is a pile of putrid crappy keech”, for they were from Govan, and continued, “and none may abide the odour thereof”.
And the Data Supervisors went unto their Information Managers, saying: “Big Data is a container of excrement, and it is very strong, such that none may abide by it.”
And the Information Managers went unto their Business Directors, saying: “This here Big Data doodoo is a vessel of fertilizer, and none may abide its strength.”
And the Business Directors spoke amongst themselves, saying to one another: “Big Data contains that which aids plant growth, and it is very powerful.”
And the Vice Presidents went unto the President, saying unto him: “This new Big Data will actively promote the growth and vigour of the company, with powerful effects.”
And the President looked upon the Big Iron, the Big Data and the Big Data Plan, and saw that they were good.
Many thanks for reading
Join The Big Data Contrarians
If you know all about Sentiment Analysis, you’ve come to the right place. Because I don’t have a clue if what I know about it is accurate or not.
I started to do a bit research into this Sentiment Analysis lark, in particular with the theoretical idea of using it to analyse and draw conclusions from comments on Pulse – assuming that this is what it can be used for.
To begin at the beginning, which is good place to start, I read the piece on Wikipedia, and this was how it began:
“Sentiment analysis (also known as opinion mining) refers to the use of natural language processing, text analysis and computational linguistics to identify and extract subjective information in source materials.
Generally speaking, sentiment analysis aims to determine the attitude of a speaker or a writer with respect to some topic or the overall contextual polarity of a document. The attitude may be his or her judgment or evaluation (see appraisal theory), affective state (that is to say, the emotional state of the author when writing), or the intended emotional communication (that is to say, the emotional effect the author wishes to have on the reader).” Source: Wikipedia Link:http://en.wikipedia.org/wiki/Sentiment_analysis
Well, that’s a fairly intuitive description. I could have almost have guessed as much.
But, back to the aim of analysing sentiment in Pulse comments, where to start and what to do.
What would sentiment analysis make of these:
On the death of an IT-business celebrity. What would sentiment analysis make of the very emotive comments of desolation, sadness and poignancy of people who didn’t personally know the departed, even remotely, or maybe didn’t even know of them until after they had ‘shuffled off life’s mortal coil’? How would that work? What would sentiment analysis make of the maudlin aphorisms, surrogate grief and bizarre sorrow of people separated by more degrees than Kofi Anan and Mork from Ork. What additional insight does sentiment analysis tell us when these comments are analysed along with the body of the text and other comments that triggers these comments?
In a similar vein, how does sentiment analysis catch instances of sycophancy? Especially considering the fact that some of it is so ‘in your face’ and blatant that it often times seems to be a bad parody of a bad parody. “Oh, Ricky, why are you such a sexy brainbox?” How does it work in those situations?
Worse than that is the preening, gushing and obtuse texts of massive, errm… fabulators[i]. If it wasn’t about Big Data or Strategy or IT, it would be about something else, usually about the writer themselves. “I give Rafa and Rodge tips on tennis! I went to the University of the Universe and got a first! I challenged Superman to a race, and won! I have read the entire works of Dan Brown, 25 times…Neeeh!” What would sentiment analysis do with that sort of gold?
Also, what does sentiment analysis do with texts so ambiguously daft that they could mean anything? Okay, it might be able to pick up a few trigger words here or there, “rubbish”, “of”, “load”, “a”, “what”, etc. However, how does it know when “excellent” is being used in a way that means anything but excellent? For example, “Excellent Big Data job there”, with the silent “if you want a job doing properly then do it yourself”.
Finally, for the purpose of this little piece, what would sentiment analysis do with term abuse, if it could actually identify it? Going back to the use of the terms such as Big Data or Strategy, how can sentiment analysis discern between the dopey and wrong-headed use of the term, and when it is actually being used in a coherent, cohesive and consistent way, in line more or less with its formal definition? I suppose we can always write a mountain of rules to help us out:
If topic in focus of piece is strategy
And context of topic is business
And author of piece is Richard Rumelt
Then the credibility of text is good (with a certainty of 100%)
But you and try and maintain a rule base with isntances like that. It soon becomes a management nightmare.
Alternatively, maybe it could be used to analyse this text. It’ll have its work cut out, that’s for sure. Does sentiment analysis do sarcasm and cynicsm?
Anyway! I bet you might know how this sentiment analysis works, don’t you? On the other hand, if not, then it will be someone else who ‘knows’. But of course, all will not be revealed, because it’s a secret so powerful, that in the wrong hands it could be used to dominate the entire galaxy.
Only joking; and many thanks for reading.
[i]To engage in the composition of fables or stories, especially those featuring a strong element of fantasy: “a land which … had given itself up to dreaming, to fabulating, to tale-telling” (Lawrence Durrell).