Tag Archives: information management

The amazing world of Fred’s Big Data

15 Sunday Feb 2015

Posted by in Big Data, Consider this

≈ Leave a comment

Tags

, , , , , , ,

Hold this thought: There are real golden nuggets of data that many organisations are oblivious to. But first let’s look at business process management.

Business process management, what is there not to like. It has revolutionised the way we organise and do business. Right?

Your business is applying, measuring and managing the process, right?

The process is king, it’s a great process, and your organisation loves it, right?

There is no process other than the known, approved, socialised and proven process, right?

Well, we’ll see about that.

You may think that your best data is in the enterprise data warehouse (Bill or Ralph). I will not argue with that.

You may consider that the gold rush starts and ends in the exploitation of Big Data (what I like to call All Data). You may have a point, or not.

You may speculate that your best data may come from external social media sources? Well, that’s a commonly held view, and who am I to challenge a commonly held view, for as erroneous and popularly acclaimed as it may be.

But, if you thought all this were true, would you be right?

Now, consider this…

In companies, good, mediocre and bad, people follow processes, they follow these processes as best as they can until they work out how to do something better, faster, cheaper and easier. They may even skip process points completely or simply rearrange them, and in ways that business analysts never thought of. You see, processes can be clever, but people are really smart. But we knew that already.

People can be astute, honestly and innovatively so, and businesses function as well as they do, not only because of the assets, systems or the processes, or even due to the capital of the business, but because of the people who work for them, and conspire together to make things run effectively.

This fact now stated opens up the whole world of hidden data and knowledge, the real data gems of the business world. I mean ‘business world’ in the widest sense of all the businesses all around the world, whether for profit, non-profit or public.

From the top to the bottom of the organisation, in the matrix, left to the right, there is a wealth of data that refuses to be digitised, evades formal capture and defies measurement.

In my precocious youth I used to refer to this data as the “data that falls between the cracks”, and would got weird looks from my management as a result.

My first encounter with “data that falls between the cracks” was with Fred’s Book.

Fred’s Book was the mother of parallel processes, and then some. In my youth I was asked to model an industrial process in a factory. My focus was on logistics, production and distribution. In the course of my investigations I discovered some glaring anomalies between the senior management views of the process and the actual interpretation of that process. I dug deeper. Formally, I was told nothing that really veered from the official business operating model. Of course, it was different story after a couple of beers on a Friday night.

It turned out there was a whole sub-process in play, and that it was all being coordinated through Fred’s Book, an almanac of all things related to the off-book business process, curated by Fred, a shop-steward and supervisor of one of the departments at the core of the process.

I interviewed Fred to find out more of the fascinating process that he was curating, but at the end of a long session he asked me not to reveal his ‘book’ to senior management.

Fred’s book was the glue that held part of the business process together, it was the data, information and knowledge that kept things working in spite of inadequacies and contradictions in the formal process. But later I found out there were more ‘books’. There was Tommy’s book in distribution, Lily’s book in production, Maria’s book in quality, and Harry’s factory floor management cookbook. The whole business was running on an efficient and effective parallel process, documented in living notebooks (the paper kind), notebooks that underpinned and provided a workable framework for CRM, MIS and ERP.

Over the years I have encountered various artefacts of the Fred’s Book kind in many organisations. Not only that, sometimes Fred’s Book is virtual and codified, making it even less tangible to the digital data twitchers and information botherers. But it still has unmeasured value. We know it has value, but we might not be sure as to how much value. Which in general is more than can be said for faddish Big Data. The personal knowledge, experience and wisdom that people bring to bear in their working and private lives is one of the most valuable assets that a business potentially has access to, even if they don’t know it or incapable of appreciating what they have and of then encouraging people to share and apply that knowledge and experience, wisely.

So, how do we ensure we don’t lose Fred’s knowledge and experience? How do we stop the good sense from leaking out of organisations? How do we ensure that we don’t throw out the knowledge baby and keep the Big Data bath water?

Driving business agility, competitiveness and strength requires far more than Data Warehousing, Big Data, Business Intelligence, Machine Learning, or Analytics and Data Science. It takes far more than supervision by numbers, shapes and colours; decisions based on sophisticated coin flipping; or, the voodoo analytics of the causation-free data bodgers. It needs the data, information and knowledge that resides in people’s heads and the wisdom needed to know when, where and how to apply that knowledge. That’s where the real golden nuggets are.

This is why I am so sceptical of the claims of Big Data “gurus” and other assorted charlatans, who seem to believe in their Homeric powers to move whole armies across the Alps with the click of a mouse, and just backed up with the invisible safety-net of the promised benefits accruable from exabytes of trashy social media feeds and bloated internet logs.

Historian Paul Kennedy, while acknowledging that no single variable can explain success, maintains the metanarrative that wars are ultimately won by a superior organization imbued with a culture of innovation that actively encourages inquiry, experimentation, and interdisciplinary problem solving.

This to me means that an organisation must focus on what is important, what it is good at, and what it has that is really important, and then apply it.

For many companies diverting the corporate attention from the important to the peripheral is a bad idea, that comes with many risks, and if that marginal and peripheral shiny-thing turns out to be large volumes, varieties and velocities of data with little or no value, then even more reason not to get caught up in the facile, superficial and peripheral issues of little or no value.

I like football, many people do. I support two teams, Real Madrid and Cordoba. When Real Madrid play FC Barcelona what preparations are made in advance? What is the focus of the preparation? Some people say that fan motivation helps the team to perform better, so maybe we should analyse the entire global fan base of Barcelona, Big Data style, in order to work out how to influence the influencers. But would I suggest such a tactic to any of the big clubs? Of course not, they would tell me to go away, and quite rightly so.

Adland legend Dave Trott tells a great story about former Leeds United football club manager Don Revie:

Don Revie had a dossier on all referees.
He used to make his teams study the referee for the upcoming game, so they could ask about his family, his hobbies, etc.
Most teams get 50% of decisions go their way.
Leeds used to get 80% of decisions go their way.

No Big Data Kool-Aid, no Data Science, no voodoo analytics, just good old fashioned knowledge, experience and street smarts.

Many thanks for reading.

Advertisements

Consider this: Big Data in Context

21 Wednesday Jan 2015

Posted by in Big Data, Consider this, Data Warehouse, Data Warehousing

≈ Leave a comment

Tags

, , , , , , ,

Big Data, together with Cloud computing and the Internet of Things, are topics that are very much to the fore in contemporary trends in Information Management.

But is Big Data really the revolution that people have been waiting for or is it simply about the next steps in the evolution of business data architecture and management? And whilst it is true that we are increasingly generating more and more data, do we really need to rethink our whole approach to data demand and data supply, or can we build on the best of what we have already accomplished in order to move with intelligence, rigour and stability to the next level?

We are told that all organisations will be overwhelmed with an existential need to leverage all data – massive data, in all formats, all volumes and quality, and which is obtained by all legal means. But is this really the case?

I am not entirely convinced by the breadth or depth or intellectual veracity of the claims, which frequently border the hyperbolic and the absurd, but I also think it is wise to be forearmed as well as forewarned.

That is why I am encouraging a discussion about where Big Data and Data Science fit into a much larger picture of business information supply and demand.

To begin at the beginning

There are many dimensions to business data and information, and here I will touch on some of those dimensions here, albeit with a broad brush.

There are also many potential sources of data; these can be divided into two broad dimensions: analogue data and digital data.

Analogue data – Any type of data not stored in a digital format. This covers the data found in analogue form. For the purpose of this piece this dimension is overlooked.

Digital data – Any type of data stored in a digital format. The source may be a conventional database, a log file, and audit trail, a spreadsheet, document, presentation, project plan or other similar source.

What is being contemplated is data that could help us to interpret history; manage the present; and, plan for the future.

Past – What happened and what can we learn from it?

Present – What is happening and what, if anything, should we do differently?

Future – What might or might not happen and what might we do or not do if this happens, or what can we do to prevent this ‘something’ from happening?

If the data in focus cannot help us with any of these dimensions does that data really matter?

The third dimension of data that I want to identify concerns place (internal and external):

Internal data – This is the information that belongs to the organisation and is obtained from internal sources, such as operational systems.

External data – This is the data from external sources, such as market data providers. that an organisation can legally and legitimately acquire and use.

And finally, for the purpose of brevity, the dimensions of demand and supply:

Demand driven data – This is when business is the driver for the flow of information. Data is delivered because there is a business demand. In simple terms this is about giving people what they want, continuously.

Supply driven data – This is about delivering data and information based upon what is available. If done on the right scale and with the aim of creating new internal markets for data, then it can be beneficial. This is where IT needs to be far more sales, marketing and Ad-land oriented.

But what do all of these considerations mean to a managers need for adequate, appropriate and timely information? Let’s take a look.

The following diagram illustrates the overall drivers for the Information Supply Framework.

From the right, the consumers and prospective consumers of data and information create the demand for information and data.

The middle process of ‘data processing, enrichment and information creation’ strives to meet the business demands and also ‘provokes’ business demands for data and information.

From the left the data sources provide data to meet real and secondary demands.

So how do we ‘make it happen’?

How do we try and obtain the business advantages supposedly accruable to Big Data without succumbing to yet another anarchic information management aberration that will take us one step forward, and two steps back?

Let me explain.

Is this the sort of diagram that gives you a warm and fuzzy feeling?

Of course not, it’s a mess. Print it out on a plotter and place it on the office wall, by all means. You can even call it art, but it’s also a recipe for data bedlam hell.

Now contrast that colourful fun-loving anarchical approach to something I prepared earlier.

There, isn’t that better?

It’s clean, cohesive and coherent. It’s a rational, structurally sound and practical model to support the business demands for data and for the speculative supply of data. The model is based on unifying strands of business data requirements, (with specific regard to operational, tactical and strategic aspects,) into to an integrated, agile and flexible whole.

In short, it is a coalescing model for business data and information.

That’s all folks

Is it absolutely unnecessary in this day and age to take a maverick, piecemeal and DIY approach to the provision of non-traditional business data and information?

No, evidently it’s not.

There is no rational business reason to use speculative, quick and dirty – and characteristically ‘throw away’ approaches to data supply, when quick, clean, useful, maintainable, cost effective and usable alternatives are available.

The Information Supply Framework provides a strategic approach to meeting the data and information needs of an organisation. It accurately reflects business process, it can change in step with the changing needs of the organisation, and it is capable of handling the emerging requirements associated with ever increasing data volumes, data variety and data velocity.

Simply stated, the DW 3.0 Information Supply Framework provides a nascent opportunity for businesses to easily transition to the inclusion of more formal and rigorous methods of statistical analysis, analysis that can now be powered by the inclusion of new, alternative and complementary sources of data.

Naturally the choice is yours. But ensure that you weigh up your options with your ‘eyes wide open’, or it may well all end in tears.

Many thanks for reading.

File under: Good Strat, Good Strategy, Martyn Richard Jones, Martyn Jones, Cambriano Energy, Iniciativa Consulting, Iniciativa para Data Warehouse, Tiki Taka Pro

Artefacts for Enterprise Information Management

01 Monday Dec 2014

Posted by in Ask Martyn

≈ Leave a comment

Tags

,

Artefacts for Enterprise Information Management

Key Data Warehousing Reference Architecture Components

Martyn Richard Jones

As an effective business process paradigm and a powerful mixture of technology, engineering and pragmatic design, Enterprise Data Warehousing (EDW) is arguably unmatched in its capacity to address complex and essential information requirements in Management Reporting, Business Intelligence (BI) and Data Analytics. In spite of the occasional outbreak of anecdotal ruminations to the contrary, key indications point to a bright future for EDW. Indeed, the breadth and richness of nascent applications, that the EDW model can adequately support, will drive an expansion in its utility, acceptance and advancement. For instance, a significant and practical way of maximising the value of innovation and investment in Enterprise Data Warehousing is by reusing and extending the Inmon paradigm to construct the enterprise information hub of the future.

With more imagination in our perception and interpretation of the EDW paradigm, and a greater sense of purpose in the way we align data demand and supply factors with the underlying analysis, development and delivery mechanisms, we can reach a better understanding of how a variety of information requirements can be satisfied through innovative uses of the EDW model. With a broader vision of the role of EDW in organisations, we can identify additional business requirements that EDW is more than adequately capable of accommodating. Indeed, if we look at the broadened market of potential EDW clients we can encounter applications in domains such as next generation Customer Relationship Management (CRM), Master Data Management (MDM), Service Oriented Architectures (SOA), Business Performance Management (BPM), Customer Data Integration (CDI) and Advanced Data Correlation (ADC).

Now, in order for the EDW model to continue to be successful, and especially in the contemporary areas of applicability, it must meet these new and evolving requirements, simultaneously and visibly, whilst satisfactorily continuing to support its central role in the field of consistent, reliable and timely data analytics and reporting. In this respect, the EDW model provides the required coherency, flexibility and usability in a framework that permits the symbiotic coexistence of the iterative requirements of the traditional users of EDW (management reporting, departmental data marts, etc.) with the provisioning of data for new services. At the same time, this approach can satisfactorily address the requirements that are emerging from the disparate and pressing needs for low-latency and high-agility data integration in commercial and governmental organisations.

So, what does this mean in practical terms? Probably a simple way of summarising the approach is by stating that the best practices, principles and technologies of the Enterprise Data Warehousing paradigm have further applicability in effectively addressing a variety of new requirements, and in ways that help organisations obtain, retain and reinforce cohesion and coherence in enterprise information management.

Simply stated, you have this great approach for building data warehouses, but now you can use and extend this great approach to deliver requirements beyond the boundaries of traditional EDW. The inherent advantages accruable from this set of happy coincidences are necessarily tangible to a broad spectrum of the organisation, from EDW architect, EDW PM and EDW developer through to CIO, CFO and CEO.

This all very well and good in principle, but how can an organisation can kick-start its way into EDW futures? One practical way to ensure that the core framework for EDW will support certain classes of emerging applications – as well as continuing to support the existing and growing demands on the EDW’s core competence – is by leveraging the best principles and practices of EDW itself. To this end, the usefulness of the concept is most clear in four predominant groups of artefacts used to assist enterprises in the successful implementation and evolution of its EDW:

  1. Reference architectures
  2. Development methodologies
  3. Industry models and solution frameworks
  4. Construction components

The Reference Architecture

According to IBM, a reference architecture is, in essence, a predefined architectural pattern, or set of patterns, possibly partially or completely instantiated, designed, and proven for use in particular business and technical contexts, together with supporting artefacts to enable their use. Often, these artefacts are products of lessons learned from previous projects.

Alternatively, for companies such as IBM, Teradata and Oracle/Sun Microsystems, etc. reference architectures primarily consist of storage, servers and software. What is common to almost all reference architectures is that they are solutions that have worked satisfactorily in a given set of circumstances.

Reference architectures for Enterprise Data Warehousing should be comprehensive models of all required components and artefacts, and should explicitly state how the pieces of the architectural whole, fit together, how the data flows through the architecture and how the architecture sits in the current and future IT landscape of the organisation. What does this mean in practice? Think of a house, a comprehensive architecture should not stop at identifying rooms, functions of rooms, doors, windows, walls and the roof – this is good start, and helps us to get an idea of the house we are conceptualising, that is where its usefulness starts. Just as we would not want to build a house, simply be following a sketchy outline. The same goes for the data warehouse, and although at a trivial level a reference architecture looks like a series of boxes and arrows, it does not stop there.

Development methodologies

It is imperative that we learn to build the EDW using development methodologies specifically designed for iterative development project cycles. Indeed, trying to use a methodology, such as the System Development Lifecycle (SDLC), in an EDW project, is a damaging, costly and senseless exercise in vanity.

There are public domain iterative methodologies that we can use as the basis for our approach to EDW project iterations. Alternatively, it is possible to license data warehousing methodologies from a number of vendors and consulting service providers. One of the most comprehensive enterprise data warehousing methodologies available today is IBM’s Iterations process model. It has the best pedigree of all Data Warehousing methodologies. Bill Inmon’s company Prism Solutions released the first commercial version of Iterations to the market in the mid-nineties.

The key development phases, common to a number of methodologies, are:

  • Management – Ensures and plans for training, support, project management, change management, internal marketing and ongoing administration of the data warehouse or data mart
  • Analysis – The assessment, scoping and modelling of potential target database solutions, source system solutions, data availability, cleanliness and completeness. The provision of high-level technical recommendations
  • Design – Designing the data environment, data access environment, data extraction environment, maintenance-processing environment and the detailed technical environment.
  • Construction – The building and unit testing of the data extraction solutions, data access solutions, maintenance-processing solutions, the technical environment and the development of end-user training.
  • Testing – The performance of various levels of data warehouse testing, including systems, integration testing and user acceptance testing.
  • Implementation – The process of making the BI infrastructure accessible. The process of putting a developed iteration into the production environment.

So, what is the purpose of using development methodologies?

There are many explanation of the role of development methodologies, but one definition that is simple, clear and comprehensive defines it as as “a means of improving the management and control of the software development process, structuring and simplifying the process, and standardizing the development process and product by specifying activities to be done and techniques to be used”[1].

  • Shortening the time to realize return-on-investment
  • Minimizing the risk of project failure by getting it right the first time
  • Gaining executive sponsorship and endorsement early in the project
  • Setting realistic end-user expectations
  • Identifying critical resources
  • Providing a roadmap to simplify project management
  • Allowing the organization to focus on managing processes; not inventing them
  • Mitigating the impact of team attrition by providing clear and comprehensive documentation
  • Enabling efficient management reporting

Industry Data Models and Solution Frameworks

So, what is the purpose of EDW data models and EDW frameworks and templates? In addition, what are industry data models?

Industry models (such as the leading models from IBM) combine deep expertise and industry best practice in a usable ‘blueprint’ form for both business and IT communities to allow them to accelerate the requirements-to-execution development span of industry solutions. The best of these industry models have been using the experience of hundreds of implementations, and decades of development.

A sound way to appreciate the potential usefulness of an industry data model is by first understanding the core reasons behind its structures, the components and facets of the structures themselves and the logic of how it can be used to store, map, classify, relate, select and retrieve data.

To this end, vendors aim to provide “comprehensive data models containing data warehouse design models, business terminology model and analysis templates to accelerate the development of business intelligence applications. Second, best practice business process models with supportive service definitions for development of a service-oriented architecture (SOA)”. Which, when applied correctly, should result in the acceleration of Data Warehouse and Data Integration project iteration time-frames and also reduced risk.

Construction Components

We could consider Data Marts, an Operational Data Store and the Data Warehouse itself as construction components. However, this section is principally about the componentisation of Data Quality and ETL processes.

I do not intend to dwell too much on this aspect; primarily this is the facet out of all of those I have covered here, that most people will be familiar with and will be aware of the options available.

ETL

We can componentise a significant proportion of Data Warehouse development. Especially components related to ETL. Many of these components (which take the forms of widgets, patterns, templates or applets) are available from ETL vendors. Useful ETL templates and widgets are also available in the public domain.

ETL job stream or project components includes Source, Transformation, Lookup, Staging, Destination, and Loader components.

  • Source components – transport data for a transformation stream. An ETL job stream or project normally starts with one or more source components. Source components have at least one output port.
  • Transformation components, Lookup components, and Staging components – apply specific transformations to the data in the transformation stream. These types of components have both input ports and output ports.
  • Target components or Destination components (also called data sinks) – write data to specific targets. Target components have one input port and no output ports.
  • Loader components – extract and load data from a source database or file into the database, without performing any transformation. Oracle’s SQL*Loader was an example of a loader component.

Data Quality

In terms of Data Quality, there are a number of components available for cleaning and quality checking the veracity of data. For example, Data Quality Components for Microsoft’s SSIS is a suite of custom data cleansing transformation components for Microsoft SQL Server Integration Services (SSIS) to “standardize, verify, correct, consolidate and update all your contact data for effective communications”. Other similar component features are available from other Data Integration vendors, such as Informatica, Oracle, SAP, IBM and Talend.

One interesting offer that I would like to highlight are the InfoSphere Warehouse Packs from IBM, which cover aspects such as Customer Insight, Market and Campaign Insight, and, Supply Chain Insight.

Component Questions

What is the purpose of the construction components? 

They provide plug, configure, program and play options to speed development, testing, acceptance and deployment.

What are the EDW construction components?

They can be widgets, templates, patterns, apps or macros.

What is the purpose of components? 

Speed and ease development. Leverage reuse of work already done, tested and proved.

Conclusions

In summary, we will see that organisations that adopt, nurture and evolve the principles, practices and technologies contained in proven EDW reference architectures, industry data models and construction components, will come to benefit from an evolving EDW paradigm that is both useful and usable. In this way, the most responsive of organisations will be able to verify, for themselves, that the EDW paradigm has a great future as a determining factor in the iterative improvement of productivity, resilience and cost effectiveness in the provision of agile collaboration, usable innovation and process accuracy in the formulation of options and the execution of decisions.

What does that mean in plain English?

  • Commit to Data Warehousing only if there is a sustainable business imperative.
  • Stick to the core DW fundamentals as defined by Bill Inmon. This approach does not clash with the super-advanced business-oriented dimensional modelling of Ralph Kimball, or the Data Vault approach of Dan Linstead.
  • Use a proven iterative methodology specifically designed for Data Warehousing and Data Integration.
  • Leverage anything and everything that will help you to reliably, cost-effectively and coherently analyse, build, test and deliver your Data Warehouse iterations.
  • Use outsourcing and offshoring for purely non-Agile technical aspects of Data Warehousing described exhaustively and unambiguously in development documentation.
  • Continually test your Data Warehouse to ensure that you are not at risk of straying into the twilight zone of Information Management failure.
  • Data Warehousing, done right, will give IT a good name amongst business people.

About the author: Martyn Richard Jones is an Enterprise Data Warehousing (EDW) and Business Intelligence (BI) Project Manager and Architect with over 25 years experience in information integration, knowledge management and management information systems, gained primarily in the financial services and telecommunications business sectors. Martyn can be contacted by email: martyn.jones@cambriano.es 

[1] Nancy L. Russo, The Use and Adaption of System Development Methodologies

File under: Good Strat, Good Strategy, Martyn Richard Jones, Martyn Jones, Cambriano Energy, Iniciativa Consulting, Iniciativa para Data Warehouse, Tiki Taka Pro

The management and architecture of Information Assets: Ask Martyn!

15 Saturday Nov 2014

Posted by in Ask Martyn, Data governance, information

≈ Leave a comment

Tags

, ,

The management and architecture of Information Assets

For more than two decades I have tried to convey the importance of treating information and knowledge as potential assets.

Around the world, the response has usually been mixed.

It is understandable that there is frequent reluctance to accept that information might have real value.

After all, we are frequently unaware of what really constitutes an asset and what a liability is.

And we are frequently reluctant to accept puerile arguments that all information has value, whether it does or not, or whether it is an asset or not.

In some organisations where the attitude has been more positive I have managed to take things to the next level.

In many of these case it is business people who will identify the significant challenges, the greatest opportunities and the primary and secondary benefits that might be accrued from treating knowledge and information as if might be an asset.

When we get to this point then it is time to get serious about what actually needs to be done.

This is the time when we start thinking strategically.

As part of this phase we start by exploring what can actually be done in terms of the key aspects and features of a knowledge organisation, and get to some common ground on understanding through constructive discourse.

We address what the organisation needs to think about in order to improve the management and architecture of information assets and

So, the purpose of this (and subsequent follow up blog pieces), is to provide a hugely simplified version of the Cambriano method for the management and architecture of Information Assets.

As an aside, Cambriano Energy is the management consulting company founded by Martyn Jones, and which is also used as a vehicle for promoting the Cambriano Knowledge Asset Management (KAM) approach.

Subsequently, in this blog I will outline a key organisational feature of this method. The Knowledge Asset Management Organisation (KAMO).

I will discuss the drivers for having a KAMO, the justification for this approach, and the benefits that might be accruable from choosing this avenue.

As part of this blog series I will also discuss how the KAM approach relates to reality and how KAM challenges, configures and compliments the vital field of Infonomics.

But before continuing I would like to get a feel for what people would like to see in such articles, especially with regards to the lines of business people are in, and their hands-on-experiences with the issues of information management and treating information as an asset, something with potential value that would need to be managed.

What do you think? What would you like to see addressed? What do you think the opportunities, problems and challenges are?

Until next time…

File under: Good Strat, Good Strategy, Martyn Richard Jones, Martyn Jones, Cambriano Energy, Iniciativa Consulting, Iniciativa para Data Warehouse, Tiki Taka Pro

Aspiring Tendencies in IM: Strength and Innocence

03 Monday Nov 2014

Posted by in accountability, Ask Martyn, Best principles, deceit, pain

≈ Leave a comment

Tags

, , , , , , , , ,

“Anger is the enemy of nonviolence and pride is a monster that swallows it up.”

Mohandas Gandhi

Aspirational trends

The predominance of strength and innocence, better known as ignorance and arrogance, is undermining Information Management, and in turn is ensuring that many Data Warehousing and Decision Support initiatives are disappointments.

2015 will again give IM professionals the opportunity to regain some dignity and professional integrity.

First, by recognizing that there are grave problems within IM; then slowing down and halting the toxic trends, carelessness and bad practices; and then in subsequently, reversing, through intelligence, perseverance and integrity, the ingenuous and decrepit habits that still trouble the profession.

Present indications

In the rush to the bottom we throw excellence in analysis, architecture, engineering and business understanding, under the bus. In IM as well as in many other branches of IT (Information Technology), mediocrity has become the new excellent, regular the new exceptional, and shoddiness the new normal.

Whether it is in Data Warehousing, Big Data, Business Intelligence, Analytics, Decision Support or Data Integration, we see that professional integrity and ethical behaviour – already enough of a rarity in IT – is being repeatedly trumped by short-term expediency, wilful witlessness, and the cultivation and perpetuation of dogmas, dysfunctional behaviour and dubious doings.

The Information Management sector is rife with elaborate charlatanry, partisan expediency and wilful self-deception. There is not a day that goes by in which we are not submitted to an avalanche of contemptible claims from rogue IM evangelists, DW neophytes and unsophisticated opportunists, who chose to simply make things up as they go along.

Manifest requisites

It is in the best interests of IM to raise the profession out of the ditch; to reform the profession from the inside; to drive sea-change improvements in knowledge, quality and professional integrity; to ensure a drastic reduction in destructive hype, deception and dogma, and, to show the artless charlatans, chancers and snake-oil merchants the door.

Data Warehousing and Decision Support – if done right, and for the right reasons – can deliver tangible benefits to many organisations. Simply stated, if business information has a value in the realm of business and strategy then it should be treated as an asset, if it is an asset then it should be managed and nurtured as such, which means aiming to do the right thing right, first time, every time, whilst focusing on maximising confidence, availability and agility.

File under: Good Strat, Good Strategy, Martyn Richard Jones, Martyn Jones, Cambriano Energy, Iniciativa Consulting, Iniciativa para Data Warehouse, Tiki Taka Pro

The Great Information Struggle: Us and Them

14 Tuesday Oct 2014

Posted by in Architecture, Assets, business, Business Intelligence, Data Warehousing, Management, Value

≈ Leave a comment

Tags

, , , , , , , , ,

In the dim and distant past most organisations struggled along with what they euphemistically referred to as Information Systems.

They were Information Systems with no overall design, no elements of management and no architecture.

Information Systems were built to show how the company had been performing in the immediate past, and that was it. Continue reading →

Silly Season! Data Warehousing is Hadoop is Big Data?

12 Sunday Oct 2014

Posted by in Architecture, Ask Martyn, Banking, Best principles, Big Data, Business Intelligence, Creativity, Data Warehouse, Dogma, Knowledge, Peeves

≈ Leave a comment

Tags

, , , , , , , , , , , ,

Let’s get this baby off the ground

This weekend I read a piece on the Information Management website by Steve Miller with the title of Big Data vs. the Data Warehouse. It’s an old piece, from March 2014.

It was in response to a piece penned by Bill Inmon, titled Big Data or Data Warehouse? Turbocharge Your Porsche – Buy an Elephant, in which he singled out for criticism the ad campaign of a big-data and Hadoop promoter.

Continue reading →

Mugged in Data Hell – Perils of embracing the Faustian Side of IT – Part 4

04 Saturday Oct 2014

Posted by in Analytics, Architecture, Business Intelligence, Data Warehouse, Knowledge, Peeves

≈ 3 Comments

Tags

, , , , , , , , , , , , , , , , ,

Continued from Part 3 which can be found here -> Part 3

“The following week the full high-powered delegation from The Taffia Connection and BogartSys fly in. It was if an army had come to do battle with us. Uniformly slick, sullen and tacky. iPhone; laptop; knock-off Marks and Spark’s; check patterns; butt ugly ties; ‘plastique’ shoes; mismatching socks; ‘business’ rucksacks; all weather hoodies; and, worst of all, off-the-peg smirks just begging to be slapped”.

Continue reading →

Mugged in Data Hell – Perils of embracing the Faustian Side of IT – Part 2

02 Thursday Oct 2014

Posted by in Best principles, Business Intelligence, Data Warehouse, Knowledge

≈ 3 Comments

Tags

, , , , , , , , , , , ,

Continued from Part 1: Crème Continues…

Part 1 can be found here -> Part 1

“We had recently acquired a media news and entertainments business – Media Macaroni International, and we were planning on integrating their general ledger into the corporate IT landscape. One morning I received a call from the CIO of the newly acquired company, inviting me to their site for a meet and greet event.”

“So I moseyed on down to Tinsel Town and got a briefing from not only the CIO, but the full board of directors of Media Macaroni.”

“To cut a long story short, they basically put me on the spot. Either I integrated the entire Data Warehousing across the corporate in 9 months, or we would have serious problems of convergence. The message couldn’t have been clearer. Either I got my act together and made this acquisition work, or what looked like a humongous hot potato could be burning my mouth anytime soon.”

“I told the directors there and then that the mission was going to be incredibly difficult to fulfil. However, the mood changed.”

“Their CIO looks across the table and tells me that he can help me out of the hole. You see, we have employed a service company that does most of our IT work for us, and according to us at Media Macaroni they are the bee’s knees.”

“So ‘who are these guys’, I ask. Back comes the ominous response: The Taffia Connection”

“So, back home, I call all of my business managers together to discuss the next steps, and despite the misgivings of most of my department heads, and indeed my gut instincts, I become convinced that we should give The Taffia Connection a chance to tell us what they can do for us”.

“So, to cut a long story short, I bring in The Taffia, they give me a dog and pony show on what they can do for us… full data integration, enterprise application integration, enterprise data warehousing, business intelligence, MDM, CRM, Data Quality, Total Quality, IT Business Alignment up the wazoo, and I find the message so compelling that we initiate contractual discussions with The Taffia Connection”.

“As a result, and against all the best advice of my own best staff, I decide to outsource a large chunk of the integration, CRM, EAI, EDW and BI work to The Taffia Connection, work which I am assured will be carried out very cost-effectively by the offshore sister company Bogartsys (whose unofficial slogan at the time was ‘powered by gerbils, motivated by greed’) To be honest, I thought it was great; hell, my board thought it was great. You have to understand the mood of the times. It was all crazy. We were all crazy”.

“Then over the following two years I lost all my friends, my staff who once worshipped the ground I walked on, loathed me, my cat Lucky died of loneliness and depression, my family refused to talk to me, and every now and then a stray dog would pee on the alloy rims of my two year old beamer”.

Part 3 can be found here -> Part 3

Many thanks for reading.

File under: Good Strat, Good Strategy, Martyn Richard Jones, Martyn Jones, Cambriano Energy, Iniciativa Consulting, Iniciativa para Data Warehouse, Tiki Taka Pro