Business process intelligence

As for the future, your task is not to foresee it, but to enable it.

Antoine de Saint-Exupery

The allure of future happiness

Companies the world over have been busily moving away from the more traditional function-based business structures, with their attendant silos of competence, overlapping roles and artificially limited responsibilities, to highly focused business-driven process models.

Well-bounded business process reengineering has often been a critical success factor in contemporary business strategies. So, new ways of looking at processes are introduced in order to bring about far greater levels of simplicity, marked improvements in service and product quality, new-found process robustness, greater customer intensity and intimacy.

This is accompanied by sea-change improvements in the ways that forward-looking organizations do business.

As part of this Information trend, businesses are working towards service-oriented business-process operating platforms, characteristically these new platforms are easily configured to capture and store a wide range of additional data, way above and beyond that which has been associated with more traditional in-house application developments and 3^rd party Enterprise Resource Planning (ERP) applications.

This new level of data intensity and process abstraction means that businesses can essentially record every step, state and decision point in a business process flow, from initiation to closure. Which notionally allows businesses to closely monitor service levels and key performance indicators right down to the finest level of granularity, and across the entire business organization.

Businesses can now look at who does what, when, where, and how, which allows for the accurate pinpointing of process hotspots and process bottlenecks, and the rapid initiation of corrective measures.

Excitement, elation and enthusiasm

The separation of operational support and strategic decision support, using a combination of data warehousing and business intelligence, is being blurred. Suddenly senior management can really have their finger on the business pulse.

The digital nervous system now promises to become a reality.

Ae we are moving boldly forward into the realms of business intelligence that will not only tell companies what they have done wrong in the past but will assist decision-makers to formulating winning strategies for the future of organizations.

What senior executive could possibly turn down the opportunity to be on top of all aspects of the business that they are ultimately responsible for?

Benefits and features

Industry experts and pundits alike are heralding a significant shift in the usefulness of Business Intelligence, a premonition which frequently leads to assurances that we are witnessing the beginning of the end of Data Warehousing as we know it, the removal of a major capital expenditure burden from corporate IT budgets, and the unchaining and liberation of Business Intelligence, from the terrible drug-like dependency on the Data Warehouse paradigm. The promise of new-age BI is compelling and simple to comprehend, and assures accruable benefits far surpassing anything witnessed in the data warehousing and business intelligence world to date; for example:

1. Business intelligence will become an integral part of business process applications and on-line transactional processing.
2. Management will be able to see, at any point in time, exactly how the business is doing.
3. Management will be able to plan ahead using the rich set of information that the new synergistic integration between BI and business process applications.
4. There will no longer be a need to constrain the freedom of business intelligence by tying it to data warehousing.
5. Far more sophisticated BI tools, technologies and techniques will appear in the coming twelve to eighteen months, driving a wedge between bleeding edge information systems and more traditional approaches to information, analytics and their management.
6. BI will support embedded data analytics, real-time data visualization, and universal use – anyone within an organization will be able to use future-proofed BI.
7. Data warehousing in business organizations will become redundant. (It’s daft, but it needed to be stated).

Hitches, glitches and biting reality

There are a number of issues that may affect the usefulness of the new-age business process oriented business intelligence:

1. The newly reengineered business processes may not in fact be well designed, appropriate, or workable.
2. The performance of operational application platforms may well seriously deteriorate if the same platforms are also used to collect large volumes of disparate process data and at the same time also be used to support unpredictable ad-hoc querying by sophisticated new-age business intelligence tools.
3. There might be difficulties in coming to agreement on service level and key performance indicator measurements, especially if the process paths are complex and full of many decision points, process activities and tasks, and parallel operations.

Failure, despondency and desolation

Someway down the road with your newly found faith in new-age business intelligence you might start to question your belief.

Typically this will come about for one or all of the following reasons:

1. Your business intelligence isn’t giving you an accurate picture of what happened in the past.
2. The quality of data is such that no one in their right mind would use it to try and predict the future, never mind analyse the past.
3. The future-proof promises of BI and claims about being able to predict the future are not turning out as planned.

Bottom-line comments

 The only way you can predict the future is to build it.

Alan Kay

Okay, let’s be frank and earnest here, and try and keep it simple without making it brainless. Let’s assume for one moment that all that has been claimed for new-age business intelligence is possible, and maybe it is, we may intuitively feel a sense of déjà vu, and dismiss the position out of hand as so much frivolous nonsense and sentimental belief, but let’s take a more balanced view, and instead pose some questions:

1. Which organizations can afford to hire the required number of staff in order to be able to effectively visualize and analyse all of the sheer volumes of data and the new wealth of rich business process data that is being produced day in and day out, by businesses all over the globe?
2. Who is really going to use all the data collected by the business processes?
3. Who is going to trust the largely unverifiable new data?
4. Who is going to get the real benefits, if any, which might be accruable from the analysis of the data?

There are some who might think that the claim that business intelligence can happily exist without data warehousing is the biggest load of nonsense ever conceived in the field of information management, and that it beggars belief that “expert” BI consultants seem to confuse “finger on the pulse” with “finger on the trigger of the gun held to the head of business”. The actuality of the real business world is at odds with so much of the BI hype and hyperbole being spun so crudely, so freely and so easily, with scant regard for the business consequences of sowing so much opinion and speculation and spreading the nonsense around like a happy farmer with a truck load of bullshit.

On reading the comments of some business intelligence experts, one could be forgiven for thinking that the intention is to convert commercial businesses into experiments in the creation of technology based busy-work. Let’s be down to earth now, do BI “experts” really think that businesses can afford the luxury of having teams of business analysts dedicated to looking at process data via business intelligence tools, all day and every day?

So here is a roundup of our position with regard to “data warehousing without a data warehouse” and new-age business intelligence:

1. The problem still isn’t lack of data, companies have more data coming out of their “processes” than most business executives have time to shake a stick at. People are virtually drowning in the damn stuff. So, the problem is not lack of data, or lack of data richness, the problem is still lack of appropriate, adequate and timely information.
2. As the father of data warehousing might say, the recurrent idea that you can have a successful data warehousing process without a real DW crops up like the flu virus; it comes around each and every year, and the same damn thing happens year in and year out, some people catch it, quite frankly far too many people – and then don’t know how to let go of it, even if it makes them, and everyone who comes into contact with them, pretty sick and as dumb as rocks.
3. Monitoring business processes, collecting a humungous expanse of data, and then pushing it through a BI tool, will tell you less about future market directions, client behaviour, next year’s fashions and fads, and your real customer satisfaction levels, than a session with, for example, Madame Molotova, the flamboyantly extravagant eastern European clairvoyant.
4. Fourthly, the only blurring of the boundaries between operational applications and data warehousing is in people’s heads, this fuzzy reasoning has led to fuzzy practice, often called pragmatism, in an effort to compound the ill-informed stupidity, but at the end of the day, they are still two sides of the same coin, but they are certainly not the same sides of the same.
5. Next generation business intelligence requires next generation business process applications, based on a comprehensive mix of service oriented architectures, message brokerage, enhanced metadata, and intelligent information interchange (extended mark-up languages etc.) as well as comprehensive process monitoring technology, otherwise you will only reap a relatively very small benefit from the exercise.

The Bullet:

The BI “experts” are at it again, the little sods – trying to bury the principles of well-engineered data warehousing and complementary business intelligence, and denouncing the well proven approaches as being passé, anachronistic, or simply inappropriate.

Business Intelligence doesn’t fail because of data warehousing, this is a load of old nonsense spread by mischievous marketers and useful idiots – useful for helping to sell crap that no one really needs; it fails either because of an absence of a well-engineered data warehouse, carefully aligned to business wants, or because of the adoption of the “pragmatic approach” to data warehousing, which usually means a collection of what are euphemistically termed “data marts” – an approach that seems to have more adherents amongst the technologically light-weight and principle free, both in terms of vendors, and in terms of competence in your everyday business organisations.

Tip for today:

Trying to do comprehensive Business Intelligence without a well architected data warehouse (see W. H. Inmon: DW 2.0, and the Corporate Information Factory,) is simply stated, the strategic, tactical and operational decision support approach of fools.

BI implementations are complex and expensive, and trying to successfully implement BI without the use of a Data Warehouse is like taking on the challenge of diving from a airplane, one mile high, without the cumbersome overhead of having a parachute to slow you down.

BI “experts” who are focusing on the vague and peripheral crapola on the margins of BI, and to the detriment of core information management issues, are not adding any value, au contraire, they are rehashing naïve and sentimental approaches that should have been put to bed a long time ago.

File under: Good Strat, Good Strategy, Martyn Richard Jones, Martyn Jones, Cambriano Energy, Iniciativa Consulting, Iniciativa para Data Warehouse, Tiki Taka Pro