Tags

, ,

 Old advice is still good advice, if it is truly good advice.

It’s an older piece (from the nineties), but a lot of it is still relevant and pertinent.

So, here is a brief run-down on how to avoid crashing and burning Data Warehousing.

Understand that a Data Warehouse should only contain subject oriented, integrated, non-volatile and time-variant information to support strategic and tactical requirements of management – Keeping the purpose of the Data Warehouse highly focused will eliminate the dilution of a Data Warehouse solutions effectiveness. If what Bill Inmon said in this respect almost invariably results in successful DW projects – and that’s not speculation but proven fact – then I don’t see any rational reason at the moment to change our views on this.

Ensure High-level Business Sponsorship – A very old commentary by the Hurwitz group pointed to a Price Waterhouse Coopers survey that said that 67% of warehouses fail, and related successes to the fact that “successful warehouses received sponsorship from key business executives. Therefore, with the need to focus on answering business questions, data warehouses should be designed in a way that the information they contain and the structure that is used should be intuitive for business users to use.

Understand and Involve The Business User. Always – The Business End-Users of the Data Warehouse should form an integral part of the Data Warehouse project team – either directly, or in the case of a large user community, through adequate, appropriate and timely end-user representation. End-User participation in a project does not imply that everyone else can take a back seat. Every assistance possible must be given to ensure that the End-Users understand what is possible, how much it costs and how long it takes. Every relevant piece of data warehouse project data, information and knowledge must be made available to the End-User and when necessary explained to the End-User in terms they can understand. Don’t expect users to have all the answers or to be able to provide you with answers in precisely the form that you may be expecting those answers. Be understanding, be flexible, and also don’t forget that business is dynamic and in a state of almost constant change. Therefore, don’t just hypothesize about users changing their minds, one must expect it to happen. Anticipate change and prepare for it, embrace change and make it work for you. Let’s not ever forget that the building of a Data Warehouse is for the benefit of the whole business and a partnership in which everyone has a major stake.

Start with a Technology Pilot – One of the best ways to ensure the initial success of an Enterprise Wide Data Warehouse is to select an initial project that is:

  • Small enough to be achievable
  • Large enough to be significant
  • High business profile if successful

Don’t Make the DW an IT Project – I recommend that a Data Warehouse is first and foremost a Business Project that just happens to need information technology products and services. Don’t just hand-over all the responsibility for your Enterprise Data Warehouse project to the IT function – Make sure that the business is the DW owner and that End-Users drive the requirements specification.

Deliver Information Promptly – You should aim at delivering information to the End-User within a 3 to 4 month time-scale for the first iteration, and below a 3 month time-scale for subsequent iterations. There is nothing so successful in setting realistic user expectations and then delivering on them promptly and accurately.

Truly Empower The Business User – The data warehouse is a way to provide adequate, appropriate and timely information to the business so that users will be able to create all their own reports and information analysis. The data warehouse should be used to throw out many of the dependencies of Business Users on IT. The data warehouse avoids the need and cost of providing them with support, that inadequate and dependency building support that they themselves should not need if they are adequately trained, encouraged and empowered.

Use the Iterative and Agile Approach to the Max – Most companies are faced with enormous amounts of data, in many formats that could potentially be located in a data warehouse. But operational systems, for example, have large amounts of transaction-level data that may or may not be required for analysis. Trying to locate all this corporate data at once is generally not feasible – for logistical and cost reasons alone. If, however, a company can build its data warehouse by moving portions of the data incrementally, as needed to solve a specific business problem, the process becomes safer, more manageable, and less costly, providing a faster return on investment. For example, a telecommunication company can start by consolidating all its customer information in a data warehouse in order to understand how to maintain better relationships with existing customers.

The use of the iterative development approach is the most effective way to balance timely delivery with the complexities of the telecommunications environment – the iterative approach is an integral feature of the iniciativa/ISF methodology. There are five major drivers that lead us to an iterative development approach:

The value of information will change.

The complete value chain of information must be understood and delivered, this is known as a dynamic characteristic.

The business processes will continue to change and be refined

Scalable technology decisions will need to change and be refined.

A flexible organization must be supported.

Iterative development speeds the delivery of benefits to users. An initial iteration can deliver limited functionality to a select group of users. Later iterations can be built upon the work of the first, decreasing the amount of effort. The iterations can be carefully planned to deliver the complete value chain of information delivery using your own business priorities as drivers.

Drive a Corporation Data Source Audit – It may be a good idea to run a parallel sub-project to identify and catalogue all possible data sources throughout the enterprise. It may be a good idea to include as much meta-data type information as possible: data source, platform, database, predicted quality and reliability. In the audit process do not forget to take into account systems either in the planning, analysis or development stage. However, don’t let this task be an excuse for letting “analysis paralysis” damage your Data Warehouse project.

Give the users what they want but don’t create unrealizable expectations – the key to business success has been described as that of knowing what the users want and then giving them what they want. However, don’t promise focused, adequate, appropriate and timely information if you can’t deliver on that promise.

Ensure accuracy and understand Data Quality issues – Ensure that the information you supply to users is accurate enough to be truly useful and that your data quality standards are realistic and cost-effective.

Scalability and Architecture – If your data warehouse is successful it will grow – in terms of data volumes, number of users and processing demand. Ensure that the technological architecture chosen for your solution is capable of adapting to the evolving needs of the business and the ability to build an adaptive framework to evolve with the business requirements

Take Advantage of Others Knowledge, Experience and Technologies – But Don’t Be Taken Advantage of – Data Warehousing is a process that requires significant amounts of know-how and experience to get it right. It pays to work with people who have done Data Warehousing in Telecoms successfully before – even if there is generally an aversion to using 3rd party consulting services pick and match your needs with what is available. Don’t limit your ability to deliver the right solution on time by short sightedness or a not-invented-here syndrome. On the other hand, beware of companies that offer to build – for example – a customized generic Telecoms DW data model with three senior consultants in three months – for more information on Data Modeling in a Banking, Government or Telecoms contact you should contact Cambriano (martyn.jones@cambriano.es), or any other reputable consulting organization.

Understand, Plan For and Manage the Impact of the Data Warehouse of the IT Infrastructure – It is important to understand the impact of the Data Warehouse on the IT infrastructure and to plan accordingly. Moving large volumes of data from Operational Systems to the Data Warehouse – Extraction, Transformation and Transportation of data – and end-user usage of the Data Warehouse signifies additional and significant impact on the IT infrastructure, i.e.:

  • Additional processing loads on the source data OLTP systems
  • Additional traffic on the corporate network – do you upgrade the network for increased band-width and network speed?
  • Additional administration and support workload for IT Infrastructure staff
  • Additional users of the DW occupying network space
  • Additional availability demands on OLTP systems – e.g. Data Warehousing ETT demands may reduce ability to plan for OLTP batch job re-runs

Therefore, understanding and planning for the Data Warehouse and its impact on the IT Infrastructure in a critical success factor.

Transform and Structure – Take full advantage of the most appropriate ETT (now called ETL or integration) tools and DW data modeling techniques to provide integrated data and information in a form that business users understand and that is easy for them to use. Don’t be put off by the apparent simplicity of the Extract/Transform/Transport process for the first iteration, although the idea of hand-crafting ETT processes using 3GL code is aesthetically pleasing the process does get more complex to develop and maintain after iteration one. Bottom line: DIY ETT sounds simple, using an ETT tool sounds more complex, and the simplicity of DIY is as false as the simplicity of DW ETT process maintenance using an ETT tool is true.

End-User Tools that best match existing and known paradigms work well in the short term – End-users who are used to working with products such as MS/Excel will also find products such as BusinessObjects fairly easy to come to terms with. Some access to the information in the Telecoms Data Warehouse will almost invariably have to be provided via a canned interface.

Market the success of the Data Warehouse – Encourage Business Users to discuss why the Data Warehouse has improved their ability to fulfil their missions. Publish newsletters containing surveys, success stories and End-User guides to getting the best from the information in the DW. Start a competition to find a good name for your Data Warehouse and Data Warehouse project A short memorable name and a simple and effective logo will create a greater sense of identity and purpose.

Getting Users to Justify the Data Warehouse Success leads to more success – a product’s best salesperson is the customer (End-User) not the provider (iniciativa in Spain and South and Central America, and Cambriano Energy for USA and Europe or even your own IT organization).

Encourage users to justify why they need the Data Warehouse – keep business users and business stakeholders satisfied and encourage this DW justification from these users and stakeholders on an on-going basis.

Trust and Confidence Ensure that End-Users are satisfied with the quality of the data warehouse service and that they can trust the quality of the DW data.

Single Point Of Contact Provide your End-Users with a single point of contact for all routine queries – Allow them simple and effective means to escalate issues if they do not get satisfaction. Make sure they understand the process fully. Make sure your Single Point of Contact has the empowerment to be as flexible, dynamic and rapid in dealing with business customers.

Partner with an experienced and knowledgeable consulting team – one who will fully understand the wide range of aspects and components required to successfully deliver the advantages of Corporate Data Warehousing. A true partner that well understands the needs and the dynamics of your marketplace. Or better still, give me a call or drop me a line.

Many thanks for reading. I hope it gave you some useful or though provoking take-aways.

File under: Good Strat, Good Strategy, Martyn Richard Jones, Martyn Jones, Cambriano Energy, Iniciativa Consulting, Iniciativa para Data Warehouse, Tiki Taka Pro