June 29, 2008
Hybrid OLAP - The Future of Information Delivery
As I get to see more Enterprise BI initiatives, it is becoming increasingly clear (atleast to me!) that when it comes to information dissemination, Hybrid Online Analytical Processing (HOLAP) is the way to go. Let me explain my position here.
As you might be aware, Relational (ROLAP), Multi-dimensional (MOLAP) and Hybrid OLAP (HOLAP) are the 3 modes of information delivery for BI systems. In a ROLAP environment, the data is stored in a relational structure and is accessed through a semantic layer (usually!). MOLAP on the other hand stores data in proprietary format providing the notion of a multi-dimensional cube to users. HOLAP combines the power of both ROLAP and MOLAP systems and with the rapid improvements made by BI tool vendors, seems to have finally arrived on the scene.
In my mind, the argument for subscribing to the HOLAP paradigm goes back to the "classic" article by Ralph Kimball on different types of fact table grains. According to him, there are 3 types of fact tables - Transaction grained, Periodic snapshot, Accumulating snapshot and that atleast 2 of them are required to model a business situation completely. From an analytical standpoint, this means that operational data has to be analyzed along with summarized data (snapshots) for business users to take informed decisions.
Traditionally, the BI world has handled this problem in 2 ways:
1) Build everything on the ROLAP architecture. Handle the summarization either on the fly or thro' summarized reporting tables at the database level. This is not a very elegant solution as everybody in the organization (even those analysts working with summarized information) gets penalized for the slow performance of SQL queries issued against the relational database through the semantic layer.
2)Profile users and segregate operational analysts from strategic analysts. Operational users are provided ROLAP tools while business users working primarily with summarized information are provided their "own" cubes (MOLAP) for high-performance analytics.
Both solutions are rapidly becoming passe. In many organizations now, business users wants to look at summarized information and based on what they see, needs the facility to drill down to granular level information. A good example is the case of analyzing Ledger information (Income statement and Balance Sheet) and then drilling down to Journal entries as required. All this drilling down has to happen through a common interface - either an independent BI Tool or an enterprise portal with an underlying OLAP engine.
This is the world of HOLAP and it is here to stay. The technology improvement that is making this possible is the relatively new wonder-kid, XMLA (XML for Analysis). More about XMLA in my subsequent posts.
As an example of HOLAP architecture, you can take a look at this link to understand the integration of Essbase cubes (MOLAP at its best) with OBIEE (BI Answers - ROLAP platform) to provide a common semantic model for end-user analytics.
Thanks for reading. Please do share your thoughts.
Posted by Karthikeyan Sankaran at 1:45 PM | Comments (0)
June 17, 2008
Agile Framework - Managing and Measuring Enterprise BI
I recently participated in the International Project Management Conference (PML-2008) by presenting a paper on Agile Framework applicability to Business Intelligence. The paper was among the final 10 nominations for the Leadership award and I thought of sharing the gist of the paper on this blog.
The Paper Abstract:
Enterprise Business Intelligence solutions are complex from an implementation standpoint because of the Develop - Support (Growth-Sustain) cycle followed concurrently. Every enterprise wide BI system continuously evolves over a period of time with new business functionality getting added at regular intervals and they need to be in conformance with existing ones. Also, with continuous evolution of functionality comes the question - "How does one measure the progress"?
This paper addresses the two major problems in managing Enterprise Business Intelligence initiatives, namely:
1) Sustenance of concurrent Develop-Support cycles
2) Calibrating the evolution of business functionality
The solution to the vexing problem in development & maintenance of large data warehouses lies in the adaptation of Agile Methodology. Agility in the Data Warehousing context is an approach that "cycles" through the different phases, with the ultimate aim of adding new functionality and stabilizing what is already present. Agile Methodology also provides the platform for measuring/calibrating the progress of Business Intelligence initiatives.
The Paper Contents:
The contents of the paper are given below at a fairly high-level:
The Project Management Framework
Agile development is a software development approach that "cycles" through the development phases, from gathering requirements to delivering functionality into a working release.
Two phases to the Agile framework implementation are:
1) Planning Phase
2) Execution Phase
Agile Framework - Planning Phase
Planning is typically done at the end of a particular year for the subsequent year, once the business plans & budgets are finalized. The steps in the Planning phase are:
1) Create and Prioritize the Stories
2) Create the Phase Plan
3) Identify the "Cycles" (Development and Stabilization Cycles)
4) Create the Release Plan
Agile Framework - Execution Phase
The Execution Phase is for implementing the periodic releases. This has the following steps:
1) Execution of Cycles
2) Delivering the Release
3) Delivering the Phase
4) Completing the Story
The Measurement Framework
The Measurement Framework for Enterprise Business Intelligence combines the practical implementation power of the Agile Methodology and the statistical robustness of the Analytic Hierarchy Process (AHP).
There are 3 levels of scorecards that are part of the measurement framework:
a) Level 1 (Highest Level) - Actual and Planned Rating of the environment shown on a periodic basis
b) Level 2 - For each period, the rating for different components ("Stories" in the Agile terminology) are arrived at.
c)Level 3 - For each component, the score till the end of that particular period is calculated using appropriate calibration factors following the Analytical Hierarchy Process (AHP) technique.
The key takeaways from this paper are:
1) Enterprise Business Intelligence systems are complex to manage as they constantly evolve over time
2) Agile Framework does provide an elegant way for managing the concurrent "Develop-Support" cycles required for Business Intelligence projects.
3) AHP based measurement techniques provide a powerful way for calibrating and enhancing BI application performance
4) AHP is a simple yet comprehensive way of determining relative importance / weightages among sub-projects that makes up complex systems.
I will elaborate on this topic in my future posts.
Thanks for reading. Please do share your comments.
Posted by Karthikeyan Sankaran at 6:15 AM | Comments (1)
