<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0">
<channel>
<title>Business Intelligence - A Practitioner&apos;s Thoughts</title>
<link>http://www.beyeblogs.com/karthikonbi/</link>
<description>This blog is focused on providing a practitioner&apos;s view of the ideas, thoughts and advancements in the Business Intelligence and Analytics space.</description>
<language>en</language>
<copyright>Copyright 2008</copyright>
<lastBuildDate>Sun, 29 Jun 2008 13:45:00 -0700</lastBuildDate>
<generator>http://www.movabletype.org/?v=3.33</generator>
<docs>http://blogs.law.harvard.edu/tech/rss</docs> 


<item>
<title>Hybrid OLAP - The Future of Information Delivery</title>
<description><![CDATA[<p>As I get to see more Enterprise BI initiatives, it is becoming increasingly clear (atleast to me!) that when it comes to information dissemination, Hybrid Online Analytical Processing (HOLAP) is the way to go. Let me explain my position here.</p>

<p>As you might be aware, Relational (ROLAP), Multi-dimensional (MOLAP) and Hybrid OLAP (HOLAP) are the 3 modes of information delivery for BI systems. In a ROLAP environment, the data is stored in a relational structure and is accessed through a semantic layer (usually!). MOLAP on the other hand stores data in proprietary format providing the notion of a multi-dimensional cube to users. HOLAP combines the power of both ROLAP and MOLAP systems and with the rapid improvements made by BI tool vendors, seems to have finally arrived on the scene.</p>

<p>In my mind, the argument for subscribing to the HOLAP paradigm goes back to the "<a href="http://www.intelligententerprise.com/db_area/archives/1999/993003/warehouse.jhtml">classic</a>" article by Ralph Kimball on different types of fact table grains. According to him, there are 3 types of fact tables - Transaction grained, Periodic snapshot, Accumulating snapshot and that atleast 2 of them are required to model a business situation completely. From an analytical standpoint, this means that operational data has to be analyzed along with summarized data (snapshots) for business users to take informed decisions. </p>

<p>Traditionally, the BI world has handled this problem in 2 ways:<br />
1) Build everything on the ROLAP architecture. Handle the summarization either on the fly or thro' summarized reporting tables at the database level. This is not a very elegant solution as everybody in the organization (even those analysts working with summarized information) gets penalized for the slow performance of SQL queries issued against the relational database through the semantic layer.</p>

<p>2)Profile users and segregate operational analysts from strategic analysts. Operational users are provided ROLAP tools while business users working primarily with summarized information are provided their "own" cubes (MOLAP) for high-performance analytics.</p>

<p>Both solutions are rapidly becoming passe. In many organizations now, business users wants to look at summarized information and based on what they see, needs the facility to drill down to granular level information. A good example is the case of analyzing Ledger information (Income statement and Balance Sheet) and then drilling down to Journal entries as required. All this drilling down has to happen through a common interface - either an independent BI Tool or an enterprise portal with an underlying OLAP engine. </p>

<p>This is the world of HOLAP and it is here to stay. The technology improvement that is making this possible is the relatively new wonder-kid, XMLA (XML for Analysis). More about XMLA in my subsequent posts.</p>

<p>As an example of HOLAP architecture, you can take a look at this <a href="http://www.oracle.com/technology/pub/articles/rittman-essbase.html">link</a> to understand the integration of Essbase cubes (MOLAP at its best) with OBIEE (BI Answers - ROLAP platform) to provide a common semantic model for end-user analytics.</p>

<p>Thanks for reading. Please do share your thoughts.</p>]]></description>
<link>http://www.beyeblogs.com/karthikonbi/archive/2008/06/hybrid_olap_the_future_of_info.php</link>
<guid>http://www.beyeblogs.com/karthikonbi/archive/2008/06/hybrid_olap_the_future_of_info.php</guid>
<category></category>
<pubDate>Sun, 29 Jun 2008 13:45:00 -0700</pubDate>
</item>

<item>
<title>Agile Framework - Managing and Measuring Enterprise BI</title>
<description><![CDATA[<p>I recently participated in the International Project Management Conference (PML-2008) by presenting a paper on Agile Framework applicability to Business Intelligence. The paper was among the <a href="http://www.qaiasia.com/Conferences/PML08/paperspractices_presentation.htm">final 10 nominations</a> for the Leadership award and I thought of sharing the gist of the paper on this blog.</p>

<p><strong>The Paper Abstract:</strong><br />
Enterprise Business Intelligence solutions are complex from an implementation standpoint because of the Develop - Support (Growth-Sustain) cycle followed concurrently. Every enterprise wide BI system continuously evolves over a period of time with new business functionality getting added at regular intervals and they need to be in conformance with existing ones. Also, with continuous evolution of functionality comes the question - "How does one measure the progress"?</p>

<p>This paper addresses the two major problems in managing Enterprise Business Intelligence initiatives, namely:<br />
1) Sustenance of concurrent Develop-Support cycles<br />
2) Calibrating the evolution of business functionality</p>

<p>The solution to the vexing problem in development & maintenance of large data warehouses lies in the adaptation of Agile Methodology. Agility in the Data Warehousing context is an approach that "cycles" through the different phases, with the ultimate aim of adding new functionality and stabilizing what is already present. Agile Methodology also provides the platform for measuring/calibrating the progress of Business Intelligence initiatives.</p>

<p><strong>The Paper Contents:</strong><br />
The contents of the paper are given below at a fairly high-level:</p>

<p><strong><em>The Project Management Framework</em></strong><br />
Agile development is a software development approach that "cycles" through the development phases, from gathering requirements to delivering functionality into a working release.</p>

<p>Two phases to the Agile framework implementation are:<br />
1) Planning Phase<br />
2) Execution Phase</p>

<p><u><em>Agile Framework - Planning Phase</em></u><br />
Planning is typically done at the end of a particular year for the subsequent year, once the business plans & budgets are finalized. The steps in the Planning phase are:<br />
1) Create and Prioritize the Stories<br />
2) Create the Phase Plan<br />
3) Identify the "Cycles" (Development and Stabilization Cycles)<br />
4) Create the Release Plan</p>

<p><u><em>Agile Framework - Execution Phase</em></u><br />
The Execution Phase is for implementing the periodic releases. This has the following steps:<br />
1) Execution of Cycles<br />
2) Delivering the Release<br />
3) Delivering the Phase<br />
4) Completing the Story</p>

<p><strong><em>The Measurement Framework</em></strong><br />
The Measurement Framework for Enterprise Business Intelligence combines the practical implementation power of the Agile Methodology and the statistical robustness of the Analytic Hierarchy Process (AHP).</p>

<p>There are 3 levels of scorecards that are part of the measurement framework:<br />
a) Level 1 (Highest Level) - Actual and Planned Rating of the environment shown on a periodic basis<br />
b) Level 2 - For each period, the rating for different components ("Stories" in the Agile terminology) are arrived at.<br />
c)Level 3 - For each component, the score till the end of that particular period is calculated using appropriate calibration factors following the Analytical Hierarchy Process (AHP) technique.</p>

<p>The key takeaways from this paper are:<br />
1) Enterprise Business Intelligence systems are complex to manage as they constantly evolve over time<br />
2) Agile Framework does provide an elegant way for managing the concurrent "Develop-Support" cycles required for Business Intelligence projects.<br />
3) AHP based measurement techniques provide a powerful way for calibrating and enhancing BI application performance<br />
4) AHP is a simple yet comprehensive way of determining relative importance / weightages among sub-projects that makes up complex systems.</p>

<p>I will elaborate on this topic in my future posts. </p>

<p>Thanks for reading. Please do share your comments.</p>]]></description>
<link>http://www.beyeblogs.com/karthikonbi/archive/2008/06/agile_framework_managing_and_m.php</link>
<guid>http://www.beyeblogs.com/karthikonbi/archive/2008/06/agile_framework_managing_and_m.php</guid>
<category></category>
<pubDate>Tue, 17 Jun 2008 06:15:00 -0700</pubDate>
</item>

<item>
<title>Let&apos;s talk EPM - Part 2 on Metrics Profiling</title>
<description><![CDATA[<p>In my earlier post on Enterprise Performance Management (EPM), I had enumerated the six steps of a practical EPM strategy in an organization. They were:</p>

<p>1. Business Process Maps - Understand the business process <br />
2. Metrics Identification - Get hold of the metrics <br />
3. Metrics Profiling - Understand the metrics in depth <br />
4. Metrics Maps - Understand the cause and effect relationships between metrics <br />
5. Metrics Visualization - Implementation of Metric Maps on BI Tools <br />
6. Watch and Improve - Monitor Metrics and Improve business process as required</p>

<p>It is important to realize that building a data warehouse (enterprise wide) or data mart (functional area wise) or simply an integrated, subject-oriented data repository (without getting lost in semantics!) is implicit in the set of steps outlined above.</p>

<p>Steps 1 and 2 (Business Process and Metrics identification) are self-explanatory. Though getting hold of the right metrics is easier said than done, it is fairly well understood that the measures/metrics selected for analysis should align itself with the organization's mission, business model and value creation aspects. </p>

<p>Step 3 - Metrics Profiling, in my opinion, is the step often missed out in EPM implementations and arguably is a major cause of failures in such programs. Metrics Profiling stated simply is a way of understanding your metrics in depth. Given below is a sample template for profiling your metrics and can be customized for each organization.</p>

<p>Profiling Parameters:<br />
1) Metric Name - Name of the metric<br />
2) Metric Definition - Brief definition of the metric<br />
3) Business Area/Process - Identify the functional area of metric origination<br />
4) Financial Impact - Indication of how the metric affects the Income Statement, Balance Sheet, Cash flow statement etc.<br />
5) Metric Type - Is it a ratio, absolute number, Trended value, etc.<br />
6) Sources of data – Identify the source of data for the metric and the owners of that data <br />
7) Calculation Involved - Define the calculation involved in computing the metric<br />
8) Application - Brief description of how the metric helps in managing the business better<br />
9) Metric Periodicity - Indicate the relevant periodicity of metric measurement<br />
10)Potentially Affected Metrics - Identify the other metrics that are impacted (positive or negative) by this metric.<br />
11) Example - Provide an example of metrics usage. (For example: ABC Computers released three new product lines during the last 12 months, generating $15 million in new revenue out of total annual revenue of $125 million. New Products Index = 15/125 = 12%)</p>

<p>Metrics Profiling is a very important step in the implementation of enterprise wide performance management system. </p>

<p>Thanks for reading.<br />
</p>]]></description>
<link>http://www.beyeblogs.com/karthikonbi/archive/2008/06/lets_talk_epm_part_2_on_metric.php</link>
<guid>http://www.beyeblogs.com/karthikonbi/archive/2008/06/lets_talk_epm_part_2_on_metric.php</guid>
<category></category>
<pubDate>Sat, 07 Jun 2008 01:15:00 -0700</pubDate>
</item>

<item>
<title>Let&apos;s talk EPM - Part 1</title>
<description><![CDATA[<p>Welcome to the world of Enterprise Performance Management (EPM),considered the Holy Grail of Business Intelligence. EPM and its various manifestations creatively named as Business Performance Management (BPM), Corporate Performance Management (CPM) etc. is a set of processes that help organizations optimize their business performance. </p>

<p>Does it sound good? - Ofcourse, Yes! Show me an organization that does not want to optimize!! <br />
Does it sound practical? - Not really! Don't know where to start!! </p>

<p>EPM means many things to many people - Optimization of business performance can mean optimization at the business processes level (local optima), can also mean optimization at the organizational level (global optima) and can also have many flavors in between. </p>

<p>With many BI vendors jumping into the EPM bandwagon, the problem is that EPM is immediately equated to the solutions provided by tools like Business Objects, Cognos, Hyperion etc. That view, in my opinion, is far removed from the truth. </p>

<p>In this series of posts, I would like to share some thoughts on making EPM a practical reality in organizations. To start with, let me enumerate the components of an EPM strategy: </p>

<p>1) Business Process Maps - Understand the business process<br />
2) Metrics Identification - Get hold of the metrics<br />
3) Metrics Profiling - Understand the metrics in depth<br />
4) Metrics Maps - Understand the cause and effect relationships between metrics<br />
5) Metrics Visualization - Implementation of Metric Maps on BI Tools<br />
6) Watch and Improve - Monitor Metrics and Improve business process as required</p>

<p>A keen observer will immediately realize that implementing EPM has lot more of pen & paper work (substitute your favorite analysis tool here!) before technology can come into the picture. Also, in my opinion, there is no silver bullet - No single metric map can fit companies across industries or even within same industry. EPM framework for an organization has to evolve in phases based on company's growth, its corporate vision, and the important numbers at different stages etc. or in other words "EPM is very personal to an organization". </p>

<p>EPM, to a BI practitioner, represents a convergence of many things: </p>

<p>a) Domain Understanding<br />
b) Quantitative Play<br />
c) BI Tool capability<br />
d) Closed-Loop BI Architecture<br />
e) Knowledge of proven methodologies like Six Sigma, Balanced Scorecard etc.</p>

<p>I will try and explain some of the interesting aspects of an EPM strategy like Metrics Profiling, Metrics Maps etc. in the next few posts. Meanwhile, you can take a look at resources <a href="http://www.dmreview.com/issues/20050501/1026062-1.html">like this one</a> to understand the "big picture" with respect to Enterprise Performance Management. </p>

<p>Thanks for reading! <br />
</p>]]></description>
<link>http://www.beyeblogs.com/karthikonbi/archive/2008/05/lets_talk_epm_part_1.php</link>
<guid>http://www.beyeblogs.com/karthikonbi/archive/2008/05/lets_talk_epm_part_1.php</guid>
<category></category>
<pubDate>Tue, 13 May 2008 21:45:00 -0700</pubDate>
</item>

<item>
<title>AHP - A Scientific Approach to BI Tool Evaluation</title>
<description><![CDATA[<p>Enterprise wide BI architecture utilizes multitude of tools within its landscape each serving a specific functionality - Extract, Transform and Load (ETL), Data Cleansing, Metadata Management, Databases (both relational and multidimensional), Reporting and Analytics (OLAP), Data Mining, etc. For example, just taking the OLAP area alone, there are more than <a href="http://www.olapreport.com/ProductsIndex.htm">40 different products</a> that can potentially solve a customer problem. You can imagine the number of combinations possible when all the tool options are combined across the overall landscape. This establishes the fact that one of the most challenging and vexing problems in Business Intelligence domain is <strong>Tool Evaluation</strong>.</p>

<p>Tool Evaluation and selection has become strategic to the implementation of enterprise wide Business Intelligence. Traditionally, tool selection involved comparing the technical features of the tools, looking at demos by product vendors, reading up industry reports, get word-of-mouth referrals and then taking a final decision. In my humble opinion - that is not sufficient any more.</p>

<p>Technical features, though important, cannot be the definitive criteria for selecting a particular tool. More crucial than technical features is what I term as the "Business Fitment Index". The selected tool should fit with the characteristics of the business process prevalent in the organization and should take into account the requirements of different classes of users. The concept of Business Fitment can be classified as a Multi Criteria Decision Making (MCDM) problem and one of the powerful tools in this category is the  <a href="http://en.wikipedia.org/wiki/Analytic_Hierarchy_Process">Analytic Hierarchy Process </a>(AHP).</p>

<p>AHP is a systematic procedure that helps to: <br />
1) Represent the elements of any problem, breaking it down into smaller constituents <br />
2) Assign weightages to each constituent by following a pairwise comparison technique<br />
3) Leverages expert judgment and intuitive feel into a coherent framework for problem solving</p>

<p>In this particular context of BI Tools, there are 3 steps to calculating the Business Fitment Index using AHP.</p>

<p>Step 1 - Pair-wise comparison of business parameters by customer stakeholders is done in this step. The parameters can be things like ? Real Time Data Integration, Data Volumes, Data Quality, Business Rules Flexibility etc.</p>

<p>Step 2 - Relative ranking of Business Parameters based on the AHP (Analytic Hierarchy Process) technique</p>

<p>Step 3 - Each of the short-listed tools are evaluated against the business parameters and a final rating is arrived at taking into account the organization readiness factors</p>

<p>Bottom-line is that the technical features of the tools have to be taken in conjunction with the fitment level of tool to the characteristics of the business. That alone would ensure the success of the tool for enterprise wide BI initiatives.</p>

<p>AHP is a simple yet powerful way of arriving at a decision by consensus. There are wide ranging applications of AHP in BI and this is a great area for practitioners to get interested. Please to share your thoughts on other applications of AHP in the BI world. </p>

<p>Thanks for reading!</p>]]></description>
<link>http://www.beyeblogs.com/karthikonbi/archive/2008/05/ahp_a_scientific_approach_to_b.php</link>
<guid>http://www.beyeblogs.com/karthikonbi/archive/2008/05/ahp_a_scientific_approach_to_b.php</guid>
<category></category>
<pubDate>Fri, 02 May 2008 02:15:00 -0700</pubDate>
</item>

<item>
<title>BI Strategy Definition - A &quot;First Principles&quot; Approach</title>
<description><![CDATA[<p>Business Intelligence (BI) Strategy definition is typically the first step in an organization's endeavor to implement BI. This phase is very crucial as the overall execution direction hinges on decisions taken in this stage.</p>

<p>A practical approach to BI Strategy definition includes the following steps:</p>

<p>1. Business Area Identification - Identify and prioritize the business area(s) for which BI is considered. Ex: Human Resource Analytics, Supply Chain Analytics, Enterprise Performance Analytics etc. </p>

<p>2. Process Mapping Document - Once the business area is identified, map out the individual processes involved in that particular domain. This can be a simple flow-chart that shows the entry and exit criteria for each sub-process.</p>

<p>3. Business Questions Enumeration - Based on the subject areas involved in the business domain, enumerate the list of questions that are to be answered by the analytical layer. </p>

<p>4. Data Elements Segregation - For each of the process steps, identify the data elements. These data elements, after subsequent validation (in conjunction with business questions) would translate into dimensions and facts during the data modeling stage. </p>

<p>5. Data Visualization - Develop a prototype (set of screenshots) on how the data would be visualized for each business question. Business Analysts and domain experts are typically involved at this stage. </p>

<p>6. BI Architecture Synopsis - At a fundamental level, BI architecture is fairly straightforward. The architecture is almost always a combination of the following processes: Extraction (E), Transformation (T), Loading (L), Cubing (C), and Analyze (Z). The number of layers, type of reporting etc. are a combination of ETLCZ components. Ex: ETLZ, ETLTLCZ, ELTZ, ELCZ are some options for BI architecture definition. </p>

<p>7. Next Steps Document - The 'Next Steps' document would list down the other requirements of / from the analytical infrastructure. These can be points around Tool Evaluation, User profiles, Data volumes, Performance considerations, etc. Each of these requirements would translate to an assessment to be carried out before the actual construction begins. </p>

<p>The most common mistake is to start thinking about technology aspects before the actual business requirement is finalized. A precise definition of business questions goes a long way in designing a scalable and robust BI infrastructure.</p>]]></description>
<link>http://www.beyeblogs.com/karthikonbi/archive/2008/04/bi_strategy_definition_a_first.php</link>
<guid>http://www.beyeblogs.com/karthikonbi/archive/2008/04/bi_strategy_definition_a_first.php</guid>
<category></category>
<pubDate>Sat, 19 Apr 2008 23:45:00 -0700</pubDate>
</item>

<item>
<title>Metadata in the BI World</title>
<description><![CDATA[<p>For as long as I can remember, the definition given for Metadata is "Data about Data". </p>

<p>We have all said this in interviews, heard it from candidates, seen it on presentations and have (almost) always nodded our heads in agreement. In the transaction processing world, where "data-in" is the paradigm, the definition is precise. The databases store the business data in the relational format and the system tables / catalogs describe the structure of that data - the columns, type, size, etc. This data about the structure of business data is "Metadata". </p>

<p>In the Business Intelligence world, that definition of metadata is incomplete. A more precise definition of metadata has two components:</p>

<p>Metadata in BI = "Data about data" <strong>plus</strong> "Information about information".</p>

<p>The first component "Data about data" is "Technical Metadata" and is similar to the metadata in the OLTP world. Having said that, the technical metadata in BI is arguably more complex, as it not only encompasses the databases but needs to cover the ETL and Reporting tools as well. Each of the tools in the overall BI landscape has its own metadata and this data has to be looked at in a comprehensive fashion to understand data lineage etc. </p>

<p>Even among BI tools, there are different categories - Tools that expose its metadata completely, tools that gives an handle to its metadata through pre-defined APIs and tools that do not allow any access to the metadata. Given the industry direction and the evolution of Common Warehouse Metamodel (CWM) compliance standards, it is only a matter of time before the tool architecture is designed to expose the technical metadata. CWM is a fascinating topic of its own and you can get a feel for it by visiting this website: <a href="http://www.omg.org/technology/cwm/">http://www.omg.org/technology/cwm/</a></p>

<p>To me, as a BI practitioner, the second piece of the metadata puzzle is the more interesting part. "Information about information" aspect of metadata is "Business Metadata" and its understanding is crucial to implementing the BI vision in any enterprise. </p>

<p>As an analytical information consumer, I have 2 important questions:<br />
       1) Need direction to access the required analytical content. Examples are: <br />
            a)	Where can I get Sales by Product for different geographies over the last 2 years?<br />
            b)	I am interested in Customer Churn Analytics. Now where do I go? ("system"ically  speaking!)</p>

<p>       2) Once the content is retrieved, need guidance on how to make sense of it. Examples are:<br />
            a)	I see the Forecasted Sales for next quarter in the chart. How is this value calculated?<br />
            b)	Total Inventory value shown in this report ? does it include the Raw material inventory or excludes it?</p>

<p>To the analytics provider, this is a complex problem that cuts across Knowledge Management and context based search disciplines. Having said that, it is important for BI practitioners to understand the true nature of business metadata and provide implementable solutions in their specific organizational context. </p>

<p>I would discuss this fascinating area of Metadata management, encompassing both technical and business metadata, in my future posts. </p>

<p>Please do keep reading and share your thoughts as well.<br />
</p>]]></description>
<link>http://www.beyeblogs.com/karthikonbi/archive/2008/04/metadata_in_the_bi_world.php</link>
<guid>http://www.beyeblogs.com/karthikonbi/archive/2008/04/metadata_in_the_bi_world.php</guid>
<category></category>
<pubDate>Sun, 13 Apr 2008 06:45:00 -0700</pubDate>
</item>

<item>
<title>Business Intelligence @ Crossroads</title>
<description><![CDATA[<p>Business Intelligence (BI) is well & truly at crossroads and so are BI practitioners like me. On one hand there is tremendous improvement in BI tools and techniques almost on a daily basis but on the other hand there is still a big expectation gap among business users on Business Intelligence's usage/value to drive core business decisions. This results in every BI practitioner developing a 'split' personality - a la Jekyll and Hyde, getting fascinated by the awesome power of databases, smart techniques in data integration tools, reporting etc. and the very next moment getting into trouble with a business user on why 'that' particular metric cannot be captured in an analytical report.</p>

<p>For the BI technologists, there is never going to be a dull moment in the near future. With all the big product vendors like Microsoft, Oracle, SAP, IBM etc. throwing their might behind BI with their acquisitions and product development and BI specialty providers like Informatica, SAS et al. showing no signs of slowing down - "the technologists can get ready to join the big swinging party"</p>

<p>For the business users, there is still the promise of BI that is very enticing - 'Data to Information to Knowledge to Actions that drive business decisions'. But they are not giving the verdict as of now. Operational folks are really not getting anything out of BI right now (did somebody say BI 2.0?) and the strategic thinkers are not completely satisfied with what they get to see. </p>

<p>The techno-functional managers, the split personality types are the ones in the middle trying to grapple with increasing complexity on the technology side and the ever increasing clamor for insights from the business side.</p>

<p>Having said that, Business Intelligence is so interesting as it presents a wide canvas for practitioners to paint their strokes. These can be related to business domains, technology (databases/tools/web etc.), statistics, visualization, project management and most importantly "common sense".  </p>

<p>Pick your strokes right away - there is more coming from this space on the fascinating world of Business Intelligence.</p>

<p>Have a great day! Thanks for reading.<br />
</p>]]></description>
<link>http://www.beyeblogs.com/karthikonbi/archive/2008/04/business_intelligence_crossroa.php</link>
<guid>http://www.beyeblogs.com/karthikonbi/archive/2008/04/business_intelligence_crossroa.php</guid>
<category></category>
<pubDate>Sat, 05 Apr 2008 05:30:00 -0700</pubDate>
</item>

<item>
<title>Data Modeling in the BI World</title>
<description><![CDATA[<p>One of the key enablers of successful Business Intelligence programs are the ubiquitous, hard-working "Data Models". Data Model is the heart of any software system and at a fundamental level provides placeholders for data elements to reside.</p>

<p>Business Intelligence systems with all its paraphernalia - Data Warehouses, Marts, Analytical & Mining systems etc. typically deals with the largest volume of data in any enterprise and hence data models are highly venerated in the Data Warehousing world.</p>

<p>At a high level, a good Data Warehouse data model has the following goals: (Corollary - If you are looking for a data modeler look for the following traits)<br />
1) Understand the business domain of the organization<br />
2) Understand at a granular level the data generated by the business processes<br />
3) Realize that business data is an ever-changing commodity - The placeholder provided by the data model should be relevant not only for the present but also for the future<br />
4) Can be described at a conceptual and logical level to all relevant stakeholders<br />
5) Should allow for non-complicated conversion to the physical world of databases or data repositories that is manipulated by software systems.</p>

<p>Extensible Data models deal with all the 5 points mentioned above and more specifically has future-proofing as one of its main stated goals. Such extensible models should also be "consumption agnostic", i.e. - it provides for comparable levels of performance irrespective of the way data is being consumed.</p>

<p>Entity-Relationship & Dimensional modeling (<a href="http://www.rkimball.com">http://www.rkimball.com</a>) has been the lingua-franca of BI data modelers operating at the conceptual and logical levels. Newer techniques like Data Vault (<a href="http://www.danlinstedt.com/">http://www.danlinstedt.com/</a>) also provide some interesting thoughts in building better logical models for Data Warehouses.</p>

<p>At the physical implementation level, both relational (ROLAP)and multi-dimensional (MOLAP) databases form the backbone to the BI infrastructure. Each of these techniques have their own strengths and weakness, hence BI data modelers need to be aware of their capabilities to ensure that the right decisions are taken for physicalization of the logical models.</p>

<p>Even among the relational OLAP vendors, traditionally dominated by row-major databases like Oracle, SQL Server etc. there are column-major relational databases of the likes of Sybase IQ, Vertica etc. gaining a lot of popularity with claims of being built ground-up for data warehousing. The physical layer is also seeing a lot of action with the entry of data warehousing appliance vendors like Netezza, Datallegro etc. (<a href="http://www.dmreview.com/article_sub.cfm?articleId=1009168">http://www.dmreview.com/article_sub.cfm?articleId=1009168</a>).</p>

<p>The intent of this post can be summed up as - BI practitioners should:<br />
a) Understand the BI/analytical goals of the enterprise before deciding the data modeling techniques - Make it extensible and future proof<br />
b) Understand the current techniques that help envisage and build data models<br />
c) Be on the look-out for new developments in the data modeling and database world  - There is lot of interesting action happening in this area right now!!</p>

<p>Data Modeling is a fascinating area that combines functional knowledge with technology skills and a good data model goes a long way in ensuring success of enterprise wide BI initiatives.</p>

<p>Thanks for reading. Please do share your views / thoughts.<br />
</p>]]></description>
<link>http://www.beyeblogs.com/karthikonbi/archive/2008/03/data_modeling_in_the_bi_world.php</link>
<guid>http://www.beyeblogs.com/karthikonbi/archive/2008/03/data_modeling_in_the_bi_world.php</guid>
<category></category>
<pubDate>Fri, 21 Mar 2008 05:45:00 -0700</pubDate>
</item>

<item>
<title>&quot;Right&quot; Time Data Integration - How &quot;Real&quot; can it get?</title>
<description><![CDATA[<p>Data Integration in the BI sense, is all about, extracting data from multiple source systems, transforming them using business rules and loading it back into data repositories built to facilitate analysis, reporting, mining etc. </p>

<p>Given that the raw data has to be converted to a different form (subject-oriented rather than being process oriented) more amenable for analysis & decision-making, there are 2 basic questions to be answered:</p>

<p>1) From a business standpoint, how fast <strong>should</strong> the "data-information" conversion happen?<br />
2) From a technology standpoint, how fast <strong>can</strong> the "data-information" conversion happen?</p>

<p>First question is related to the concept of "Right-Time" BI while the second one deals with "Real-Time" data integration. You can get a feel for this topic at the link below: <a href="http://www.tdwi.org/research/display.aspx?ID=7095">http://www.tdwi.org/research/display.aspx?ID=7095</a></p>

<p>Traditionally, BI being used more for strategic decision-making, we were happy with the batch mode of data integration with periodicity of a day or later. But increasingly, business demands that the data to information conversion has to happen much faster and that technology has to support it. </p>

<p>Since the answer to the first question above from the business side, is fast becoming "as fast as possible", the focus has shifted to the technology side. Some solutions to the problem are highlighted below:</p>

<p>1) <strong>Enterprise Information Integration (EII)</strong> - The paradigm here is to "Leave the transaction data where it resides". Business Intelligence reporting/query/analytical tools have to seek data from the OLTP systems through a semantic layer that defines the required analytical relationships. This is probably as real time as you can get!</p>

<p>2) <strong>Active Data Warehousing </strong>- The most popular proponent of this approach is Teradata. This is the concept of "BI on the Fly". By intelligently combining the hardware and software power, tools like Teradata and other DW appliances can provide analytical outputs from transactional data with terrific performance.</p>

<p>3) <strong>BI with EAI Architecture</strong> - In the traditional approach to DW construction of integrating multiple sources through ETL tools, one area where I foresee a lot of activity is in the close interaction of EAI tools like IBM Websphere MQ, TIBCO etc. with data integration tools like Informatica etc. At this point in time, though the technology is available, there aren't too many places where messaging is embedded into the BI architectural landscape.<br />
 <br />
Bottom-line is that there is significant value gained by ensuring that raw business data is transformed to information by the BI infrastructure, as fast as possible, with the limits being prescribed by business imperatives. The best explanation I have come across to explain the value of information latency is the article by Richard Hackathorn <a href="http://www.tdan.com/view-articles/5132">(http://www.tdan.com/view-articles/5132)</a>.</p>

<p>Thanks for reading. Please do share your thoughts.</p>]]></description>
<link>http://www.beyeblogs.com/karthikonbi/archive/2008/03/right_time_data_integration_ho.php</link>
<guid>http://www.beyeblogs.com/karthikonbi/archive/2008/03/right_time_data_integration_ho.php</guid>
<category></category>
<pubDate>Sat, 08 Mar 2008 04:45:00 -0700</pubDate>
</item>

<item>
<title>Business Intelligence and Six Sigma</title>
<description><![CDATA[<p>I just finished a Six Sigma project and was left wondering as to why BI practitioners are not using more of that Six Sigma power in Business Intelligence. Let me delve on this subject a bit more. </p>

<p>The Six Sigma project that I just completed was on "Developing a Function Point based estimation model for ETL loads". Essentially, I was facing a lot of problems in estimating the effort for ETL (in this case, Informatica) loads that led to "Effort variances" beyond specified limits. So we kicked off a Six Sigma project that had the following DMAIC phases: </p>

<p>1) Define - Definition of the problem (Ex: Estimation process is out of whack)</p>

<p>2) Measure - We measured the effort variances before the start of the project and also set ourselves a target of where it should be.</p>

<p>3) Analyze - Analyzed the root-cause of the problem. The solution was to let go of the complexity based estimation that was done initially and to adapt Function points. In fact, this FP based estimation model was presented at the International Software Estimation Colloquium last year and won the Runner-up prize <a href="http://www.qaiasia.com/Conferences/sec2007/leadership.htm">(http://www.qaiasia.com/Conferences/sec2007/leadership.htm)</a></p>

<p>4) Improve - Based on a pilot within the project, the Function points based linear regression model was arrived at and the team was educated on the estimation process. The improvements to the estimation process (effort variances) were measured on a regular basis.</p>

<p>5) Control - Periodic checks to ensure the institutionalization of the process and also fine-tune wherever necessary.</p>

<p>That in a nut-shell is what my Six Sigma project was all about. Basically, Six Sigma tries to improve process efficiencies by following the phases mentioned above. </p>

<p>Now let's see the connection to Business Intelligence. Analytics at this stage of evolution (in majority of organizations) are being used to find the improvement area at a given point of time. The improvement area can be a problem (Ex: Trend chart showing that the Sales in the West region is dropping by 10% every quarter for the last 3 quarters) or an opportunity (Ex: Market potential for a product is huge and our share is small). BI is reasonably good at providing this information and it will only get better. But BI by itself does not enforce the process or execution rigor that is required for successful organizations. </p>

<p>To summarize, Six Sigma needs an improvement opportunity as the starting point for it to unleash its power to improve processes. BI generates lot of these opportunities with its DW/Reporting/Analytics components but does not enforce the process implementation rigor. I feel that there is lot of synergy in bringing both together ï¿½ Six Sigma, the left hand and BI, the right hand when brought together can earn a lot of claps in the quest to create learning, performing organizations. </p>

<p>Just to sample the power of Six Sigma techniques, please take a look at the following link: <a href="http://www.kaushik.net/avinash/2007/01/excellent-analytics-tip-9-leverage-statistical-control-limits.html">http://www.kaushik.net/avinash/2007/01/excellent-analytics-tip-9-leverage-statistical-control-limits.html</a>, which illustrates the use of control charts (one of Six Sigma's potent tools) in metrics / KPI management. Fascinating!</p>

<p>Keep reading and please do share your thoughts!</p>]]></description>
<link>http://www.beyeblogs.com/karthikonbi/archive/2008/02/business_intellligence_with_si.php</link>
<guid>http://www.beyeblogs.com/karthikonbi/archive/2008/02/business_intellligence_with_si.php</guid>
<category></category>
<pubDate>Sat, 23 Feb 2008 09:45:00 -0700</pubDate>
</item>

<item>
<title>Zen and the Art of Data Management - The Starting Point!</title>
<description><![CDATA[<p>BI practitioners and business users agree that for good decision making - "Data is everything". After all, the input to any strategic information is raw data and there is enough realization that "Good data is a source of competitive advantage and not just any data".</p>

<p>Having said that, even now, many organizations don't have a comprehensive focus on data that is present within its boundaries. I attribute the problem to the fact that data management strategists haven't been able to get their arms around organizational data in the MECE sense. In consultant speak, MECE turns out to be an acronym for "Mutually Exclusive and Collectively Exhaustive".</p>

<p>I have found it useful to categorize data into the following 6 MECE types:</p>

<p><strong>Type 1:</strong><br />
<u>Transaction Structure Data </u>-Business processes are a series of never-ending transactions. All these transactions has a context and this is defined by this category of data. Examples are: Products, Customers, Departments, Geographies etc.</p>

<p><strong>Type 2:</strong><br />
<u>Transaction Activity Data</u> - These are the transactions themselves. Ex: Purchase Order data, Sales Invoices etc.</p>

<p><strong>Type 3:</strong><br />
<u>Enterprise Structure Data</u> - These data elements are unique to each organization and the inter-relationships between data elements are important. Ex: Chart of Accounts, Org Structure, Bill of materials, etc.</p>

<p><strong>Type 4:</strong><br />
<u>Reference Data</u> - Set of codes, typically name-value pairs that drives business rules. Ex: Region Codes, Customer Types etc.</p>

<p><strong>Type 5:</strong><br />
<u>Metadata</u> - Data that defines other data thus making the collection a self-defining entity</p>

<p><strong>Type 6:</strong><br />
<u>Audit Data</u> - With so much focus on regulatory compliance, this is that data that tracks history of all amendments to business data within the enterprise.</p>

<p>Type 1,3 & 4 together is defined as Master Data and its management is the subject of numerous BI articles and white papers.</p>

<p>The topic of Data Management would be discussed more in detail in the subsequent posts.</p>

<p>Thanks and Keep Reading!</p>]]></description>
<link>http://www.beyeblogs.com/karthikonbi/archive/2008/02/data_management_where_to_start.php</link>
<guid>http://www.beyeblogs.com/karthikonbi/archive/2008/02/data_management_where_to_start.php</guid>
<category></category>
<pubDate>Sat, 09 Feb 2008 03:00:00 -0700</pubDate>
</item>

<item>
<title>Operational BI - Can the OLTPs scale up?</title>
<description><![CDATA[<p>In this discussion on Operational BI, let's begin at the very beginning - The "Transaction Processing Systems". The whole fascinating world of Data Warehousing, BI, Analytics et al owes its existence in large measure to the ubiquitous, all powerful, business transaction processing systems, in short, OLTP systems.</p>

<p>Business, when stripped of its abstractness, is actually a continuum of activities or transactions. The two broad categories of business operations, viz. making & selling, are operationalized through infinite number of activities taking place both within & across organizational boundaries. Thanks to the giant strides that have taken place in the Enterprise Resource Planning (ERP) area, the business transactions are ably & beautifully (the artist in me!) captured by those systems. Traditionally the data that is captured by these OLTP systems are fed downstream to the Business Intelligence infrastructure for collation, cleansing, aggregation & analysis.</p>

<p>It is no surprise that the rapid growth witnessed in the BI market both for products & services, is due to the proliferation of powerful operational systems like SAP, Oracle Apps, Peoplesoft, Siebel, MS Dynamics etc. </p>

<p>Alas, that is where the good news ends. Though the powerful OLTP systems are good at providing business data to downstream analytical systems, they are not as good (as yet!) in receiving & making use of information coming back to them from the analytical systems. The feedback loop is crucial to address the issue of Operational BI. </p>

<p>This leads to my key enabler for Operational BI - <strong>Proliferation of agile, modular & robust transaction processing systems</strong> <br />
- Agile to adapt to changing business conditions fed in from analytical systems<br />
- Modular to accommodate for new components needed to close the feedback loop<br />
- Robust to ensure that the increase in complexity of the OLTP systems does not break it</p>

<p>Major ERP systems seems to be moving in that direction. BI practitioners interested in Operational BI would do well to understand the architecture of ERP / OLTP systems. To add more variety to your thoughts on Operational BI, you can take a look at resources like the one below:</p>

<p><a href="http://www.b-eye-network.com/view/6281">http://www.b-eye-network.com/view/6281</a></p>

<p>Thanks and Keep Reading!</p>]]></description>
<link>http://www.beyeblogs.com/karthikonbi/archive/2008/02/operational_bi_can_the_oltps_s.php</link>
<guid>http://www.beyeblogs.com/karthikonbi/archive/2008/02/operational_bi_can_the_oltps_s.php</guid>
<category></category>
<pubDate>Sat, 02 Feb 2008 04:15:00 -0700</pubDate>
</item>

<item>
<title>BI and SOA for Analytical Smorgasbord</title>
<description><![CDATA[<p>Service Oriented Architecture (SOA) and its closest identifiable alter-ego "Web Services" is an example of hyped-up, much maligned technology buzzword that takes at least 2 or 3 slides in any "bleeding-edge" technology presentation. Having said that, whatever I have seen and heard on Service Oriented Architectural concepts till now, is enough to warrant its listing as one of the key enablers for Business Intelligence Utopia.</p>

<p>There are many powerful ways through which SOA can add significant value to the BI environment. The kind of BI, performance management and data integration artifacts that can be developed and published as web services include: Queries, Reports, MDX queries, Scoring and predictive models, Alerts, Scorecards, Budgets, Plans, BAM agents, Data integration workflows, Federated queries and much more. You can get more information at the link: <a href="http://www.b-eye-network.co.uk/view-articles/4729">http://www.b-eye-network.co.uk/view-articles/4729</a></p>

<p>But the idea that fascinates me with respect to BI on SOA, is the concept of <strong>"Analytical Smorgasbord"</strong>. Imagine a scenario where the business user can assemble their own analytical components from a mélange of available ones, resulting in complete customization of information for the user to take his/her decisions. Each of these available analytical components is self-contained and performs a particular piece of BI functionality. These components are 'Web-Services' and the SOA in such an enterprise is all about:</p>

<p>a) How are these components created?<br />
b) How do the components interact?<br />
c) How is the information published and consumed, in secure manner?</p>

<p>The concept of "Analytical Smorgasbord" truly empowers the business users and is a powerful way to enable, what Gartner terms, as "Information Democracy" in the enterprise. It is important to note that the concept of analytical aggregation changes the Data Warehousing paradigm in a profound way - From one of "Pulling data" to "Seeking data". In more simplistic terms, the end-user analytics should go and fetch data wherever it is rather than expecting all data to be consolidated into one data repository (typically a data warehouse or data mart).</p>

<p>The true intent of this post is to encourage the BI community to start looking at SOA from the end-user analytical standpoint, so that web-services does not remain a mere technology toy but really helps in "Putting the business back in BI" - <a href="http://www.tdwi.org/Publications/display.aspx?id=7913">http://www.tdwi.org/Publications/display.aspx?id=7913</a></p>

<p>I have intentionally left out the technology details related to SOA. You can find wonderful resources on the web like this one: <a href="http://www.dmreview.com/portals/portal.cfm?topicId=1035908">http://www.dmreview.com/portals/portal.cfm?topicId=1035908</a>.</p>

<p>It is becoming increasingly important for BI practitioners to acquire/develop knowledge on Web technologies, XML, SOAP, UDDI, etc. as different domains are converging at a rapid pace.</p>]]></description>
<link>http://www.beyeblogs.com/karthikonbi/archive/2008/01/bi_and_soa_for_analytical_smor.php</link>
<guid>http://www.beyeblogs.com/karthikonbi/archive/2008/01/bi_and_soa_for_analytical_smor.php</guid>
<category></category>
<pubDate>Sat, 26 Jan 2008 02:30:00 -0700</pubDate>
</item>

<item>
<title>&quot;What Management Is&quot; - The crucial link between Business and Intelligence</title>
<description><![CDATA[<p>Let us for a moment accept the hypothesis that the true intent of Business Intelligence is to help organizations manage their business better. Better in this context tends to be a rather elastic adjective as it straddles the entire spectrum of firms using BI for simple management reporting to the other extreme of using BI to "Compete on Analytics" in the marketplace.</p>

<p>"Managing business better" presents the classic question of - What aspects of business can BI help manage better?</p>

<p>The primary reference for listing down the different areas of business is the best management book I have ever read till date - <strong>"What Management Is"</strong> by Joan Magretta and Nan Stone. <a href="http://(http://www.amazon.com/What-Management-Works-Everyones-Business/dp/0743203186).">(http://www.amazon.com/What-Management-Works-Everyones-Business/dp/0743203186).</a>This book really helps in drawing the boundaries around management concepts and for BI practitioners, like me, shows the direction for the evolution and applicability of BI.</p>

<p>In this post, I would just list down the different business areas that ought to be managed for the better and drill down into the applicability of BI for each of these areas in future posts.</p>

<p>1) Value Creation - BI can help in providing the critical "Outside-in" perspective<br />
2) Business Model - Is this the right business to be in?<br />
3) Strategy - Validation and tuning of Strategy through BI<br />
4) Organization Boundaries - BI can help solve the Build vs Buy conundrum <br />
5) Numbers in Business - Really the sweetspot for BI applications<br />
6) Mission and Measures - Connecting the company mission with the measures <br />
7) Innovation and Uncertainty - Domain of Predictive Analytics & its ilk<br />
8) Focus - Realm of Pareto Law versus the more recent "Long-Tail" phenomenon<br />
9) Managing People - Human Resource Analytics is one of the most happening analytics applicability areas at this point in time.</p>

<p>To me, the list above presents the most comprehensive high-level thought process when confronted with implementation of BI in organizations. In my consulting engagements, the litmus test is to really see whether the BI strategy covers the aspects of business as noted above - "More the coverage better is the BI vision".</p>]]></description>
<link>http://www.beyeblogs.com/karthikonbi/archive/2008/01/what_management_is_the_crucial.php</link>
<guid>http://www.beyeblogs.com/karthikonbi/archive/2008/01/what_management_is_the_crucial.php</guid>
<category></category>
<pubDate>Tue, 22 Jan 2008 10:30:00 -0700</pubDate>
</item>


</channel>
</rss>