« January 2008 | Main | March 2008 »
February 23, 2008
Business Intelligence and Six Sigma
I just finished a Six Sigma project and was left wondering as to why BI practitioners are not using more of that Six Sigma power in Business Intelligence. Let me delve on this subject a bit more.
The Six Sigma project that I just completed was on "Developing a Function Point based estimation model for ETL loads". Essentially, I was facing a lot of problems in estimating the effort for ETL (in this case, Informatica) loads that led to "Effort variances" beyond specified limits. So we kicked off a Six Sigma project that had the following DMAIC phases:
1) Define - Definition of the problem (Ex: Estimation process is out of whack)
2) Measure - We measured the effort variances before the start of the project and also set ourselves a target of where it should be.
3) Analyze - Analyzed the root-cause of the problem. The solution was to let go of the complexity based estimation that was done initially and to adapt Function points. In fact, this FP based estimation model was presented at the International Software Estimation Colloquium last year and won the Runner-up prize (http://www.qaiasia.com/Conferences/sec2007/leadership.htm)
4) Improve - Based on a pilot within the project, the Function points based linear regression model was arrived at and the team was educated on the estimation process. The improvements to the estimation process (effort variances) were measured on a regular basis.
5) Control - Periodic checks to ensure the institutionalization of the process and also fine-tune wherever necessary.
That in a nut-shell is what my Six Sigma project was all about. Basically, Six Sigma tries to improve process efficiencies by following the phases mentioned above.
Now let's see the connection to Business Intelligence. Analytics at this stage of evolution (in majority of organizations) are being used to find the improvement area at a given point of time. The improvement area can be a problem (Ex: Trend chart showing that the Sales in the West region is dropping by 10% every quarter for the last 3 quarters) or an opportunity (Ex: Market potential for a product is huge and our share is small). BI is reasonably good at providing this information and it will only get better. But BI by itself does not enforce the process or execution rigor that is required for successful organizations.
To summarize, Six Sigma needs an improvement opportunity as the starting point for it to unleash its power to improve processes. BI generates lot of these opportunities with its DW/Reporting/Analytics components but does not enforce the process implementation rigor. I feel that there is lot of synergy in bringing both together � Six Sigma, the left hand and BI, the right hand when brought together can earn a lot of claps in the quest to create learning, performing organizations.
Just to sample the power of Six Sigma techniques, please take a look at the following link: http://www.kaushik.net/avinash/2007/01/excellent-analytics-tip-9-leverage-statistical-control-limits.html, which illustrates the use of control charts (one of Six Sigma's potent tools) in metrics / KPI management. Fascinating!
Keep reading and please do share your thoughts!
Posted by Karthikeyan Sankaran at 9:45 AM | Comments (0)
February 9, 2008
Zen and the Art of Data Management - The Starting Point!
BI practitioners and business users agree that for good decision making - "Data is everything". After all, the input to any strategic information is raw data and there is enough realization that "Good data is a source of competitive advantage and not just any data".
Having said that, even now, many organizations don't have a comprehensive focus on data that is present within its boundaries. I attribute the problem to the fact that data management strategists haven't been able to get their arms around organizational data in the MECE sense. In consultant speak, MECE turns out to be an acronym for "Mutually Exclusive and Collectively Exhaustive".
I have found it useful to categorize data into the following 6 MECE types:
Type 1:
Transaction Structure Data -Business processes are a series of never-ending transactions. All these transactions has a context and this is defined by this category of data. Examples are: Products, Customers, Departments, Geographies etc.
Type 2:
Transaction Activity Data - These are the transactions themselves. Ex: Purchase Order data, Sales Invoices etc.
Type 3:
Enterprise Structure Data - These data elements are unique to each organization and the inter-relationships between data elements are important. Ex: Chart of Accounts, Org Structure, Bill of materials, etc.
Type 4:
Reference Data - Set of codes, typically name-value pairs that drives business rules. Ex: Region Codes, Customer Types etc.
Type 5:
Metadata - Data that defines other data thus making the collection a self-defining entity
Type 6:
Audit Data - With so much focus on regulatory compliance, this is that data that tracks history of all amendments to business data within the enterprise.
Type 1,3 & 4 together is defined as Master Data and its management is the subject of numerous BI articles and white papers.
The topic of Data Management would be discussed more in detail in the subsequent posts.
Thanks and Keep Reading!
Posted by Karthikeyan Sankaran at 3:00 AM | Comments (0)
February 2, 2008
Operational BI - Can the OLTPs scale up?
In this discussion on Operational BI, let's begin at the very beginning - The "Transaction Processing Systems". The whole fascinating world of Data Warehousing, BI, Analytics et al owes its existence in large measure to the ubiquitous, all powerful, business transaction processing systems, in short, OLTP systems.
Business, when stripped of its abstractness, is actually a continuum of activities or transactions. The two broad categories of business operations, viz. making & selling, are operationalized through infinite number of activities taking place both within & across organizational boundaries. Thanks to the giant strides that have taken place in the Enterprise Resource Planning (ERP) area, the business transactions are ably & beautifully (the artist in me!) captured by those systems. Traditionally the data that is captured by these OLTP systems are fed downstream to the Business Intelligence infrastructure for collation, cleansing, aggregation & analysis.
It is no surprise that the rapid growth witnessed in the BI market both for products & services, is due to the proliferation of powerful operational systems like SAP, Oracle Apps, Peoplesoft, Siebel, MS Dynamics etc.
Alas, that is where the good news ends. Though the powerful OLTP systems are good at providing business data to downstream analytical systems, they are not as good (as yet!) in receiving & making use of information coming back to them from the analytical systems. The feedback loop is crucial to address the issue of Operational BI.
This leads to my key enabler for Operational BI - Proliferation of agile, modular & robust transaction processing systems
- Agile to adapt to changing business conditions fed in from analytical systems
- Modular to accommodate for new components needed to close the feedback loop
- Robust to ensure that the increase in complexity of the OLTP systems does not break it
Major ERP systems seems to be moving in that direction. BI practitioners interested in Operational BI would do well to understand the architecture of ERP / OLTP systems. To add more variety to your thoughts on Operational BI, you can take a look at resources like the one below:
http://www.b-eye-network.com/view/6281
Thanks and Keep Reading!
Posted by Karthikeyan Sankaran at 4:15 AM | Comments (0)
