BeyeBLOGS | BeyeBLOGS Home | Get Your Own Blog

June 10, 2010

Daily Dose - The BAP Magic Quadrant - June 10

Towards the end of last year, Gartner created a new magic quadrant for global business intelligence, analytics and performance management (BAP) service providers to analyze the market place for implementation services of these solutions. The service providers were evaluated on ability to execute and completeness of vision. So, who ended up being in the much sought after leadership quadrant? Who would you really trust with your BAP system if your life depended on it?

IBM? Accenture? Infosys? Cognizant? All of them?

Well see it for yourself - http://www.cognizant.com/images/newCognizant-images/MQ_image.jpg.

No leaders, no visionaries. And no wonder 75% of all BI projects "fail"!

Posted by Rajesh Ramaswamy at 10:45 PM | Comments (0)

June 9, 2010

Daily Dose - www.data.gov - June 9

Today’s digest is focused on a website that celebrated its first anniversary recently. You may have been to this website –.Data.gov is the first such initiative and ushers in a new era of transparency in the government.

The purpose of Data.gov is to increase public access to high value, machine readable data sets generated by the Executive Branch of the Federal Government.

Data.gov is a treasure-trove of data and some of the data catalogs, including data on grants, loans, toxic releases, auto support program details etc. can reveal very interesting trends to the data addict and one who is willing to spend some time converting this data into “information”. There are also links to external sites that have converted the raw data into a more readable format. Probably the real unsaid value of data.gov is that 8 other countries have taken the lead and started publishing high-value data for public consumption. So do visit www.data.gov whenever you get a chance – and use the data sets to get insights!

Posted by Rajesh Ramaswamy at 8:00 AM | Comments (0)

June 8, 2010

Daily Dose - SSIS Best Practices - June 8

Today’s daily dose is rather more mundane – it looks at some of the best practices to use when developing your ETL packages using SQL Server Integration services

- http://consultingblogs.emc.com/jamiethomson/archive/2006/01/05/2554.aspx has one of the most exhaustive lists of best practices – most of them are pretty good. Some of the information in the supporting links is even better. For his more recent posts check out - http://sqlblog.com/blogs/jamie_thomson/

- There are some very interesting musings from BI Polar: http://bi-polar23.blogspot.com/2007/11/ssis-best-practices-part-2.html. I had never been to his blog before today. Have book marked it now – should remember to visit more often.

- Finally, the best of the lot, one of the famous top 10 lists from SQL CAT: http://sqlcat.com/top10lists/archive/2008/10/01/top-10-sql-server-integration-services-best-practices.aspx

So, do you ETL? Do you have anything to add to these?

Posted by Rajesh Ramaswamy at 7:45 AM | Comments (0)

June 7, 2010

Daily Dose - What constitutes DW/BI? - June 7

Today’s mailer is inspired by one of the responses I received to Friday’s digest. To quote

What all constitutes BI? It seems like this is a perennially big subject and is easy to get lost! Can you please help us by creating some sorts of boundaries?

Well, the answer is not simple and I do not claim to know the right answer to this question. So let me throw the question right back at you. Consider the following 3 scenarios:

In exhibit 1, I bring you the work of Charles Minard (1781-1870), a French civil engineer. “His best-known work, Carte figurative des pertes successives en hommes de l'Armee Français dans la campagne de Russe 1812-13, dramatically displays the number of Napoleon's soldiers by the width of an ever-reducing band drawn across a map from France to Moscow. At its origin, a wide band shows 442,000 soldiers left France, narrowing across several hundred miles to100,000 men reaching Moscow. With a parallel temperature graph displaying deadly frigid Russian winter temperatures along the way, the band shrinks during the retreat to a pathetic thin trickle of 10,000 survivors returning to their homeland”. Check http://upload.wikimedia.org/wikipedia/commons/2/29/Minard.png

As exhibit 2, this article from the NY Times (http://www.nytimes.com/interactive/2009/11/09/world/europe/20091109-berlinwallthennow.html), published on the 20th anniversary of the Berlin wall demolition, displays with its innovative photo mash-ups and adjoining commentary how life has changed for the better in unified Germany.

Finally, as exhibit set 3, let us move to sports. Check out the ubiquitous “worm” now well publicized in all televised cricket (http://www.cricinfo.com/zim-tri2010/engine/match/452150.html?view=graph) or the much more sophisticated Guardian Chalkboards (http://www.guardian.co.uk/football/chalkboards) that allow you to create your own interactive visualizations to analyze player/team performance. Finally, for those looking forward to the football (soccer) world cup – here is the mother of all planners (http://www.marca.com/deporte/futbol/mundial/sudafrica-2010/calendario-english.html )

Now, do any or all of the above, fall under the umbrella of business intelligence? Your answer is as good as mine. What I do know of Business Intelligence is that, like our major cities, it constantly grows, evolves and stretches it boundaries.

Posted by Rajesh Ramaswamy at 7:30 AM | Comments (0)

June 4, 2010

Daily Dose - Tree Maps - June 4

Tree Maps - an excellent method for representing complex hierarchical data.

What are Tree Maps? http://en.wikipedia.org/wiki/Treemapping

History of tree maps from the "founder": http://www.cs.umd.edu/hcil/treemap-history/ (Also, down-loadable tools for generating tree maps along with some excellent algorithms)

How can I create Tree Maps? Most leading commercial BI tools support creation of tree maps in their dashboards, but if you don't have access to one and want to play with and understand it, you could do worse than go here: http://manyeyes.alphaworks.ibm.com/manyeyes/page/Treemap.html. 'Many eyes' is a great place for your information visualization needs!

A good example of tree-maps that I stumbled upon yesterday is here: http://news.bbc.co.uk/2/hi/technology/10187248.stm. The interesting stuff on supercomputers is a bonus!

Posted by Rajesh Ramaswamy at 8:30 AM | Comments (0)

June 3, 2010

Daily BI-DWH Digests

After the long hiatus, I am back again. More determined to blog regularly. This will take the form of daily digests that carry information on a variety of topics in DW-BI.

Welcome to the Daily Dose series... :)

Posted by Rajesh Ramaswamy at 7:30 AM | Comments (0)

December 11, 2009

Trends for 2010

The holiday season is upon us and while it might be nostalgic to think over how the year went, here is a sneak peek into our glimpse of the future. Well, at least, of the trends that we believe will impact the BI world in 2010. A quick disclaimer before we start (you can never have enough of them these days): these trends are forecast from a practitioner's view-point (based on Marlabs' experiences with our clients) and we hope they help organizations plan their budgets and roadmaps for the year ahead. So without much further ado (cue trumpets and drum-rolls), here are our 5 trends for 2010.

1. iBI Systems: The only thing that can match the raw excitement of watching yet another contestant fall flat in Takeshi's castle is having the power to coin innovate monikers for seemingly unrelated stuff. iBI is one of those. Perceive the subtle italicization of the "i". iBI is the collective noun for those innovations in BI that are brought about as a consequence of the internet age and the technology innovations that have affected it. These include the hotter-than-molten-lava trend of cloud computing as well as much cooler stuff like mashups and advanced visualizations. 2010 should see all of these adapted into the BI mainstream.

2. Very Large Data Warehouses: I remember a time while I was merely old and not senile that any DWH in gigabytes would give me goose-pimples. Now terabytes are passe, petabytes give you something to think about and exabytes and zettabytes are where the goose-pimples start to emerge. (No, I hadn't heard of zettabytes before, either; but check this out). To cut a long story short, traditional relational databases are about as effective in handling these data volumes as pigeons were in carrying urgent information from one kingdom to another. Appliances, columnar databases and distributed applications like hadoop (and other variants of the map-reduce algorithm) are the way ahead in 2010 for organizations that perforce have to handle huge volumes of data. (think online organizations, telecom providers and energy/utilities players who have implemented smart-grids)

3. Mobile Analytics: I was tempted to name this mAnalytics but decided that trying the same gag twice in one page might be stretching it. With the ever increasing adoption of mobile phones and with every man, woman and dog having at least 3 of them (business, casual and sports), cell phone as a delivery mechanism for BI was more inevitable than death and taxes. Well, at least death. At approximately 3.5 billion mobile devices, the phones population is in a neck and neck race with human population, the latter leading only because they have longer necks. All leading commercial BI vendors now have a way of delivering nearly full-functionality reports over smart phones. This delivery mechanism is bound to see an exponential growth in adoption in 2010. In addition to this, GPS enabled smart-phones, will help bring "location" based intelligence into BI helping make it more "event-based"

4. Embedded BI: If your drew an organizational pyramid in white and shaded the people who use BI in black, it would probably (very loosely) resemble a polar bear - vast acres of white and a black nose. (Interesting polar bear fact/myth). Most of the people who use BI today are those who pull out a report for compliance/regulatory/statutory reasons or the people who frame organizational strategy. In other words tending towards an infinitesimally small percentage of the organization. Embedding BI in applications that line-of-business decision makers use, will help them make better data-driven decisions and increase BI adoption. One of the biggest factors that influences this trend is the growing maturity of open source players. 2010 shall see OEMs and SaaS providers embedding BI in applications that a vast majority of the organization uses.

5. Re-defining the boundaries of BI: Finally 2010 shall probably be the year when BI stretches and re-defines its boundaries, like Napolean Bonaparte did all those years ago. Emails, documents, videos, voices and my 3-year old daughter's scratch book shall all come under the purview of BI as unstructured data becomes increasingly important. We shall also see BI being less of an IT activity and more driven by the business user as self-service reports and mash-ups become more common. Increasingly innovations in BI will be driven not by technology, but by business.

So, that is our vision of 2010. While not quite 2012-esque yet, radical changes next year could make it one of the defining years in BI history. This is, as I mentioned upfront, only a sneak preview. If you want to read a more detailed white-paper on these trends, add a comment to this blog and you will get it totally free - not just free of cost but free of the cheap gags in this blog as well.

Posted by Rajesh Ramaswamy at 6:45 AM | Comments (4)

December 9, 2009

DW/BI Courses in school

This particular blog entry may not make much sense if you are from outside India. Or it may. Well, not too much anyway.

This piece on the web caught my attention. DW/BI has evolved to be one of the most popular career paths in the business technology space, yet schools have been much slower in adopting DW/BI as part of their curriculum.

Hiring organizations would surely appreciate candidates who come in with sound fundamental knowledge in the DW/BI space; today most of these organizations resort to extensive training for the freshers that join. These training programs last from anything between 3 months to 1 year. It is only after this that the new entrants can be gainfully employed.

A 4-pronged strategy could be adopted by universities to plug this gap:

- DW/BI fundamentals should be a mandatory course in all CS/IS/MIS/BCA/MCA programs
- Focused courses on ETL, Dimensional modeling, reporting and analysis, dashboarding and visualizations, performance management etc. could be offered in under grad courses. Hands on sessions on these could be facilitated through use of MS or open source licenses.
- Certificate programs in DW/BI which mandate completion of a certain specific number of DW/BI units could be offered to full-time as well as part-time students
- Specializations on DW/BI could be offered as post-grad courses after tie-up with a commercial BI vendor.

With all this industry-university linkage in prevalence, is it too much to hope that some of this will be a reality in the not-so-distant future?

Posted by Rajesh Ramaswamy at 6:15 AM | Comments (0)

January 27, 2009

PPS on its way out

The big BI news this month is that PerformancePoint (that was launched with much back-slapping) is being sunset. The dashboarding and scorecarding features will be integrated with Microsoft Office Sharepoint (MOSS), but the planning module is effectively being "retired".

You can view the original announcement here or read about Cidi Howson's (of the BI scorecard fame) view here.

The impact of this decision by Microsoft is split down the middle. Supporters say that this could help reduce BI costs and help it reach more end consumers. The ABM (anyone but MS) camp insists that Microsoft has given up on the Performance Management battle all together. It is a little bit of both, I guess. It is undeniable that Performance Management is the big wave in BI currently and it shows no sign of ebbing away any time soon. Planning and forecasting are key components of the PM space and by moving away from it (or by integrating this with MS ERP), Microsoft is essentially asking us to us another tool for PM (if you need to). Considering that most of the spread of MS BI is in the SMB sector, I do not expect it to have an impact on the market share in the short-term. However, if MS wants to be a market leader in the BI space, Performance Management is not something that it can ignore for much longer.

One of my colleagues also referred me to this blog. It is a good read!

Posted by BeyeBLOGS at 3:15 AM | Comments (0)

December 3, 2008

Challenges in BI Implementations

All of us will have our own, unique take on this; but hey, a blog is all about opinions!

I think it is imperative to classify the challenges faced by the host company and the solution provider separately and stress that they vary based on the type of implementation and the current state of the host.

From my experience these are probably the top 3 challenges faced in larger implementations (in no particular order):

Host Organization

1. Resistance to change: Introduction of a BI system changes the operational processes and might change business processes. Every individual prefers a status-quo and getting their buy-in (especially when we are dealing with operational BI) is critical. Moreover, embracing a BI system often requires the users to get trained on and be familiar with a new set of tools. That decision never wins the popularity sweep-stakes. The best designed and implemented BI project is a failure if user adoption is low.

2. Master Data Management: Integrating and/or reconciling existing, disparate master data across various silos is a key challenge for most large organizations. It may be overwhelming to drive consensus on this, but it is imperative to keep moving forward in this exercise.

3. Justifying RoI: BI business benefits are difficult to quantify. How do you prove that an increase in sales is because of the better strategy adopted from the insights your BI system provided and not because your sales personnel became �super� sales personnel? Quick wins and consistent visibitlity of key metrics to track project progress is a major driver of success

Solution Provider

1. Estimating for a BI implementation: Accurately estimating a BI project is difficult because the scope is extremely difficult to nail-down and the expected output is often never known till it is actually seen. Prototyping plays a critical role in the entire exercise. There is no industry standard methodology, though the �agile� approach tends to work well and ETL is easier to estimate than reporting and analytics. I personally use a complex excel template that relies on several �weights� provided by past experience. But, on the whole, making an effort or schedule estimate for implementing a BI system running out of an EDW can be extremely hazardous. It takes a brave person to make a "fixed price" bid!

2. Data Quality: Everyone knows about this, yet it continues to be a critical bottle-neck. Often the only solution to this is to change the way the transactional system handles data, bu that is easier said than done. Anyway, garbage in is garbage out. Enough said.

3. Contain scope, yet be flexible; deliver quick wins, but don�t lose sight of the big picture: It is necessary for the provider to wear many hats at the same time � ensuring that varying objectives are met is critical to this. The client CIO, COO and CFO may have divergent, even contradictory objectives � yet the solution provider has to meet them all! Be prepared for lots of prototyping and throwing away developed and working components.

Posted by Rajesh Ramaswamy at 9:30 AM | Comments (0)