BeyeBLOGS | BeyeBLOGS Home | Get Your Own Blog

« Surnames and nicknames, Mr. Trump, an unusual phonebook, and data quality. | Main | Beer and diapers revisited - not just an urban legend »

February 19, 2007

Predictive Analytics for the Excel user ... and that means for you.

I was speaking at a BI event in Ireland a couple of weeks ago. After one session, one of our partners decided to collar me with some friendly criticism of Microsoft's well-known slogan BI for the masses. "You can have all the dashboards and reports you want," he said, "But until I can open up Excel and do cutting-edge analysis right there, it's all just blowing smoke. Excel is where I do my work, and that is where my BI needs to live to be efficient. It is all well and good having data mining or some such on the server, but it’s too complex. I need it in Excel, and I can't wait for that day to arrive." I could feel, every time he stressed Excel that he was mentally prodding me, with good humour I must say, until I got the point.

I hope I did not look smug - but forgive me if I did. For the answer was simply, "You only have to wait a week or so. SP2 of SQL Server 2005 has exactly what you need." At the next session, I specifically demonstrated the data mining features. We took a table of customer sales data in Excel. With a few clicks, we had detected categories of customers - base on their demographics in that example. We labeled our rows with these new categories. Detecting outliers based on patterns discovered by the mining add-ins was only a few clicks more. We built predictions of future sales, directly inside Excel. We used goal seeking to find how to move customers from low-value categories to those with a higher potential.

I watched my friendly critic's expression out of the corner of my eye. He was grinning hugely. I wonder if he thought I stayed up all night coding the demo, as it so exactly matched what he asked for.

All these features and many more are in the SQL Server Service Pack 2 released today - February 19th. I believe the Microsoft Data Mining Add-ins for Excel are a real game-changer. They bring the power of predictive analytics to every desktop, along with clustering and segmentation, and some powerful data preparation features. You can read more about the add-ins by following the link, and you can download them freely too.

Why are these features so significant? For years now, business users have too often looked on data mining as cryptic - at worst almost mystical - in its complexity. It was the realm of experts in the backroom, applying intimidating algorithms for the greater benefit of all. This miasma of complexity has hung around largely because it has been so difficult for end users - business analysts mainly - to get their hands dirty with predictive analysis. Until now, specialists served the business user with carefully prepared and presented data mining results that were simply not in the business users’ domain of expertise to criticize. At best they could fall back on the saying of the wonderful Scottish scul?tor George Wyllie who memorably said, "Ye don't need to know how a thing works, to know that it is nae workin'."

I remember when OLAP had the same air of intimidating difficulty. Multi-dimensional hierarchical browsing sounded like something only Doctor Spock could really understand. When Microsoft brought OLAP to the desktop through Pivot Table Services, end users could start to grasp the power of the dimensional modeling method, and to gain some insight into the techniques and capabilities, and limitations, of the technology.

The Data Mining Add-Ins will do the same for predictive analytics. Just as not every analyst rushed off to become a dimensional modeler, I do not expect them to become a data miner either. However, they will learn more about the techniques, will discover their usefulness and limitations, and will respect and trust the technology more because of it. Just as with OLAP, I expect there will be nay-sayers who maintain that there really is something special and arcane about predictive analysis: they'll still be saying that, even as the technology is commoditized on desktops all around them. Because I’ll tell you something else about these Add-Ins, too: they are beautiful to work with.The SQL Server Data Mining team have done a wonderful job with them - the user interfaces are as rich, informative, and nicely crafted as any other Excel feature.

Now, I’m not going to pretend that the Data Mining Add-Ins are a miracle application that somehow can deliver perfect and reliable predictive technology to the inexperienced user. There are still many issues of data preparation, and model validation that we’ll explore in future posts. However, analysts close to the data, and already experienced with BI (as most of readers of this blog will surely be) - such users will love these new tools; and I in turn would be very happy to hear more of your experiences with them.

Posted by Donald Farmer at February 19, 2007 7:51 PM


Post a comment

Remember Me?