The problem with the term "complex event processing" is that it seems to imply a complex UI--hence, that CEP's potential user base is limited to rocket scientists, Wall Street quants, IT industry analysts (gasp!), and other folks who are professionally obliged to handle (indeed, embrace) complexity.

But complexity is part of the problem, not the solution, where CEP for I&KM is concerned. We can't stop the world of business from growing more multifaceted. But we can and should filter it all down to the simplest, most meaningful, most actionable experience, targeted specifically to each user, and contextualized precisely to each and every occasion.

Conceptually, James is clearly right--who's going to disagree with simple, meaningful, actionable and perfectly contextualized--but how do we get there?

Most obviously, this is a great opportunity for BI/CEP software to take a page from web 2.0 software and give users the freedom to customize their environments, and the tools to see how other users have done it. End users whose environments deliver precisely what they want will deal much more effectively with information than those who can only use what a developer (no matter how skilled) gives them.

However, even beyond tools for easy interaction and customization of their visual environment, in order to make users really effective--whether the context is CEP or anything else--applications should be driven by business problems, rather than the tools and technologies available to IT professionals. Users are the experts in their domains, and they will know what qualifies as "targeted specifically" and "contextualized precisely."

Ideally, you would have a platform upon which users and/or their colleagues in IT departments could build applications across the full range of business processes that exist in an enterprise. That way you could leverage IT resources and let users take advantage of a common environment, but still address their business processes in the way that best suits them.

Hmm, I wonder where I could get a platform like that...

Certainly, the challenges that bedevil candidates in trying to sell analytic policies are the same ones that people trying to implement analytic initiatives in the business world often face: Because numbers convey a sense of certainty, how do you use analytics without appearing to know more than you often do, while still clearly communicating the value of analytical thinking?

In order to get there, people who want to bring analytics into their organization need to understand how to talk about the value of analytics to an audience which may not be interested in the behind-the-curtain details.

Effectively conveying the importance of analytics is a worthy cause, but trying to win over the uninitiated shouldn't be done to the exclusion of thinking about where we're going from here. As I mused in the last bit of the piece on CIO.com:

Politics has been on the forefront of some interesting cultural and business revolutions. Polling, cinematic television advertising and relentless "messaging" have all been part and parcel of successful political campaigns. The analytics tools exist to extend into the policy beyond the politics, and it may not be far off.

We've already stated to see the beginnings of such a change in Health Care with the advent of "Evidence-Based Medicine" and I expect it won't be long before the arrival of other areas where it's simply expected that decisions will be based upon the best available evidence.

As with most things, the public sector will likely lag the private, where we can already see the evidence- based methodology appearing in some novel places. For instance, Fog Creek software makes a bug tracking product which incorporates "Evidence Based Scheduling," a very clever method for producing reliable estimates for software development from developers who work at different rates and have different levels of accuracy in their own forecasting.

As for public policy, we already see this move to some extent in the debate over global warming, with both sides of the discussion arguing over the details of research and what it implies for policy decisions (this debate shows how important it is to have good data and lots of it, if you want to avoid the situation where the same data can be used to tell contradictory stories). I have a feeling that as the tools, and in particular, the data sources continue to develop, we'll see much more of this in the future.

HFM: So I think in comparison to the overall quality of mortgage origination in the last, call it, 3 or 4 years, ours was really much better. So I think they're happy we did a better job than our competitors--but they're not happy they lost money.

n 1: Is the person who ran that--is he going to get fired?

HFM: He was already fired.

But--to get back to the paradigm shifts--here was a guy who knows the market really, really well, who is a real expert in the nuts and bolts of mortgage lending, and really knew the collateral really well--but he was a true believer, and I think a lot of people were... true believers in the paradigm.

And there were other people at the firm, say, at the middle of last year, who were not mortgage experts, who were saying, you know, "I see the run-up in housing prices in some of these geographies, and I just don't really get it. I go down to Florida and see the forest of cranes, and I just really wonder, who's going to be in all those apartments... I see these commercials on TV, you know, about low-doc, no-doc mortgages--and there is no way, there is no way that this is not going to end badly... This is the perfect example of a bubble--and we should be short, we should be short sub-prime paper."

But the person who was the expert, the person who ran the sub-prime business, who traded sub-prime paper and issued the CDOs, he was a true believer in the paradigm: "In 2003, people said that the credit quality of the average sub-prime mortgage was deteriorating, and now look, those mortgages have performed fine. The sub-prime market works."

And, hey, he was the expert--you defer to the expert.

This touches on something which I've been thinking about recently: As we rely more on predictive tools and analytics, how can we avoid missing the forest for the trees? Nobody doubts the value of expertise, but what about perspective?

What if the non-expert traders in this scenario were able present a convincing data-driven case that the sub-prime mortgage was a bubble? How many tens of millions of dollars could that have saved?

It seems like traditional BI tools, which were designed by and for experts, rather than line-of business professionals, encourage the sort of tunnel vision that's described in the interview.

Go check it out.

I was honored to serve as a panelist during the session on data and analytics. I was a bit intimidated to be sitting next to Analytics guru Tom Davenport (yes, that Tom Davenport), and Kurt Thearling, who as Vice President of Strategic Technology at Capitol One--one of the companies which Tom refers to as "born analytic" in Competing on Analytics--clearly knows a thing or two about winning with analytics.

The analytics discussion was interesting, but what I found most compelling about the conference was the two visions of technology's role in the enterprise presented by the different sessions. Analytics, as presented by Tom and Kurt, tends to be high-powered statistics, and as such, not something that every line-of-business professional is going to be comfortable with. On the other hand, Web 2.0 is all about empowerment and openness, and we discussed how employers can take advantage of things like wikis and social communities while maintaining the structure necessary to ensure regulatory compliance, accountability and corporate standards.

I don't think that these two things exist in opposition, at least not to the degree that one might imagine. It's clear that most business professionals are never going to be extensive users of heavy-duty statistical tools, and even if some companies, like Capitol One, install SAS on every analyst's desktop, this is unlikely to become standard procedure.

However, if the tools are easy to use, and provide rigorous analysis capability to business professionals in an appealing visual manner, I think that there's a great possibility that they can be rolled out widely, and more importantly gain wide acceptance within an organization. More than that, good tools will allow the results of these analyses be shared and updated by a wide variety of people, allowing insight to be generated and updated in a collaborative process.

To my mind, that's the promise of Pervasive BI, and widely-deployed analytics more generally.

By way of introduction, I am an Analytics Evangelist for Spotfire, a division of TIBCO Software Inc. Essentially, I work with our customers to help them implement analytics and analytics best-practices across their organizations.

As the title of the blog (hopefully) indicates, I'll be blogging about interesting developments in Enterprise Analytics and Business Intelligence.