April 22, 2008
Closing the loop
Through the years many people have talked about "closing the loop" in business intelligence. There have been many definitions - whether it's about focusing on areas where business rules can capture the entire work flow and in a semi intelligent fashion drive actions without human involvement (or with just limited alerts to humans), tying real-time and historical data into an analytic workflow, or whether it's about letting consumers of reports be able to rapidly share insights and potentially take actions in operational systems (e.g. letting a user of a SAP report affect settings and procedures inside SAP).
What we're now getting closer to is to be able to pair up the speed and processing power of computers with human intelligence - letting computers analyze potentially thousands of rules per second and when new opportunities emerge or problems occur, let humans into the loop presented with pre-defined applications, pre-loaded with data and analytic workflows, that allows those humans to rapidly assess problems and opportunities - and either update rules or take actual actions in operational systems. We're making some great headway in this direction and will look forward to speaking more to you about that. There is a unique opportunity in the history of computing here, finally pairing up the very best of human intelligence and computer smarts, speed, and stamina!Share:
August 5, 2007
Thinking big about the currency of the 21st century
As the world has developed throughout the centuries the key currencies have shifted over and over from staples such as grains, foods and crops, copper, bronze, iron, salt (with wonderful variations over geographies as well as time periods) into actual coinage and later into paper money when coinage for larger amounts turned clumsy (first happened in Sweden in 1661). Many of these currencies all the way up until the early 1970s were highly correlated to and underwritten by the value of the actual metals they were representing - i.e. you were guaranteed the ability to exchange a $dollar(US) for the corresponding value in gold until 1971 when the United States' suspended convertibility of dollars to gold.
As we've seen financial markets develop over the last 25 years with a wealth of data surrounding them something very interesting has happened.
Knowledge of the future has always carried an enormous premium - think back at Julius Reuters in 1850 using a fleet of 200 pigeons to transfer information between Brussels and Aachen within two hours, beating the railroad by six hours. or think about the billions and billions being spent by governments every year in intelligence - just to be able to get a glimpse into the future.
Being able to know something about the future of a financial instrument has arguably become more valuable than owning it itself (you can clearly argue that you'll have better leverage on capital in such future knowledge than having your money locked up in an actual underlying instrument).
What the financial markets have taught us here is that information that can tell us something about the future is a currency in itself, and we're quickly entering a world where every (!) industry will come to realize that such insights will carry greater value than the underlying assets or processes from which they were derived. Information that can tell us something about the future is becoming a currency itself - and I will argue it's by far the most important currency of the future.
Below I will lay out some examples. Obviously you can shoot down each and one of them based on our current (in)ability to actually say something about the future - as well as based on the stronger argument that some things may never be very predictable - which are fine points, but my over reaching premise is simple: the pendulum is swinging towards an era where we can enable man and machine combinations to say something about the future better than ever (if you don't believe that, stop reading).
- Would you rather be a producer in an oil field or be able to predict where oil can be found?
- And accordingly be able to either sell such information
or be able to buy/sell oil fields/patches without bothering with dirty/complicated oil drilling...
- Would you rather hold lots of inventory of Christmas toys locked up in a global supply chain or be able to predict what ports and shipping terminals are going to go into gridlock before Christmas?
- And accordingly be able to buy up port/shipping capacity elsewhere...
- Would you rather have 10,000 customers that you know will be loyal over
5 years or 100,000 customers and being really unsure about who will stay and who will leave?
- And accordingly be able to invest in improved service for those 10,000 loyalists.
- Would you rather own a factory and its ability to turn out goods or the ability to predict when it will come to a standstill?
- And accordingly be able to tell the factory manager what he needs
to do right, or if you don't like consulting businesses, be able to
sell him a product that predicts when problems will arise, or even
better be able to provide back up manufacturing capacity when needed
(at a nice premium price point).
- Would you rather run/own a pharmaceutical company and its ability to produce cancer drugs that might in best cases work in 30% of the cases or have the ability to predict what drug will work on what cancer patient?
- And accordingly be able to actually improve that chance of success
dramatically for your own drug, or even be an information arbitrator
that doesn't even need to develop the drug.
- Would you rather insure 10M people at the same rate or segment them very carefully to a point where you are matching risk and premium exceedingly well?
- Yes, of course this can be morally complicated - but stop crying, it
will happen, and if the insurance companies aren't using it towards
consumers, consumers will be ganging up against insurance companies
to gain an information advantage against them. The insurance business
will be very different in the future.
- Would you rather have an army of 1M men with old rifles and equipment or an ability to predict where these 1M men, if they were your enemy, will show up and have an army with 50,000 men ready to meet them at the exact right time and place?
Now, of course you may say "is this just some dumb old artificial intelligence overconfidence in what computers are/will be able to do?!". I think that is a good argument actually, but the interesting point is that there is a counter effect built into all of this. The better we get at nailing one of these prediction problems you can be very certain that other people will be able to replicate it, and accordingly markets will neutralize any gain you can get to faster than may feel comfortable. And rest assured that this argument doesn't hold true only for financial markets (we all know the old argument that if you had a great stock picking algorithm the knowledge would soon slip out and the algorithm would be worth nothing) - this will hold true in every market and domain. If you devise a magnificent way of predicting supply chain flow and problems, somebody else will counter, and fast.
So, given that you might say "why even waste your time on this?" Here the point is I think that you can and will be able to gain some truly majestic information/competitive advantages - but you have to expect them to be short lived in themselves, so you need to equip yourself to invent over and over again. You will be in a constant information and analytics warfare. A brutal contest. And btw, if you don't you'll be outplayed. You will not even have a ticket to the game.
What does it mean to be in information and analytics warfare? Step #1 I can't really help you with. You need to hire and retain remarkable people - who understand both your business and understand what it means to compete on information/analytics. But you've always needed to hire and retain remarkable people to win, so perhaps no difference there.
The other end of the coin, Step #2, is that you need an infrastructure that is built to be remarkably adaptable to new business problems. Remember, you will be in a constant innovation game, and you will win based on your ability to innovate.
This means you need an infrastructure that is equally adaptable for both man and machine, as fast as business owners (yourself perhaps!) come up with clever ideas, and you need to be able to convert these ideas into action. This means deploying new user experiences, new algorithms, new applications, new data feeds, etc. - basically at the pace of business (think carefully about what that means, pace of business - not pace of software development, not pace of traditional systems roll-out, etc. - pace of business!). Building such an infrastructure should not be a pipe dream by any means. Perhaps you will not get there through some magic overnight transformation, but the bottom line that I want you to carry away from reading this is that these architectures exist, and you can put yourself in the pole position today.
Have fun doing so!
May 1, 2007
TIBCO To Acquire Spotfire
Today we announced that Spotfire is being acquired by TIBCO (NASDAQ: TIBX), a leading supplier of real time infrastructure software.
Over the past 10 years, we've built Spotfire to be the leading and most innovative (not so modest!) designer, developer, and supplier of software for finding interesting insights in data - based on an awesome user experience and application infrastructure. We've had the opportunity to work with fantastic customers (you!) across the world to make this happen.
Spotfire is at a very exciting point in time. I've talked in this blog about business intelligence 2.0 and what is driving the change in the market - user experience, speed, interactivity, relevance to business problems, etc. We've seen an incredible momentum in the market for our offering across nearly every type of business.
At the same time we've also seen how customers are looking for robust infrastructure as it relates to driving timely data to decisions. The traditional approach which results in a 2 year data warehouse project followed by mountains of static reports emailed out every week just doesn't work anymore. So - it makes a whole lot of sense for us to combine with the team at TIBCO - allowing us integrate our user experience and next gen BI platform with absolutely the industry’s leading SOA infrastructure.
Together TIBCO and Spotfire will be able to provide an unprecedented user experience coupled with an infrastructure that can drive data in real time to that user experience. I believe this will disrupt the BI market as it stands. Think of portfolio managers, sales managers, or manufacturing managers – in fact, anyone! – who need to make critical decisions in a world drowning in data. Now they'll be able to interact with that data in a real time interactive experience configured for their type of work (their "business process")! Who else can do that for you?! (not so modest again!)
I am very excited as you might read into the above text. We will have a lot of fun continuing to build Spotfire as part of TIBCO. We will be the Spotfire business unit of TIBCO, preserving the Spotfire name - but will have great opportunities to increase our presence in the market, increase investments in R&D and product development, and of course, leverage the TIBCO product infrastructure to help you create an information advantage for your business.
Looking forward to seeing you in the market place!