« new expressor eval edition demo video on YouTube! | Main | expressor gains strong momentum »
September 29, 2009
This work is actually quite embarrassing
The need for improved performance in data integration applications requires a new way of thinking about parallel processing. We need an 'a ha! moment.'
The same way chip manufacturers so radically improved hardware performance with multi-core processors. When you think about it, it is almost too simple a concept. I mean, if you wanted a six pack of soda you wouldn't run to the store and pick up a can at a time would you? I wouldn't.
But it wasn't that long ago when the heat and communication challenges of multiple CPUs brought chip manufacturers to the physical limits of current technology. Their 'a ha! moment' was simply combining multiple CPUs onto a single chip to fix the communication issues. The heat issue solved itself because these multi-core processors required less printed circuit board space and power.
The problem with most software vendors is that they have been riding the coattails of the hardware manufacturers for years - adding endless new features on the same old technology base rather than start with a 'clean sheet' like chip manufacturers did with multi-cores. Case in point: back in the mid '90's my word processor was faster and more responsive than it is today. When is it acceptable for software running on a single 486 33MHz processor to be just as fast as or faster than software running on a 3GHz quad-core? To me the answer is simple: never.
Some even defend this steady evolution - as an IBM software exec did in Stephen Swoyer's article in Enterprise Systems Journal when expressor first hit the market, rejecting 'the idea that just because a technology is old - it's somehow inefficient or obsolete.' But it was pretty clear to us that what was really needed was a radical mutation. A clean break with the old way of doing things.
Now there IS one segment of the software industry that has consistently innovated to improve performance, and that is the gaming market. The drive to innovate that motivates the gaming industry should be the model for all computing technology companies. So maybe it's not so surprising that expressor uses a new concept for parallel processing developed for the gaming industry - called 'embarrassingly parallel' - to deliver the breakthrough DI performance that sets us apart from existing players.
Embarrassingly parallel processing simply reduces or, in the case of expressor, actually eliminates communication between parallel data processing tasks. As with all data integration software, data movement is the main task, so we were open to any approach that improved data movement. But most existing data integration software continues to rely on outdated, communications-intensive parallel processing approaches that cost you computing resources.
Can they fix this and bring back your missing data processing resources? Not very likely, as it would require a complete rewrite of their software and a subsequent forklift upgrade for customers. So when looking for a high-performance data integration solution, you should look at the vendor that started with an 'a ha! moment' and got it right from the beginning - expressor software.
- John Russell, co-founder and chief scientist
Posted by expressor software at September 29, 2009 11:45 AM
Twitter 
LinkedIn 
YouTube