Main | April 2007 »
March 23, 2007
Lure of the Multifunctions
An increasing number of vendors (particularly those positioned in Operational BI) are advocating their low-latency data stores as a way of monitoring Key Performance Indicators from various enterprise sources. Vendors typically name this their “Analytic Engine” which provides such features as:
- Agents that collect, filter and transform content from various sources (databases, messages, applications)
- Memory-resident (or disk if the recovery option is turned on) storage of information harnessed by the above Agents
- Business rules that set off actions (such as Alerts) when certain conditions are met
Although such solutions may provide a method for an organization to be up and running quickly because the extraction, storage, business rules & presentations (sometimes even the hardware) are all packaged together, their true role in the Enterprise needs to be closely examined.
Upon reflection one may come to the conclusion that these solutions do not openly compete with the Enterprise Data Warehouse yet they may be furtively discouraging the organization from making the larger investment in building a comprehensive reporting infrastructure based on a relational data model. Each of the functions provided by these products assumes the role of some part of the EDW stack:
| Function | Replaces |
| Agents | ETL |
| Store | Database |
| Business Rules | SQL |
However organizations need to realize that the price of this seeming simplicity and accelerated delivery may by be these solutions’ inability to handle complexity and their less than enterprise-class performance and scalability characteristics.
Conversely as activity increases in the EDW space, (Netezza IPO announcement and DATAllegro 3.0 release) , EDW vendors should take note of what these vendors are up to and respond by including greater presentation and analysis capabilities in their stacks to avoid losing customer mind-share.
Share:
Posted by Arshak Navruzyan at 11:59 PM | Comments (0)
March 17, 2007
Barrage of Capital Markets Apps
The resume of a Derivatives Business Analyst caught my eye because it made reference to more systems than the typical IT resume:
Bloomberg, Open Link Endur, Charles River Compliance, Charles River Investment Management, Charles River IMS, Sungard InTrader, Reuters, SunGard Global Trader, Summit, Portia, Calypso, Triple Point energy deal valuation application, Charles River Compliance, Latent Zero (Sentinel, Minerva and Tesseract modules), Wall Street Systems, Quotron, Dow-Jones Telerate, Bloomberg Gateway, Bloomberg Portfolio Order Management System, SunGard Middle Office Manager
and if thats not enough, the person is also proficient in generic analysis tools:
SQL Query, business intelligence tools such as Business Objects, Cognos, MS Excel Pivot Tables, SQL, and Show Case Query and Report Writer
Becoming truly proficient in such a multitude of tools has to be a career in itself, which makes you wonder when do they find the time to do the actual analysis?
It can be a sobering experience to see what analysts and traders use on a daily basis to do their jobs: a combination of home-grown web apps, highly sophisticated excel applications that may have more macro code than data, industry apps, etc. which all leaves you with the impression that one place where time is not money is on Wall Street.
Enterprise Web 2.0 can change all this by creating composite applications that do everything that a Wall Street analyst or trader needs from one user interface in real-time and there is a pot of gold awaiting the software company that gets it done in less than a decade.
http://blogs.appfusion.com/?p=14
Share:
Posted by Arshak Navruzyan at 6:23 PM | Comments (1)
March 16, 2007
Sudden Rise of Appliances
In the same week that Gartner declared the term “Open Source” meaningless, “Appliance” is starting to run the same risk.
Vendors ranging from Enterprise Data Warehouse to Business Intelligence have made acquisitions and put together alliance programs to hastily position themselves in the appliance market.
- Cognos acquired Celequest (LAVA appliance)
- Business Objects Open Appliance Initiative (OAI)
- DATAllegro announced V3 on commodity hardware made up of Dell, EMC
The BO announcement doesn’t make it clear if the appliance vendors are going to allow the BI apps to run on their hardware. The statement “customers will have the flexibility to choose the right mix of data warehouse and business intelligence technology” suggests that “mix” means “blades” whereby an a certain number of blades are devoted to (or shared with) data warehouse and the remaining blades are for the BI engine.
If this is in fact what’s behind the BO announcement, there is a name for this type of a “multi-purpose” appliance, it’s called a “server.”
Of course the real reason could be much more sinister such as some obscure accounting rule that let’s you immediately recognize revenues from software if it comes pre-installed on a piece of hardware.
Nonetheless, whatever the motivation, it has to be self-serving. By coupling BI with a DW appliance, both vendors hope to do more business than they can independently.
Lack of customer involvement in the BO announcement is a good indication that this wasn’t customer driven.
Share:
Posted by Arshak Navruzyan at 8:36 AM | Comments (0)
Loading the Active Warehouse
Numerous options are available for loading data from operational systems into the Active Warehouse in real-time or near real-time. The options vary based on cost, complexity and not all options are available for every environment.
Most techniques have been put together to overcome the limitations of application level triggers, timestamps and table differencing. Almost all include some level of filtering and transformations.
The following three categories represent the main approaches. Some products may be a slight variance on one or more of these categories.
| Application Messaging | Change Data Capture, Transactional Data Management | Replication | |
| Method | Publish / Subscribe messages via a message queue (JMS, ActiveMQ, etc) | Log sniffing | Database replication (converted to messages) |
| Synch / Asynch | Asynchronous | Asynchronous | Synchronous |
| Source system cost | System Resources | System Resources | Transaction delay |
| Latency | Near real time however varies w / topology | Near real time however varies w / topology, checkpoint & log switch interval | Real-Time |
| Transformation | Application Server | Database or TDM | Database |
| Advantage | Messages can be also pushed to BI applications. | Can handle extremely large data volumes | Closest to real-time |
| Limitations | Provided only by ultra-new enterprise applications. Data Warehouse vendor may not support the particular the messaging facility used. | May not be supported by RDBMS vendor and possible security issues | Database specific. May not support all message formats 1 |
| Examples | SAP JMS Provider, iWay Adapter for Oracle E-Business Suite | GoldenGate, Informatica, Oracle Change Data Capture, PowerExchange: Real-Time Option, DataMirror | Sybase RepConnector, Oracle Synch CDC |
1. IBM MQSeries, Java JMS, Microsoft MSMQ, Oracle AQ, Apache ActiveMQ, HTTP, TCP/IP, and SOAP
Share:
Posted by Arshak Navruzyan at 6:39 AM | Comments (0)
March 15, 2007
Gartner's View of Open Source BI
Not surprisingly Gartner is not enthusiastic about Open Source BI. At the Summit, Conference Chair Bill Hostmann is quoted as saying:
"[the term is becoming ] kind of like the word "organic" in the grocery business. It's starting to lose its meaning, with some "open-source" vendors demanding licensing fees. Open source is promising, but the business models and products haven't kept up with the commercial products."
The point is well taken and by looking at other software projects that have succeeded in OSS, you can see that OSBI (JasperAnalysis, OpenI, Pentaho, and SpagoBI and others) are still missing two key ingredients:
- Product superior in price/performance, price/functionality to that of commercial counterparts
- A community of outside developers that genuinely believes and is devoting time & resources to the betterment of the software
A further hindrance could be the fact that other famous OSS projects have been geared toward IT and particularly the developer whereas by and large BI starts out by making its pitch to the business user.
Undoubtedly you can argue against many of these statements by pointing to large organizations that have standardized on this or that OSBI product. But where those honest wins over the commercial counterparts or wins because there was a CIO that is philosophically OSS ?
Share:
Posted by Arshak Navruzyan at 3:50 PM | Comments (0)
March 14, 2007
IBM's Dynamic Warehousing Announcement
IBM Dynamic Warehousing announcement is interesting however its difficult to tell what exactly is net new. Further perplexing are the case studies that theyre siting of, Omnium (they have to tell you who Omnium is) and some unidentified law enforcement agencies.
Determined to get to the bottom of whats behind the announcement I found the following white-paper:
ftp://ftp.software.ibm.com/software/data/bi/dynamic-whitepaper.pdf
which contains the following statement:
Dynamic warehousing is not a product, tool or simple one-off solution. It is an approach that enables you to deliver more dynamic business insights by integrating,
transforming, harvesting and analyzing insights from structured and unstructured information.
Which begs the question, when did companies start putting out press releases about "an approach and if they must, why not make that clear in the announcement to save time?
Share:
Posted by Arshak Navruzyan at 2:49 AM | Comments (0)
March 13, 2007
Column-Oriented Databases Come of Age
Vertica's column-oriented database which claims a 3.5 order of magnitude improvement on price/performance compared to row-stores (traditional relational engines) is getting serious attention from high-profile VC sources and industry leaders.
Last month Vertica Systems, Inc. (www.vertica.com) announced $16.5 million in Series B financing from New Enterprise Associates (NEA) and Kleiner Perkins Caufield & Byers on top of the $7 million it had already raised from Bessemer and Highland. Also as part of the announcement Vertica said that Kleiner Perkins general partner Ray Lane, formerly president and chief operating officer of Oracle will serve as a special adviser to the company.
In addition to the world-class investment team, the company has an experienced management team including Dr. Michael Stonebraker, CTO who was the main architect of the INGRES relational DBMS.
Paper that discusses a benchmark they ran against an appliance RDBMS
http://nms.csail.mit.edu/~stavros/pubs/osfa.pdf
Also note an MIT Open Source project in the same area (may or may not be related)
C-Store: A Column Oriented DBMS
Posted by Arshak Navruzyan at 12:31 AM | Comments (0)
March 12, 2007
Oracle's acquisition of Hyperion: 1 + 1 = 1
Articles with ominous titles like “It’s The End Of The Business Intelligence World As We Know It” notwithstanding, what can we expect from Oracle’s acquisition of Hyperion? My guess is not much when it comes to technology and equally little when it comes to marketing. Oracle is positioning the acquisition as a competitive strength against SAP. Oracle President Charles Phillips was quoted in various articles as saying “Now Oracle’s Hyperion software will be the lens through which SAP’s most important customers view and analyze their underlying SAP ERP data.” I seem to recall that SAP was in the same predicament with Business Objects but managed to terminate its OEM relationship with BO and migrate customers to NetWeaver, BW. Also SAP has made it’s own share of acquisitions in related areas:
- Pilot Software (150 customers worldwide) announced just last month
- Virsa Systems for compliance solutions
Furthermore SAP had already been making waves with an Oracle Killer, columns based database technology, which they will now probably redouble their efforts on. In short don’t expect SAP to roll over and play dead.
Now for the technology piece:
Hyperion was the leading provider of OLAP technology, Essbase. Oracle has Oracle OLAP which purportedly is part of Oracle EPM but otherwise has had a tough time gaining acceptance from customers since it didn’t provide a query language such as Microsoft’s MDX but rather required Java code or SQL/PLSQL which would allow you to see slices of the cube from a relational prospective (kind of defeats the purpose).
Then there is HyperRoll which for a while had a patent infringement lawsuit against Hyperion that was settled as a result of Hyperion making an investment and licensing HyperRoll’s “Database Acceleration” technology, similar to a materialized view except it ran outside the database and performed the aggregations much faster (http://www.tdwi.org/News/display.aspx?id=8158).
It remains to be seen if the HyperRoll capabilities will be fully incorporated however in the short run I’d expect the integration efforts to be similar to what Hyperion was doing with Teradata before the acquisition. (see recent issues of Teradata Magazine for numerous articles and Teradata white papers). In a nutshell the Teradata/Hyperion Hybrid Analysis message was this, “use Essbase’s Aggregate Storage Option to pre-aggregate data at a high level and drill down to the lowest level in the Hyperion hierarchies and when you run out of levels we’ll do a pass-through query to Teradata.”
In fact substitute the word “Teradata” with the word “Oracle” in these joint marketing materials and you have their version 1 integration strategy which they’ll be pitching for the next 3 years.
What is yet to be seen is what move Teradata will make in responce (now that it is getting ready to spin off from NCR). “Where did we put those DecisionPoint manuals? …” (http://www.teradata.com/t/page/143692/index.html)
Posted by Arshak Navruzyan at 5:36 AM | Comments (0)
March 11, 2007
Flex Related Open Source
If youve followed posts on Flexcoders about issues with Flex Data Services (FDS) 2.0.1, like this one, you may sense the necessary frustration and disenfranchisement that has given birth to so many other Open Source projects.
While the FDS Hibernate assembler seems to still have issues when dealing with complex data structures, an Open Source project may beat Adobe to the punch by creating a reliable, scalable and relatively bug-free implementation of ActionScript object serialization/deserialization.
The various Open Source projects are aiming at supporting both AMF3 and RTMP (Real Time Messaging Protocol) in order to provide:
- ActionScript object serialization/deserialization and remote method invocation
- Messaging including the ability to subscribe to JMS topics
- Video / Audio Streaming
The specification of AMF3 and RTMP has not been published by Adobe however these Open Source projects have managed to mimic the protocols:
- Granite Data Services (http://www.graniteds.org)
- Red5 : Open Source Flash Server Open Source Flash (http://osflash.org/red5)
One of the unique capabilities that RIAs can offer is of course messaging which in many enterprises isnt confined to just JMS. In good time, Id look to these OSS projects to also support ActiveMQ, MQSeries and MSMQ.
http://blogs.appfusion.com/?p=5
Share:
Posted by Arshak Navruzyan at 5:59 PM | Comments (0)
March 9, 2007
FlexManiacs 2007
June 25-26, 2007. Washington, DC
http://flex2conference.figleaf.com/
Highly Coordinated Visualizations using Charting
Coordinated visualizations allow users to simultaneously view inter-related data from multiple perspectives to perform correlation analysis and discover patters that might otherwise go overlooked. The session will discuss how Flex 2 Charting component coupled with a coordination API can deliver sophisticated coordination actions including hovering/filtering, coordinated selection, coordination paging/scrolling and others. It will also discuss coordinations with custom developed visualization components such as Treemaps.
http://blogs.appfusion.com/?p=4
Share:
Posted by Arshak Navruzyan at 5:56 PM | Comments (0)
