Sybase IQ

« February 2010 | Main | April 2010 »

March 29, 2010

Guest Blog: Introduction to Fuzzy Logix and In-database Analytics

Hello, my name is Mike Upchurch and I’m the COO of Fuzzy Logix. Our company was founded based on 12 years of research and development related to creating analytics and optimizing their performance. When we started, our goals were to develop a library of analytical models that had very high performance and which could be deployed pervasively.

To meet our performance goals, our team of engineers wrote the models in C/C++, designed them to take advantage of parallelism and to leverage other performance enhancing techniques. Once we had a highly tuned set of models, we found another challenge. We noticed that the performance of analytics was hamstrung due to the time required to move large datasets. As we researched the solution, our CEO, Partha Sen, had a revelation. His question to our engineers was ”Why move the data to the analytics, if we can move the analytics to the data?” And thus begun our quest to convert our large library of data mining and predictive models so that they would run beside the data source; in-database. Effectively, we perform the analytics on the data as it leaves the database and we only move the results to the presentation layer. In terms of speed improvement, we see a 10X to 100X (or more) gain in performance. In fact, the larger the data, the greater the speed advantage. Since the amount of data being created and stored is growing daily, this is one of the best ways for companies to meet the challenges of deriving insight from their ever expanding databases without seeing a corresponding (and linear) growth in processing time.

Once we attained our speed goal, we set out to find a way to make analytics pervasive. There were a number of reasons based on years of observation and experience. For example, we noticed that most analysis was being performed by a small group of people with specialized skills. Not only did they need to understand quantitative models, but they had to learn special languages and complex analytics platforms. We also noticed that statisticians and quantitative teams spent a lot of time running and re-running models on behalf of their business colleagues. Most of the people requesting the analysis could understand and leverage the results of the models, but didn’t have the skills or access to the expensive tools required to run them. This time our question was ”How could we create models that could be run by any device, presentation layer or development tool and push the models out into all areas of an organization?” And the answer was to use the most common data access language ever invented; SQL. To use our solutions, you simply write SQL statements and pass parameters.

The result of combining performance and pervasiveness was that we saw an exponential increase how companies were leveraging analytics. Statisticians no longer had to own all modeling tasks, but could instead build models and give access to end-users via standard reporting tools such as MicroStrategy and Excel. Now users could run their data through the models by simply selecting items from a drop down list or off a menu. Because almost any tool can call SQL, we were able to embedded analytics into business processes and operations. That meant that large numbers of people, in many different areas, could make decisions based on the output of analytic models. The quantitative teams saw other benefits. Not only did they have more time to work on custom models, but they also gained a library of building blocks that could be combined to create higher order models. Since they were able to run the models so effectively on entire datasets, we saw a great leap in their efficiency.

Since we now we had a set of analytical solutions that delivered incredible performance and were easy to deploy, the next question was ”So how do we leverage these advantages to create value?” And the answer will be covered in our next entry; where I’ll highlight some examples of how companies have found success using in-database analytics and Sybase IQ.

Posted by Sybase IQ at 6:37 AM

March 22, 2010

Sybase IQ In-database Analytics: Unlocking Value in Big Data with Partner Solutions

I had recently co-authored an article on Predictive Analytics in Sandhill.com where buzzwords like Data Mining, Forecasting, Predictive Modeling, and Decision Optimization were generously sprinkled. Surely, we had no intention of sounding glib unless these words were significant in what they represent in practical terms. Just to understand how critical these advanced analytical concepts have become in today’s world, you just have to appreciate the context in which they are applied.

In a series of articles on information management in The Economist the focus was on uncovering value from Big Data. Big Data being the context for business intelligence. In one of these articles, The Economist points out that according to a 2008 study by International Data Corp (IDC), a market-research firm, around 1,200 exabytes of digital data would have been generated that year. The critical issue, however, is not just the mangement of such vast amounts of data, but analyzing them using the buzzword concepts above that will lead to a myriad of effective business decisions, or at least corroboration of such decisions, on the basis of computer algorithms rather than hunches. And these new business insights don’t just come from analysis of ”dead data”: stored information about past transactions but also through analysis of real-time information flows. The application areas are both deep and broad – from analytics in capital markets to mining customer data in telecomunication, e-commerce and healthcare. The common roadblock, however, continues to be inefficiencies in sifting through big data in these scenarios. Speed, Accuracy, and Data Volume continue to confound traditional database and analytical tools. Enter Sybase IQ In-database Analytics.

Back in summer of 2009, we introduced Sybase IQ v15.1 with advanced In-database Analytics capabilities that was built ground up to specifically show the business world a new and efficient way of dealing with analysis of Big Data. The key elements of Sybase IQ In-database capabilities rests on the simple (yes, simplicity does offer elegance) fact that if we push complex analytical computations closer to data – good things happen. We have not only embraced this concept ourselves but also invited partners to plug-in their sophisticated algorithms into the high performance Sybase IQ engine. One such partner is Fuzzy Logix – who have ported their extensive array of efficient statistical and predictive analytics algorithms library under the package name DBLytix on Sybase IQ using the In-database Analytics API. In a three part series blog following my write up today, Fuzzy Logix COO, Michael Upchurch, will outline his views on the world of In-database predictive analytics from a real world application angle. On to Mike ….

Posted by Sybase IQ at 3:07 AM

March 15, 2010

In God We Trust – All Others Bring Data!

My friend, Tim, is the plant manager of a custom window company, aptly called Custom Window in Englewood, Colorado.  Their expertise is designing and building windows that have special thermal, blast, seismic and acoustic performance requirements.  They also mimic old designs to retain the original look of historic buildings.  This is a challenging business, since no project is the same – factory lines are continually designed, used, torn down and rebuilt for the next project.

With the maturity that comes from many years of experience, Custom Window is arguably one of the best in the industry.  But a tough economy has rattled this mid-sized company, and business has been slow during the last couple of years.  The meltdown in financial markets and resulting tightening of credit has had a domino effect on business across the board.  With commercial building in a slump, orders for windows have slowed to a trickle.  But Tim’s company is forward thinking. Rather than sitting idly by, waiting for an economic turnaround, management made the decision to implement a new and more sophisticated ERP system.  This would incur significant expense at a time when cost control was vital to survival.  But maintaining the status quo was not an option.  The team embarked on an intense project to significantly restructure their information processing systems.  They hunkered down and looked forward to the promise of better information for planning and management, improved customer service, lower business risk and potentially increased revenues and reduced costs.

The past year of implementation has been painful – technical problems, difficulties running the old and new systems in parallel, and resistance to change from some of the older and more conservative plant workers.  But the end is in sight, and the new ERP system is moving into production.

And Tim is thrilled.  He is getting reports out of the system that are giving him insights into his factory processes that he never had before.  Last week, he wrote me the following note:

”I have created a new labor metric sheet which compares two different ways of recording our labor. It gives us data comparing estimated hours verses actual .... I also decided to record time per lite [pc of glass] to later develop metrics to predict future production based on number of lites per department. It is giving us some really valuable data in which to measure ourselves and eventually give data to the estimating department so they get the hours correct.... I am carrying more people than I need now so to try and beat it I have to run production very efficiently. I am also using this metric sheet to record problems that worked against us and some provide detail showing how we planned and set up this job. Eventually I will compare data on different series and window types, analyze the data, and come up with some good information that will help us run production.”

Tim says that he loves the new ERP system because he can pull all kinds of metrics he never had available before. His new motto is ”In God we trust, all others bring data”. He says ”my shop team is really working hard to redesign and improve our processes. Now that I have data, I can see the fruits of our labors. It has made it all more fun. ”

This is the promise of BI – capturing your data and deriving information from it that will give you the leverage for improved decision making and forecasting.  Sybase’s offering in the BI arena is Sybase IQ.  Sybase IQ is a highly optimized analytics server designed specifically to deliver faster results for mission-critical business intelligence, data warehouse and reporting solutions. Running on standard hardware and operating systems, it works with diverse structured and unstructured data sources to deliver unsurpassed query performance at the lowest price/performance available.

With better information to power your business, you, like Tim with his new ERP system, can be enjoying the benefits of insight based on real data and metrics.

Read more about Sybase IQ here:

http://www.sybase.com/products/datawarehousing/sybaseiq

Until later,

Courtney Claussen

Posted by Sybase IQ at 9:00 PM

March 3, 2010

Sybase in Magic Quadrant for Data Warehouse Database Management Systems

Sybase has been positioned in the Leaders Quadrant in Gartner’s 2010 Data Warehouse Database Management System Magic Quadrant

According to the report, Gartner includes in its Leaders’ quadrant for data warehouse databases ”those vendors that demonstrate the greatest degree of support for data warehouses of all sizes, with large numbers of concurrent users and the management of mixed data warehousing workloads. These vendors lead the market in data warehousing by consistently demonstrating customer satisfaction and strong support, as well as longevity in the data warehouse DBMS market, with strong hardware alliances. Because of this track record, leaders also represent the lowest risk for successful data warehouse implementations, such as lower performance with increasing mixed workloads and database sizes and complexity. Additionally, the maturity of this market demands that leaders maintain a strong vision regarding the key points emerging during the past year: mixed workload management for end-user service-level satisfaction and data volume management.”

The complete report can be found here.

Posted by Sybase IQ at 7:57 PM