Main | May 2006 »
April 28, 2006
The Spotfire Type System vs. SAS
The type systems of SAS and Spotfire DecisionSite are quite different, SAS uses two basic types, numbers and strings, and then there are a number of different input and output formats that can be used to translate the data. Spotfire DecisionSite has a much tighter type system with six basic data types (string, integer, double, date, time and timestamp, for these we have defined rules for how comparisons and distance calculations in between values are performed. On top of these types we also have the concept of output formatters.
When importing SAS data to Spotfire we have five different methods to choose in between for conversion:
- Use the raw SAS data without conversion (in string or numeric format)
- Use SAS output formatters to convert the raw values to strings formatted in the same fashion as the SAS user is used to see the data
- Use SAS output formatters to convert the raw data to string, this string is then parsed to one of the six native Spotfire types
- Convert the raw SAS data directly to a native Spotfire type
- Use the raw SAS data in Spotfire DecisionSite and apply a Spotfire data formatter
In the Spotfire DecisionSite 8.2.1 version of the SAS data file access we have used all of these methods except number five. The choice of what method is used is controlled by what SAS output formatting is stored in the SAS data file. The user also have the option to use the raw data (option one) for all columns.
To set up the rules for what type of conversion to apply for different SAS formats was one of the most difficult tasks during the development of the SAS data file importer.
In most cases we could find good ways to map SAS formats to Spotfire types, the biggest issue we found hard to resolve is the handling of SAS time based formats. The main culprit was that the Spotfire time data type only has a valid range of 00:00:00 to 24:00:00 and the SAS time type supports both negative times and more than 24 hours. This was one of the cases where we have choose to fallback to importing the raw data, to make sure that no information is lost. Converting it to string would yield nice output but the distance (and possibly order) would be lost.
More information on how the type mapping is done can be found in the help system of Spotfire DecisionSite 8.2.1 :-).
Regards,
Jonas
Posted by Jonas Lagerblad at 4:15 PM | Comments (0)
Integration of SAS data files into Spotfire DecisionSite
From version 8.2.1 Spotfire DecisionSite offers a seamless integration with SAS data files, in this and some upcoming articles I will describe some of the challenges we had to overcome to make this work.
The goal for the development of the SAS data file ntegration into Spotfire DecisionSite was:
- Make data from SAS data files (.SAS7BDAT) as easy to use in Spotfire DecisionSite as data from text files and excel files. It must be possible to open SAS files via File/Open, File/Import and drag and drop. It must also be possible to add new columns of data from SAS files
- Allow usage of SAS data files as targets for linked data in a Spotfire DecisionSite analysis
- Import SAS data in a format that is as similar as possible to the experience to how the data is presented in SAS
- The imported data should be useful for analysis in Spotfire DecisionSite, i.e. numerical data should still be numerical etc.
- No information should be lost during import
The technology platform that we choose for the integration is the local data provider, from the SAS Providers for OLE DB, which is available for free download from SAS. This OLE DB provider supports reading data from SAS data files, which are generated by SAS version 6 and newer.
More on type systems in the next article.
Regards,
Jonas
Posted by Jonas Lagerblad at 3:45 PM | Comments (0)
April 21, 2006
Exciting new features in Spotfire DecisionSite 8.2.1
Last week Spotfire released version 8.2.1 of Spotfire DecisionSite – this is really a great release.
One of the most exciting new features is the seamless integration of data import from SAS data files into the product. I will cover this feature more in a later blog entry.
/Jonas
Share:
Posted by Jonas Lagerblad at 10:00 PM | Comments (0)
The Spotfire interactive data experience co-Founder
I'm Jonas Lagerblad, VP Product Platform and co-founder of Spotfire.
This month Spotfire celebrates its 10 year anniversary of Spotfire as a company. 10 years ago, we were 5 entrepreneurs with a great idea of how interactive visualizations should save the world sub-renting a small office in Gothenburg (Sweden), now we’re a global company with 40 times more employees.
My primary focus has always been to get high quality products that address business problems in the hands of customers. To achieve this goal I've had quite a few different roles at Spotfire, developer, software architect, project manager, VP Engineering, product management, IT support, … (I'm also the one to blame for the name Spotfire)
I will use this space to share some ideas and reflections that have come up during this time.
I invite everyone to come with feedback both on our products and on ideas of how to apply data visualization on business analytics.
See you later,
/Jonas
Posted by Jonas Lagerblad at 9:30 PM | Comments (0)
