« Ab Initio Demo | Main | More Fun »
February 1, 2007
Ab Initio Profiler
I read the documentation for the Profiling tool from Ab Initio. I've worked with their ETL tool before, but not had the chance to use the profiler. They are possibly coming for a demo in the next month or so; and if it looks useful, we will try to convince the people using Ab Initio to pick up the Profiler as well.
Apparently, it not only collects statistics, but does analysis across and within data sets for dependencies and correlation; as well as being able to generate transformation code to use for validation.
Looking through the docs, the only thing I can think of that its missing is trending analysis; and it may just be because I missed it.
-------------
Added Feb. 1, 2007:
We had the demo, see the next entry. I realized that I did't say much about the Profiler in that entry, and it seems more relevant here.
The Ab Initio Data Profiler is better than others I have seen. You can profile an entire dataset, or a sample thereof (with parameters), or you can profile a single run of a graph against that data; allowing you to schedule the heavy load of a full profile, and then keep it up to date piece by piece.
We did run into problems with our initial profile runs on some data, but that is because our server we can run the profiler from is in a different location from where the data is, and we were filling up the pipe between. We were able to throttle the process and keep from hogging the pipe, but it makes things run quite slowly.
What was really impressive was that we got a pre-view of the demo about 3 or 4 weeks prior, and my manager asked about analyzing trends between runs of the incremental profile, and using that trending to determine, in the map, wheter we wanted to load the data to the target. They weren't able to do it at the time, but when they brought the tool in for the demo this week, they had a way to accomplish that task.
I know, it was probably something they were already working on, but it was impressive none the less.
Posted by RDM at February 1, 2007 9:45 AM
