« happy new year | Main | q4 is turning out to be a huge quarter for expressor »
January 1, 2010
semantic rationalization blog series: part 4 - the benefits of abstraction
In my previous blog, I discussed our approach to semantic rationalization, and introduced the functionality necessary to implement our semantic metadata abstraction layer. I’d like to continue this discussion by talking about the benefits of our abstraction layer.
The key advantage is that all integration and cleansing rules created in our system are built on business definitions, not external names. So regardless of how the data is named or stored externally, once a rule is built for a particular piece of business data, it is automatically available to be used with another datum with the same business definition – regardless of what it is called externally.
This is radically different than other ETL implementations, where even if a business glossary is provided, it is generally built on top of the ETL component. This approach is understandable for prior-generation ETL tools, since it would be cost-prohibitive to re-architect those tools to incorporate this capability at even a rudimentary level. Placing this functionality on top of the ETL component allows for reporting and analysis, but does not accommodate automating the reuse process at the developer level, where it can provide the most influence on productivity.
By automating reuse at the developer level, the expressor semantic data integration system can suggest a list of rules to a developer when he or she picks a column with which to work, for example. It also provides the ability for an analyst or developer to see all the rules associated with a logical name regardless of the interface with which they are working.
expressor’s use of metatyping provides an even deeper form of reusability and productivity. Metatypes are the vehicle used to define the way data is used in the business and can provide guidelines for how the data should look and what operations should be supported against it. From a purist viewpoint, this is similar to the concept of associating methods with objects in an object-oriented paradigm.
This is a very exciting approach that has the promise of fundamentally changing how we view integration. At the core of the approach is the ability to convert data from a complex external environment to a harmonized environment within the abstraction layer. The properties of the metatype assigned to the business definition define the internal representation of the data and the operators that read and write the data (which we call “motors”) are responsible for marshaling the data from the external complexities to the internally aligned formats.
The promise of this approach is that we can isolate the complexities of data type conversion from the developer/analyst, allowing him or her to focus more intently on the business problem at hand rather than the details of how the data is persisted externally. As we begin to deemphasize the physical complexities and accentuate the business complexion of the data, we converge on providing an environment in which the technical business users can participate in more readily.
In the next and final blog in this series, I’ll wrap things up with a discussion of reusability and ease of use.
- Michael Ruland, field engineering
Posted by expressor software at January 1, 2010 1:45 PM
Twitter 
LinkedIn 
YouTube