During both that conversation with the storage team and another casual one I was having with our Unix server team, I mentioned the idea that our BI/DW team is starting to have discussions with some data warehousing specialists. I'm paraphrasing, but "ugh!" was the universal response. "We don't like proprietary solutions here. We've standardized on X storage vendor and Y servers. We don't want to support that."

We're only in preliminary conversations about bringing in a specialty data warehouse solution, but I certainly want to maintain a good relationship with our server and storage teams. Any advice on how to help them see the logic and design behind specialty data warehouses?

One of the key operations in the Triadic Continuum data structure is "setting context," and I would argue that this key operation might be prohibitive in anything but the simplest examples. I haven't fully studied and analyzed real-world scenarios, but here's my argument:

First, remember that in the Triadic Continuum, data itself is never duplicated. Every "sub component node" in the tree structure contains not any observed data, but rather a pointer to "sensor nodes" where data is actually stored. Like a columnar database, this is great in reducing the amount of data being stored.

Second, think about the ideas of context, constraint, and focus. These are three key concepts in getting information from the Triadic Continuum. Context is the idea that you have to provide some level of initial boundary to the question being asked: "I can about sales" for example. I can imagine this initial definition of context being brutally difficult. Imagine if you had a single index in a database that represented each column value that appeared anywhere in any column of any table. Here are a couple of concerns:

• That gets to be a pretty big single tree or hash table or anything to search through at the beginning of each query.
  
• Tracing from the sensor node back to the associated subcomponent nodes may result in a very large list of nodes to use in the context. Potentially millions, hundreds of millions depending on how broad the context is.
  

If anyone wants to pay me something equivalent to my current salary to go back to school and study this more thoroughly, let me know!

This particular article is about something called the triadic continuum - yes, sounds like something out of Star Trek, doesn't it? Still, it is interesting - especially if you have a PhD in cognitive science. There's a book about this invention that goes into more detail for those of you who are curious.

After a couple of back and forth comments in which I tried to look smarter than I am (and considering pitching the idea that if my boss wanted to send me back to school for my PhD that I'd be happy to go), this VP ended with: "But they said simple, scalable, universal." Yeah, right there on the package! ;)

My snide remark (being in a healthcare field) was going to be something like:

Right, and so is DNA!
Simple - only 5 compounds in the whole thing!!
Scalable - from a flee to a whale!!
Universal - it is a defining characteristic of life!!

I didn't send that response, yet.
DNA is simple, universal, and scalable, too.

You'll find Dan Linstedt blogging about the Triadic Continuum in the future, I'm sure. I'm still busy reading the book, but my gut tells me there are some very interesting things to be learned from this data structure. I don't think it's the solution to moving along Ackhoff's data-information-knowledge-wisdom curve, but I've already starting thinking about how to leverage some of the algorithms from the book in relational data models (even though that's what the inventors would discourage us from).

You'll see more from me in future posts as I try to decompose the ideas in the book and relate them back to the day-to-day information management challenges we all face.

I know that other people think in different ways. My wife, for instance, reads a sentence (or more) at a time. It took me a while to understand that... "by a sentence, you mean one word at a time trailing into sentences?" "No, I mean I look at the page and see phrases and sentences, not single words." Wow. That's so different than the way I read. Maybe that says something about my intelligence, or maybe it just says something about how different people consume data in different ways.

I have a colleague that I've worked with for some years who likes reports with lots of numbers on them. Now, I'm a bit of a math geek, too -- I got a shirt from my daughter a couple years back with this on it:

01000100
01000001
01000100

Still, I think there's more value that can be derived from that sheet of numbers than can be delivered in the bare numbers themselves. Choosing the right pictorial representation can be very hard, but it can also be very powerful. Here are a couple of examples of relatively simple (and underused) visualization techniques that I think are very powerful.

Radar Charts:
If you plot two or more series on a single radar chart (or spider chart) you can deliver information about both which of the series "covers the most area" as well as in which dimension each series "out performs" each other series. So, you've got both point-by-point comparisons as well as an overall comparison available in a single view. The numeric equivalent might be detailed rankings and an overall average ranking.

Bubble Charts:
This kind of chart also allows you to deliver more than one kind of information in a single view. Bubble charts provide for both a horizontal and vertical plot as well as a third measure that is conveyed using the relative size of the marker on the chart. So, you could plot "number of infections" versus "number of flu shots" for a set of regional clinics and use the size of the marker on the chart to represent "number of nurses on duty" -- if you had a reason to think some pattern might come out of those three metrics.

Gantt Chart:
Don't think I'm just referring to project plans, dependencies, and resources here. Gantt charts can be used convey more information than that. A gantt chart, after all, isn't much more than an "activity" versus "time" chart anyway. Imagine a chart with patient volume on the Y-axis and time on the X-axis. Great. You know how patient volume changes throughout the day or over the year. Add a set of gantt charts to the bottom showing holidays, the school year, and other special events and your chart now allows you to correlate patient volume with various other pieces of information other than "time" generically.

You can imagine adding more dimensions to each of these kinds of charts using shape and color. You could add animation over time to show a trend. You could add sound as another mechanism for delivering information, reflect an upward or downward trend, for example. Three-dimensional charts are fine, but imagine a 7-dimensional chart that uses X, Y, Z, color, shape, size, and animation to convey so much more information. Clearly there's a practical limit to how much information someone can consume from a single view of some piece of information, though.

Still, reports with lots of number on them probably isn't enough to justify a multi-million dollar reporting tool any more.

I'm in the middle of a 5 year initiative that is gutting and reimplementing the underpinnings of a all of the operational systems of a 20,000 employee organization right now. This is happening, though, because the powers-that-be in my organization decided it was part of their mission to become an "information enabled" organization. Besides implementing new and standardized applications across the company to ensure that we could collect lots and lots of information, the leadership also create a Performance Management department, which funds the Business Intelligence and Data Warehousing group. Insightful, I think.

Because Business Intelligence has such a prominent role in the organization, the implementation of reporting functionality (operational and some analytical) is happening alongside the implementation of the new ERP and billing systems. What's even more insightful, I believe, is that in some cases, there are Performance Management objectives influencing how the new systems are being rolled out - specifically in the are of master data management and conformed dimensions.

The theoretical concert between operational changes and business performance management makes a lot of sense - someone with great analytical modeling skills looks at data and determines that the business could perform better if some business process were altered. So, the business process is changed... sometimes by implementing a new application system that can help make people more productive... but that also means the business model and underlying data on which the original analysis was build changes... and the analytical model breaks. So, the data warehousing group waits for the application to go-live so they can rework their technical infrastructure to get the data and put it in some new set of data structures for the analysts to build new models on top of and go through the next iteration of process improvement. Those cycles take years to happen.

Wouldn't it make more sense if the operational applications that are built to improve business performance used a strategic analytical model as an input into what the system should look like and what kind of data it should capture and deliver back to folks doing performance management? Would it make any sense for the data warehouse to be built before the operational application, and be used as a validation that the new operational processes being implemented actually match what was designed to improve business performance? Once a business has gone through an initial cycle of process improvement, who should be pushing whom?

In the world of Business Intelligence and Data Warehousing, here we sit with gobs and gobs of data (rarely enough or exactly the right data, it seems, but gobs none the less). Here we sit with the world in front of us represented in tables and records, entities and attributes, facts and dimensions. The work it takes to derive value from that mound of information is very much a mind over matter challenge. Often, it looks simple on paper, but as you dive into the complexities of reality and start revealing "exceptions to rules" and "obscure business logic" and "poor quality data," it takes real cleverness and fortitude to turn that information into action.

Maybe this wasn't Michael Stipe's intent, but it works for me.

Consider the first phrase "Sharpening Stones." The image I get in my head is of Neanderthals in the stone age chipping flat rocks into point spears that they use to hunt mammoths and saber-toothed tigers. It took tens of thousands of years to get from there to metal spear tips. Lucky for us, technology moves a lot faster. We've gone from the mere idea that a computing machine could be built in the 1800s to incredible small and powerful electronic devices in only 200 years. Still, I think that the information age we're in today is closer to the stone age than the bronze age. The computers that we build are giving us the ability to create and gather information in a way that we couldn't have ever imagined before.... but we're still barely figuring out what to do with all of that information that technology allows us to collect.

We're still just sharpening stones. We take a "collect everything that'll fit on the biggest hard drive approach" to information hoping we find something valuable. We don't have the precision and power of metalworks.