BeyeBLOGS | BeyeBLOGS Home | Get Your Own Blog

June 11, 2010

SAP BI for the Rest of Us

My first entry is really more of a white paper than a blog entry, so be forewarned: it's a bit on the long side. But if you want to get a good idea of how I see SAP BI as it exists today - and what can be done to make it better - read on . . .

For almost 40 years now SAP has been the industry leader in providing software that provides companies with a way to standardize and automate business processes in order to provide for never before seen levels of enterprise-wide efficiencies and productivity. As the SAP ad slogan states, “the best run companies run SAP”.
However, in recent years as more and more customers have begun hearing of the wonders of this exiting new thing called simply “BI”, SAP has found out the hard way that it can be difficult to do everything yourself – or at least do it well.
Starting in the mid 90's SAP teamed with Seagate Software (followed by Crystal Decisions / Business Objects) to provide their customers with a third-party solution to supplement the built-in reporting functionality available in SAP R/3. This solution was Crystal Reports, which was positioned as the preferred solution for “highly formatted reports”. Later in 2002 support for Crystal Reports was expanded to include SAP BW 3.x.
Around 2004 SAP made the decision to no longer look to a third-party BI solution like Crystal Reports to meet the more demanding BI requirements of its user base. They announced that a new native solution would be made available with NetWeaver 2004s that would replace the functionality previously provided by Crystal Reports. Because of this many SAP customers made the decision to forgo any planned Crystal Reports develop efforts and wait for the new solution to be released. As is true with virtually any new software product the solution took a bit longer to release than originally anticipated. When the new Report Writer was released the SAP customer base was understandably anxious to get their hands on it.
As it turns out, SAP found out the hard way that it takes more than a couple of years of development effort to replace the functionality of a product like Crystal Reports, which at the time had been through over 15 years of continual development. It became immediately evident to the SAP customer base (especially among those who were familiar with Crystal Reports) that the new Report Writer was a far cry from Crystal in terms of functionality.
To SAP's credit they quickly realized that the strategy of providing a home-grown BI solution wasn't working out as planned and made the decision to purchase Business Objects in the fall of 2007. It is now almost two years later and the response to this new direction has been mixed, with many SAP customers still unsure as to what to think in regards to the on-going development of new BI content.
Part of this uncertainty naturally stems from the fact that these new BI tools are unfamiliar to many SAP customers, which is to be expected. For some, it comes down to a simple matter of trust. Regardless of the underlying reason or reasons, many SAP customers find themselves still sitting on the “BI sidelines” as we reach the mid-point of 2010. This represents a tremendous challenge (and opportunity) for SAP as it attempts to re-assure its customer base that it has what it takes to lead to next level in terms of their business intelligence capabilities.
In a word, the SAP customer base no longer wants (or needs) be told, but shown how. And quickly.

The Challenge
It is impossible to know for certain how many SAP end users would consider themselves satisfied with the amount and quality of information they are able to access from their SAP environments on a daily basis. However, if they are anything like the average knowledge worker at large in business today, most likely the majority would be at least partially, if not greatly unsatisfied. Even today in the year 2010 this remains for many knowledge workers the single biggest point of frustration as they attempt to do their jobs day to day, and it certainly stands true for the SAP user community.
You can read a lot of very interesting and well-written articles posted on numerous websites today that in one way or another attempt to explain the reason for this continuing, pesky disconnect between the raw capabilities of today's advanced information systems (like SAP ECC) and the persistent lack of satisfaction among the knowledge worker community. Many of these theories focus on need for more advanced, easier to use interfaces (better tools) or the inability of end users to use the tools (better training) or perhaps some combination of the two.
While no one would argue that either one of these two factors are not important to consider in their own right, I would like to present at least one more underlying factor that has persisted ever since the creation of the very first computerized business information system:

The people who make computer systems tend to greatly overestimate the capabilities of the business end user community or, if they don't, they tend to greatly overestimate the capabilities of their new and improved “user friendly” interfaces.

My guess is that most information technology solution providers lie in the first camp – they think their end users are capable of handling a lot more than they actually can. This has always been the case in the world of computer technology, whether it be hardware or software. Perhaps there is no better example of this mentality than the “more is better” philosophy that all but permeates the entire IT industrial complex. And, like any cultural fixture that has persisted over time, virtually no one inside the culture recognizes it as either existing or, if they do, as any sort of problem.
Beyond a doubt the most pervasive example of this “more is better” mentality would be the Microsoft Office suite of office productivity tools (Word, Excel, PowerPoint, Outlook, Access, etc.). The sheer number of features available now in any one of these productivity tools is mind-numbing. For a variety of reasons (primarily incremental revenue), Microsoft has felt compelled over the years to add more and more functionality to each of these tools to the point of becoming almost ridiculous.
It can be argued that, even though the average Office end user may only use a small fraction of the complete feature set available, there will always be a small percentage of end users who are fully capable of utilizing virtually the entire feature set and who's productivity would be diminished if those more advanced features were not available. This of course is quite true. Within any specific end user community (whether it be MS Office, SAP ECC, or Blackberry users) there will always be that top 5 to 10 percent of users who are capable of fully leveraging most if not all of the available feature set of a given technology. In some user communities this number may even approach 20 percent or more.
But one thing is for certain: the 80/20 rule is alive and well and very much applies to your typical technology end user community. What this means is that, while it certainly may be both appropriate and expected to add advanced functionality into any given system (whether is be SAP ECC or an end user BI tool), don't expect many of your end users to be able to leverage that additional functionality, no matter how much you may want them to. That simply is not realistic and ignores a very simple fact about your average office worker: they will only learn what they absolutely have to learn to get their jobs done. Anything beyond that is a luxury that most of them cannot afford. In other words, if I know that someone else in my organization can provide something for me I won't take the time to learn how to do it myself.
The best example of this is something I call “BEx 1.0”. When SAP released SAP BW into the world years ago the only option for anyone wanting any real information out of SAP BW was to use the Business Analyzer (or, as it's know, BEx), which is simply an Excel plug-in that allows data to be pulled directly into Excel from SAP BW.
Now, in terms of complexity of the end user interface it can be argued that the BEx interface is one of the simplest around. Essentially all the end user has to do is open a query, perhaps answer a few prompts, click “Execute” and then, viola, the data appears in my spreadsheet. Nothing could be easier. Or at least it seemed at the time.
In reality, people found out there's a bit more to gathering real, usable information from any data system than simply running a query inside a spreadsheet. Without going into all the various reasons why, the bottom line is that, for the majority of end users who were “trained” on running reports using BEx, the last time they used BEx themselves was in a training class. Or perhaps they tried it out once or twice after training. No one knows for sure the final percentage of “trained” BEx end users who actually went on to use the tool in any useful way, but I imagine it would be too painful for many SAP customers to find out. Most would rather just move on.
What actually ended up happening was quite predictable: by the time the first break came around in the first BEx training class at least half the class had figured out there was no way they would ever “get it”. By the end of the first day of class much of the other half had it figured out as well. Then it became a matter of figuring out who were the ones in the class who did “get it” and make sure you make friends with them as soon as possible, because guess who you're going to go to to get the information you need after the class is over?
So, while the original idea behind “BEx 1.0” was the creation of the self-sufficient, self-servicing office worker, in actuality it made a few key individuals even more popular (and valuable) than ever. These few who rose to the top of the class became the keepers of the infamous “spreadmarts” that have since sprouted up within SAP customer sites everywhere. The end result is you have a few highly specialized individuals within each organization who are servicing the BI needs of the majority.
Now, anyone looking at this situation might conclude that the reason why most end users never “got it” and ended up relying on someone else was because they had overestimated the capabilities of the average end user. And that would be true. However, this is where it becomes easy to make the second mistake commonly made by the producers of technology: to overestimate the capabilities of some “new and improved” user interface.
On the surface it would seem that no one in their right mind would argue against a better or more user friendly interface for anything. Why would it ever a bad thing to make something easier to use?
When you think about it, though, it is actually quite easy to make any given tool too easy to use. Any tool becomes too easy to use when it becomes too limited to accomplish the task at hand. Sure, it's easy to use. Too bad I can't get my job done with it.
This is the basic problem with the various “natural language” query tools on the market: they are not only simple to use – they are simplistic. Can they be useful? Absolutely. But it is careful that you keep perspective with these tools. They work fine in more predictable, simple, and limited scenarios. Simple answers for simple questions. However, as soon as you get beyond the most basic types of queries (“How many widgets did we sell in March?”) these natural language tools tend to fall apart. Or they take so much work on the front end as to make it impractical.
So, while everyone dreams of having a computer they can walk up to Captain Kirk-style and ask “How should I run my business in a down economy?”, it's not likely to happen in our lifetimes. Until then, we have to live within the limits of the technology available today – and within the constraints of the real capabilities of our end user base.
Of course there currently exists a wide range of “user friendliness” among the BI tools available on the market. This is definitely true within the toolset available from Business Objects. Right now a lot of excitement is being generated among the SAP customer base about the promise of “self-service” BI using tools like Webi and Pioneer. And while these tools will certainly help in providing more direct access to information housed in SAP BW, I am afraid that, if it isn't careful, SAP will end up with “BEx 2.0” on their hands real soon.

Avoiding BEx 2.0
What do I mean by “BEx 2.0”? Simply this: these new tools are nowhere near the rather simple but limited “natural language” tools we mentioned earlier in terms of ease of use. And I would argue that this is a good thing. What this means is these tools have greater functionality and flexibility to allow end users to get real answers to the more complex questions they face. These are tools with teeth.
However, because these tools are quite powerful they also require a certain level of user to fully leverage the functionality available within the tool. While this is certainly true in the case of Pioneer it is also true with Webi, albeit to a lesser degree. Any time you give a tool to an end user that requires them to know anything about the structure of the data or how to organize it in any sensible way, you just cut your potential audience dramatically. In other words, it doesn't take much with the current state of the end user population at large to tip the cart.
Obviously this is changing. The level of sophistication of the end user base in general is increasing all the time. There are business end users entering the ranks of corporations today that were practically born with a mouse in their hand. These younger, technologically aggressive workers many times would actually rather do it themselves rather than be dependent on someone else, especially when they know all they need is a computer and a little know-how.
That said, this just means that some corporations are approaching (or are on the verge of breaking) the magical 20% mark we mentioned earlier in terms of the number of truly “self sufficient” knowledge workers in their organization. However, there are plenty that have yet to get north of 10%, and most likely will not do so for some time because of the nature of their business and end user base. The point is this: Pioneer and Webi require more from an end user in terms of sophistication and willingness to “get their hands dirty” than a tool like Crystal Reports. And while there certainly will be a number of users within any organization ready and willing (and able) to use these tools effectively, it is hard to argue that they are anywhere near the majority.
At the end of the day, there is not a lot of difference between these two BI tools and the BEx interface in terms of pure ease of use. Sure, they look a lot better. But let's not confuse a better looking interface with ease of use. In terms of what is actually required from the end user these tools are not much different from the original BEx user interface. To get the most from each tool you need an end user who has both the know-how and willingness to manipulate and format data on their own.
The bottom line? Sure, the end user base has gotten a bit more sophisticated and the new “end user” BI tools (Webi and Pioneer) have gotten a bit easier to use, but have we now reached the point where the majority of end users in any given organization can now be treated as truly “self sufficient?”. I would argue that it would be a bit premature to declare the days of the “dependent” end user over and gone, not by a long shot. My concern is that SAP will end up committing the very same mistake they made when they originally introduces BEx to their end user community years ago and end up over-promising and under-delivering on the capabilities of these new BI tools. That would be very big mistake.
(For another perspective on the topic of “self-service BI”, here's a great article by Wayne Eckerson of TDWI:
What is needed is a more realistic approach to SAP BI – an approach that recognizes the real-life limitations and constraints that are still very much in place in the SAP end user community.

Being a BI Realist
One thing is for certain: the technology industry has a very short attention span. It seems that as soon as the latest and more advanced solution hits the streets everyone begins scanning the horizon for “the next new thing”. This mentality is a close cousin of the “more is better” mentality we discussed earlier. Always ready to push the envelope, it seems we can never settle down and get really good at any one thing. It often seems like, as an industry, we have a really bad case of A.D.D.
Along with this comes a tendency to get really excited about the latest and greatest “cool” technology and completely forget the tried and true. Unfortunately for our customers, we oftentimes end up promoting these newer technologies without first giving them a decent chance to prove themselves in the real world. And let's not forget the 800 pound gorilla in the room: the newer, “sexier” stuff sells a lot better than the “old” stuff.
The obvious problem with this mentality is it makes the very flawed assumption that the new stuff is somehow better (or more appropriate) than the old stuff simply by virtue of the fact that it's newer.
I am concerned right now about the lack of balance in the SAP BI arena in terms of attention being paid to the various new BOBJ BI tools that were recently acquired by SAP. If SAP were buying cars instead of BI tools in 2007 then Xcelsius would be a brand new 2007 Porshe Carrera, Webi would be a 2005 Camero Convertible, Pioneer would be a Lamborghini concept car, and Crystal Reports would be a 1988 Volvo station wagon. Now, when it comes time to have your friends over for drinks, which car are you going to roll out of your garage first?
Crystal's biggest problem right now is that it's old news. And it's just a “report writer”. Nothing else. It doesn't do any one thing with much flash or pizzazz. It just gets the job done. Dependable and boring. Kind of like a 1988 Volvo station wagon.
And Crystal Reports suffers from a big identity complex. In a way, it's a victim of it's own success. It has always been so good at creating “pixel perfect” (or “highly formatted”) reports that it now has become synonymous with the terms. If you ask anyone in IT what Crystal Reports is good for, the most likely response would be “to create pretty reports”.
While that may be true, I would like to present to you what may be the most compelling reason of all to include Crystal Reports as a central part of any organization's BI strategy: it is perhaps the most approachable and versatile BI solution on the market today.
By “approachable” I mean that Crystal Reports allows organizations to create easily consumable BI content that is appropriate for the majority of business end users in the organization. It can be argued that the end product of Crystal Reports (a report) is perhaps the most easily recognizable and simplest information delivery vehicle known to man. It looks and feels familiar.
And it can be argued that Crystal Reports is the most versatile of all the new BOBJ BI tools. You could think of it as the “Swiss Army knife” of BI tools. When it comes right down to it, there is very little that you cannot accomplish in terms of packaging and delivering information using Crystal Reports. It terms of sheer flexibility and content producing capability, it can be argued that there is no better tool on the market today.
So why isn't Crystal Reports used more often as a BI interface in an SAP BW environment? Even though it has been integrated with SAP BW for over 8 years now SAP customers have been very slow to incorporate it into their SAP BI delivery strategy. Why is that? In addition to some of the reasons mentioned above, here are some potential reasons why SAP customers may be hesitant to deploy Crystal Reports in against SAP BW (I will first list the reasons and then examine them afterward):
Crystal is not a “true” OLAP interface. This is quite true. Crystal Reports is not an OLAP interface, but rather a “flat” report writer. In other words, you cannot do any real “slicing and dicing” with Crystal Reports.
Crystal is a static reporting environment. It is true that Crystal Reports has been traditionally used to create static “one look” reports. Additional “looks” at the data would require the creation of a new Crystal Report.
Crystal Reports have to be developed by IT or by a power user. Once again, Crystal is guilty as charged. Developing anything beyond the most basic reports typically requires someone who is trained and skilled at creating content using Crystal. It is definitely not a tool for the casual end user.

Now let's examine each of these different reasons for potentially excluding Crystal Reports from your SAP BW front end solution set:
Crystal is not a “true” OLAP interface.
Yes, it is true that Crystal is not an OLAP interface and therefore, at least on the surface, does not appear to be a good match for examining data housed in a data warehouse environment like SAP BW.
At least on the surface.
In reality, it turns out that Crystal is fully capable of leveraging the most important feature of a data warehouse: the fact that the data has been cleansed, extracted, de-normalized, and made ready for reporting. This process requires the lion's share of effort in any data warehouse environment and is the single most difficult part of the entire reporting process. This is the main reason why it is so difficult to report directly off of SAP ECC.
Now, it is true that Crystal cannot fully exploit the fact that the data has then been aggregated into a cube structure to allow for multi-dimensional analysis on the fly using an OLAP interface (like BEx or Pioneer). That said, we refer back to our discussion of the “80/20” principle in terms of the number of end users who can fully utilize this type of advanced functionality. In reality, the percentage of end users who can handle an OLAP interface effectively is typically far less.
Also, just because data has been brought over into SAP BW doesn't mean that it has to then be fully modeled and structured as an Infocube. Crystal is a great interface for reporting off of DSO's (Data Store Objects). This means that you don't necessarily have to take the time and effort to roll all of your SAP BW into an Infocube.
Here's another way to look at this: do you really want to spend all that time and money moving your transactional data into a data warehouse just so 10 to 20% of your end user base can access it using a “true” OLAP interface?
Crystal is a static reporting environment.
This has always been one of the biggest “push backs” against deploying Crystal Reports in any reporting environment. And for good reason. Very rarely are end users content with just a single “look” at their data. In fact, it can be said that the creation of a single Crystal Report is the best argument for the creation of a second.
One of the most significant enhancements of Crystal Reports 2008 is the ability now to create reports that can be dynamically formatted by the end user. This latest version of Crystal incorporates a new feature called “view time parameters” that allow the report developer to create multiple layouts in a single report. The end user can then simply select the desired layout from a list of available options.
And Crystal has for some time now provided support for basic navigational functions like drill-down and something called on-demand subreports, which are analogous to “jump” reports in BEx (or Report-to-Report Interface).
So while it would be a stretch to call Crystal Reports a fully end user customizable reporting interface, it does provide a significant degree of flexibility in terms of the final output and basic navigational functionality. In fact, it can be argued that the fact that the options available to end users are limited is actually a benefit for many end users. One of the problems with the more advanced OLAP interfaces is that many end users end up becoming confused and unable make any real sense of the data.
Crystal Reports have to be developed by IT or by a power user.
This is probably the single biggest argument against using a tool like Crystal Reports in an SAP BW environment. And once again, at least on the surface, it sounds perfectly reasonable.
One of the promises of the concept of the data warehouse is end user independence from IT. The ideal scenario would be where IT creates the warehouse, provides the end user community with a BI interface (or tool), and the end users simply dive in and find whatever they need themselves. This is whole concept of “self-service BI” and you would have to be crazy to say it's not a great idea. The problem is simple: it just doesn't work. At least not for everyone.
Certainly there are those end users in any organization who are fully capable of servicing themselves in finding what they need in a data warehouse. But, as we keep saying, they are certainly in the minority – normally a very small minority.
Not surprisingly, those who have proven themselves capable of “diving in” on their own in any organization end up becoming the “go to” people in the organization for getting information into the hands of the silent majority. In SAP BW environments these are the few who were actually able to figure out how to use BEx effectively and have set up shop as the de-facto “information broker” for their department or workgroup. In fact, if you were to chart the flow of information throughout any organization today, you would start with an image of your data warehouse, then draw a set of spokes radiating out to boxes representing that small subset of power users who “get it”, and then from there draw a lot spokes to a lot more boxes representing the rest of the end user base. In many ways, this is just the way it is and there may not be any way (or need) to change this natural “alignment” of resources.
I would argue that, instead of trying to bypass the power users in an organization and attempt to provide a direct path to the data for the rest of the end user base (which will, at best, produce limited real results and end up wasting a lot of time and money) it makes much more sense to make the existing process more efficient and manageable. In other words, the current arrangement is quite natural and, for the most part, unavoidable. So rather than replace it, we should fix it.

Improve the Process – Don't Try to Replace It
There are really only two significant problems with the current process of getting actionable information into the hands SAP knowledge workers: it is terribly inefficient and and almost completely unmanageable.
I would argue that, especially in the SAP end user community, the single biggest point of frustration is the sense that there is a whole lot more information in the system than what is currently available. And there reason for this is quite simple: it is almost universally true. Beyond something to help run your business, SAP R/3 (ECC) is essentially huge data vacuum that is capable of capturing a lot of data. Users innately understand this and naturally can therefore can feel quite frustrated when they see only a small subset (often very small) coming out the other end. I call this the problem of “information constipation”.
As almost every organization has discovered a long time ago, the bottleneck in the process of getting information into the hands of the at-large end user community is the power (or “super”) user in the middle. I prefer to call these very important people “Information Brokers”.
A side note: everyone has become tired of hearing the terms “power user” and “super user”. These terms have been so overused they have become almost meaningless. What exactly do we mean when we call someone a “power user” or “super user”? I would like to propose that we come up with a new way to identify those individuals in an organization who's primary role is to function as a conduit of information from the corporate data source(s) into the end user community. One idea would be to use the term “information broker”. Think about it: this is exactly what these people do. They “broker” a transaction between the information system(s) and the end users.
I like the term “information broker” for this reason: any transaction between two entities that cannot happen “naturally” requires a third party to step in to facilitate the transaction – a broker. The only reason brokers exist (or should exist) is they add value to the transaction. In this case, the value-add proposition is quite simple: they possess the necessary data access “know-how” to find the necessary data, convert it to usable information, and deliver it the end users. As we have already discussed, this is definitely not something the average end user can do by themselves.
For the most part, the problem with the current process is not that the current “information brokers” don't know what they are doing, but rather how they are doing it. Essentially the process is as follows:
Access and run a BW query using BEx.
Manipulate/transform/format the data in Excel.
Make the information available to the end users within their immediate circle.
There are lots of problems with this process that make it so inefficient. First of all, it is all very manual and labor intensive. Just the task of running existing reports every day/week/month/quarter can take a significant amount of time. And the job of distributing the information to the end users can also take a lot of time and effort (saving to a file share, emailing, creating PDF's, etc.). But both of these pale in comparison to the often overlooked task of making the data presentable.
Everyone in the BI industry these days has this idea (dream, wish, hope) that someday all information will be consumed on screen and we can finally get away from being in the printing business. And you know, someday we just might get there. But that someday is a long time away.
There are numerous reasons for this, but allow me to present two very basic reasons. First of all, it's hard to see a lot of detail on a computer screen. When a business decision maker is trying to make sense of any set of data (especially detail data) they need to be able to quickly and easily navigate across a potentially wide range of data points. They also need to be able to do this over lunch, or at home, or in the car. There is still no easier way to do this than printing off a hard copy and scanning through it manually. You can then use decidedly low-tech highlighters and/or sticky notes to identify critical data points and move on to your next meeting. Quick and easy.
We have the marketing profession to thank for the second reason: image. Or, put another way, appearances. No one wants to look unprofessional, and the more polished you look the more professional you seem. This has always been true for the way we dress and is equally true for how we present information. Like it or not, the way information is formatted and presented oftentimes means as much (or perhaps more) to the target audience than the information itself. Just ask any sales rep who has been “out gunned” in a sales presentation.
The bottom line: a big part of the process of gathering and delivering information to and end user community is making the information presentable. And this can take a lot of time. Especially if you're using the wrong tool. And Excel is definitely the wrong tool.
However, Crystal Reports is exactly the tool for the job. And when combined with Business Objects Enterprise (or SAP's Enterprise Portal) there is no more efficient way to deliver highly formatted, print-ready information to the end user community.
Very quickly we need to discuss the second problem with the current process: management. Or, more precisely, lack of management. A lot has been written about the data governance issues that arise when a company relies heavily on “spreadmarts” to run its business, so I won't add much here. I will say this: one of the primary benefits of deploying a BI interface/tool like Crystal Reports is that the information being delivered to the end user community is not modifiable by design.
Every organization should have a mechanism in place to insure that there is at least on set of consistent, standardized, tested, and verified operational reports that can be used as the “gold standard” by which all other information is compared. Without diving into the whole “one version of the truth” debate, I think everyone will agree that anything that can be done to move a business community toward clarity and consensus should be encouraged. Using “locked down” BI content is one way to do that. If you and I show up a meeting with our spreadmart-generated “analysis” and our numbers disagree, wouldn't it be a good idea if there was a separate, agreed-upon standard by which both our numbers could be evaluated for truthfulness? This is one very basic governance feature that Crystal Reports provides.
My conclusion is this: rather than trying get rid of the “spreadmart” or find a way around it, why not “legalize” and regulate it? Just as we as a society provide for ways to regulate a variety of industries and commercial markets, perhaps it makes sense to do the same with the process of gathering and distributing information within organizations. And if at the same time we can provide the keepers of the “spreadmarts” a efficient and effective way to convert data into easily consumable, presentation-ready information, isn't that all the better?
The last thing I will say is this: it is a mistake for any organization to position Crystal Reports simply as a way to create “pretty” reports. While it is certainly true that you can create virtually any layout you want with Crystal, the real value of Crystal is that it provides a way for the “information brokers” in your organization to gather, format, and deliver on-going information in the most effective and efficient way possible. In a word, when combined with Business Objects Enterprise, it is an “information engine” for your organization.

Posted by Mike Garrett at 10:15 PM | Comments (7530)