« Some books for the Christmas list to get you thinking | Main | Surnames and nicknames, Mr. Trump, an unusual phonebook, and data quality. »
January 4, 2007
The Shock of the New
What a start to the New Year! Few things could have made me smile more than to see the Intelligent Enterprise Reader's Choice Awards. Microsoft over all had a great showing, but for me the highlight was to see Microsoft taking the "Best ETL Software" gong. Let me tell you why it was especially sweet ...
About 6 years ago, I moved from startups to work at Microsoft, in the SQL Server BI group. The change was fascinating for many reasons - many good and some ... well, let's just say they were still fascinating. One thing was enjoyable and difficult in equal measure: I now had to deal with the somewhat fixed expectations of thousands of vocal users about what our product should do and how we should do it. The installed user base - especially one as large as Microsoft's - can overwhelm you with requests and suggestions. In fact it would be easy to be merely reactive and spend entire development cycles polishing scratches and filling dents as customers point them out.
However, at some point, if you want to move the software, the business, and the customers along significantly, you have to take the plunge and make some radical changes. But you know that doing so will cause some pain to the existing, loyal and even enthusiastic users.
In SQL Server Integration Services we faced this problem in bucketloads. The previous product, DTS, was lightweight but smart, and hugely popular. However, it was also very limited in its capacity to tackle the increasing demands placed on it by existing and potential customers. A complete, ground-up, no-line-of-code-left-standing rewrite was in order. And, as it turned out, the market needs forced architectural changes that made ugrading from the previous version almost impossible.
Naturally, we tackled the resulting issues on a technical level to some extent - but more importantly we had to tell a compelling story to users that the changes and their pain was worth it.
In that light, I can look to the award as an endorsement of the decisions we made, and the astonishing commitment of the team that drove the product along. Especially so, as the award comes from users who have to live and work with the software on a daily basis and therefore represent both the base we had to move, and the basis for our next versions. It's a great start to the New Year for the team, and for me personally a good time to look back and reflect on the long road, and all the various detours and diversions that we passed on the way.
Posted by Donald Farmer at January 4, 2007 4:30 PM
