Main | How Not to Be Successful in Direct Marketing »
March 28, 2007
How to Select a Modeling Vendor
As an analytics vendor for the past fifteen years, including the last ten years as the head of two different modeling/analytics companies, I have some definite thoughts regarding best practices in selecting a provider.
1) Competency -- I believe that only about a third of companies that sell analytical services are consistently competent. By this I mean that these companies always (or almost always) produce analytic solutions that are competitive with what the best vendors would produce using the same data. This impression comes from numerous validations, assessments and benchmarking projects I have been involved with over the years. How to you gauge competency? Here are a few thoughts:
a). The best way is to have the vendor develop a test model, where some of the data is held out as a validation sample. I would expect that the results would be consistent between the development and validation samples.
b). The product should, in most cases, make business sense. In line with this, each attribute should be logical and the 'signs' should match -- meaning that an attribute that has a positive correlation with what you are trying to predict should not have negative points.
c). I would make sure that the vendor does NOT use a purely automated modeling system. With the current state of the technology, I firmly believe that an experienced modeler will produce a better solution. The 'art of modeling' has yet to be automated.....
2) People -- While many/most vendors have some experienced people on their staff, it is important to assess the experience and reputation of the people that will actually work on the project. I would recommend working with a smaller vendor that can provide an experienced team, rather than selecting a larger company that will assign a relatively inexperienced team.
3) Interest -- Based on my experience, you are always better off with a vendor that views your account as a 'big deal'. If you are not a big deal to your vendor, getting relevant attention is going to be a problem.
4) Reputation -- While word of mouth can sometimes be misleading, it is at least directionally correct. If a vendor has a spotty reputation, it is important to find out why.
Of course, all of the above assumes that there is a strong and synergistic relationship in place between you and the modeling vendor that you select.
Posted by BeyeBLOGS at March 28, 2007 2:45 PM
Comments
I would also recommend looking for vendors that have flexible solutions, those that can be easily adjusted to fit your company's needs, demands and business goals.
Posted by: SOgundi at July 11, 2007 5:26 PM
