How To Measure Success
Given the significant amount of resources usually invested in a data warehousing project, a very important question is how success can be measured. This is a question that many project managers do not think about, and for good reason: Many project managers are brought in to build the data warehousing system, and then turn it over to in-house staff for ongoing maintenance. The job of the project manager is to build the system, not to justify its existence.
Just because this is often not done does not mean this is not important. Just like a data warehousing system aims to measure the pulse of the company, the success of the data warehousing system itself needs to be measured. Without some type of measure on the return on investment (ROI), how does the company know whether it made the right choice? Whether it should continue with the data warehousing investment?
There are a number of papers out there that provide formula on how to calculate the return on a data warehousing investment. Some of the calculations become quite cumbersome, with a number of assumptions and even more variables. Although they are all valid methods, I believe the success of the data warehousing system can simply be measured by looking at one criterion:
How often the system is being used.
If the system is satisfying user needs, users will naturally use the system. If not, users will abandon the system, and a data warehousing system with no users is actually a detriment to the company (since resources that can be deployed elsewhere are needed to maintain the system). Therefore, it is very important to have a tracking mechanism to figure out how much are the users accessing the data warehouse. This should not be a problem if third-party reporting/OLAP tools are used, since they all contain this component. If the reporting tool is built from scratch, this feature needs to be included in the tool. Once the system goes into production, the data warehousing team needs to periodically check to make sure users are using the system. If usage starts to dip, find out why and address the reason as soon as possible. Is the data quality lacking? Are the reports not satisfying current needs? Is the response time slow? Whatever the reason, take steps to address it as soon as possible, so that the data warehousing system is serving its purpose successfully.