A Harvard Business Review article by Tom Davenport caught my eye with this wonderful title as a neat distillation of our philosophy.
I often say to people that if a report is not used as the basis of a decision, then it actually has a negative value taking up valuable time reading it when that time could be more profitably spent on something else!
The article discusses how data is often captured, processed and stored, but then hidden away, “never seeing the light of day”, and this is something we see often in the world of web analytics. There are so many out-of-the-box reports that it is easy to become overwhelmed by the data, giving up and not using any of them. The key here is to sift through them, determine what is going to be important on a regular basis and hide the others. At the beginning of most implementation projects we typically hide 80-90% of the reports so that the only ones that are left are those that are going to contribute on a regular basis to the organisation’s decision making.
That is not to say that the other reports should never be looked at. For one client who had to produce monthly web traffic reports we looked at the trend of Visits over both days of the week and time of day. Interestingly traffic at weekends was 80% of that on a week day and, even more surprisingly, traffic at midnight was still 80% of the daytime peak.
This turned out to be entirely logical as their website was aimed at lorry drivers, who would be out working during the day and catching up on administration at home. However, it hadn’t been thought about before and all website maintenance happened at 6:00 pm as they thought no-one would be using it then. So, it was an easy and indisputable decision to minimise disruption by moving maintenance to the early hours of the morning instead (sorry IT guys!).
We put these graphs in their monthly report so that the entire organisation gained a better insight to the website usage as a one-off feature. However, after that, we put both these reports on a shelf and didn’t bother with them again for a while, only checking them occasionally to ensure that there was no change.
For me, the best bit of the HBR article came at the end, which recommends a simple framework for communicating each analysis:
- My understanding of the business problem
- How I will measure the business impact
- What data is available
- The initial solution hypothesis
- The solution
- The business impact of the solution
which encapsulates the process we follow with our clients, ensuring that we solve real problems with a measurable success.
Read the full article at http://blogs.hbr.org/cs/2013/06/data_is_worthless_if_you_dont.html