Organizations often swing between “we have too many metrics!” to “we are not tracking enough metrics!” - which can drive data professionals and their business partners a bit crazy.
These reflect deeper issues, and ironically can be true at the same time.
1. “Too many metrics” - this reflects a metrics dump culture where various functions collect a hodgepodge of metrics and dump an exhaustive list into a central pot for leadership to review.
The main issue here is lack of contextual editing and classification.
Some metrics are critical outputs, but their importance can change slowly over time. Others are input metrics that represent levers of control, which change depending on the allocation of resources.
Some metrics are useful cuts that have organizational resonance - like churn rate by cohort is useful to track even if not a directly controllable input lever. While other cuts are nice to know - like churn rate by recent activity. Since every single dimensional cut of a metric is a candidate to track and report, this is where bloat is likely to happen.
Pruning and classifying metrics into inputs and outputs, main and dimensional cuts, useful and nice-to-knows, based on what matters in the moment is essential to reducing the cognitive load and improving alignment across the business.
Metric trees can be a powerful tool to organize and align metrics and drivers across the organization.
2. “Not tracking enough” - sometimes, this can be driven by a real blind spot in the tracking. An example would be not having granular product usage data to inform features or to inform sales and marketing. Or, not collecting frequent qualitative surveys from your user base.
But many times, this just reflects cuts of metrics that aren’t looked at consistently. When a metric goes south, and you realize it’s driven by a specific segment that isn’t often looked at, there is a desire to promote that slice to the top for visibility.
If there were easier ways to fathom slices ongoing, then this would be less of an issue, as it directly contributes to bloat over time. Metric trees can streamline root causing metric movements over time.
If you are a data practitioner, a key take-away is that a crucial of your job is related to how metrics are classified, disseminated, understood and utilized by the business. And any confusion and mis-alignment has to be collaboratively addressed and resolved.
Otherwise, you run the risk that all the hard work building source data models and running various kinds of reports and analysis produces little returns.
Good luck!
