Not all metrics are created equal. In this post, I hypothesize that in aggregate, data and tech teams have over-indexed towards tracking, analyzing and impacting product engagement metrics vs core business operational and financial metrics, and that a meaningful course-correction is needed.
First, some explanations why product metrics are in the ascendant.
- the cultural influence of FAANG or “Big Tech” in general. In the biggest success stories in the last decade, product metrics really mattered. For social media giants like IG or LI, user engagement and network effects equals bigger audience and $. For consumer apps like Netflix or Spotify, user engagement translates into retention and monetization.
- product data hits the sweet spot of being granular, that is we can track every click and every scroll of every user from every browser or app version, while also being complete, and within the team’s control to capture with acceptable hygiene. In a world where important data is either unavailable or incomplete, digital product data (even with its own technical challenges) is an oasis.
- it’s completely reasonable to analyze the onboarding flow of new users or behaviors of highly engaged users to optimize aspects of the product. Tech teams are well positioned to ship features in order to drive these product metrics - especially when they are closely related to growth metrics.
Don’t get me wrong - I think product metrics and the ability to impact them is a key lever but there are many organizations where it may not be the first order problem, and can take away valuable time and effort from the harder-to-measure-and-analyze parts of the business.
Let’s look at the other side of the coin:
- Even for a pure e-commerce business where product metrics are key, the selection, pricing and availability of the items being sold may have a bigger impact on user acquisition or retention versus improving the browsing or checkout experience.
- There are businesses where what happens outside the digital product experience has disproportionate impact - where operational execution of the service offered, the customer experience, or the users intent, beliefs and behaviors “offline” - matter deeply. I suspect this is the case for several businesses in healthcare, education, logistics, financial services etc., which represent large swathes of the economy.
- We need to recognize this and make the effort to track, analyze and understand the entire business process, not just the digital product engagement - that is, study the operational execution metrics of a fulfillment workflow, or of project or task completions, analyze customer service guarantees, response times and quality, capture user attitudinal data via surveys or rigorously analyze user behaviors to infer which ones are both predictive of outcomes, and can be “controlled” via the user experience. This is where the magic of data lies - in identifying actionable levers for the business.
- Finally, product metrics, to the extent useful, correlate to user growth metrics but in this corrective cycle, managing costs and driving profitability is crucial. Measuring and building company-wide visibility into the costs involved in delivering the business is an under-rated activity. This may not be as straightforward as measuring user clicks, but with the right assumptions and putting the work in, cost-related data modeling can be transformational to the business (no pun intended).
In the next 2-3 years, as growth at all or unknown costs recedes, as finance teams take a more active role in shaping goals, every function will be asked to re-examine their contribution to driving profitability, and adjust their teams, their measurements, their analyses, and of course their actions.
This presents an opportunity for data teams to be excellent partners by building out the necessary data models, executing ongoing analysis, monitoring and alerting against goals, and in doing all this, push core business and financial metrics and associated drivers to the forefront of the organization.
