Common Mistakes In Defining Metrics

This list is far from exhaustive, but these are the ones I see the most often:

1. Metrics before Strategy 🛒 🐴

   Your metrics are a reflection of your strategy. They help answer, is the strategy working? Metrics with out strategy is like looking at a bunch of random numbers. You need to define the strategy before you define your metrics. What are the key hypotheses of the strategy? What metrics would indicate those hypotheses are true?

2. Definition Is More Important Than A Dashboard 📊

   Most people focus on "building a dashboard." But what is 10X more important is choosing which metrics are important and defining ******those metrics well. Defining is more complicated than people think. For example, there are many ways to define a retention metric depending on your product. Your dashboard is a method to communicate your metrics, which is important, but useless if you choose and define poorly.

3. Outputs vs Inputs ℹ️

   Most metrics like a retention metric or revenue metric are output metrics. These are metrics you should should monitor. Giving output metrics to teams as goals can be dangerous. They need to know how to break them down into input metrics to make them actionable. When this doesn't happen it leads to teams thrashing between things.

4. Usage First not Revenue First 💸

   This is the most common version of outputs vs inputs. Usage creates revenue, revenue does not create usage. As a result the most important metrics in terms of creating growth are not your revenue metrics, they are your usage metrics.

5. Mixing Up Retention and Engagement 😕

   I see a lot of teams think retention and engagement are the same things. They are not. Retention is binary. It answers the question, was this person active within my defined time period? Yes or no. Engagement is depth. It answers the question, how active were they within the defined timed period? 0→N. Engagement is one of three major inputs into driving retention.

6. Customers vs Users 👩‍💻

   A customer and a user is not the same thing in most business models today. A customer is defined as the person/group that is paying you. A user is a person using the product. In subscription products, often times there are multiple users associated with a single customer. Or people are users before they are a customer. You need to separate the definition and language between these two things for teams to clearly act on them.

Related Reads: Why Most Analytics Efforts Fail