4. Metrics in Day-to-Day Product Management

Metrics play a central role in day-to-day product management, guiding decisions, tracking progress, and helping product managers (PMs) understand how their product is performing. In this document, we will explore how to effectively use metrics in your daily work as a PM, focusing on tracking KPIs, result metrics, and how to combine metrics with key initiatives.

Regularly Tracking KPIs

Every product has a set of key performance indicators (KPIs) that need to be monitored regularly, regardless of the current focus of the product team. These metrics provide a general overview of product performance and serve as early indicators of potential problems or opportunities.

The number of KPIs you track depends on how easily you can obtain the data. If you have a robust data infrastructure, you can track a wider set of metrics, making it easier to spot deviations or trends. However, if obtaining metrics is challenging, it may be wise to start by tracking only the most critical KPIs and gradually expand your set of metrics over time.

When selecting KPIs, it’s essential to begin by focusing on macro metrics—those at the top of your metrics tree. These higher-level metrics provide a broad view of product performance. You can then drill down into more detailed metrics to identify specific issues or insights. Tracking smaller, disconnected metrics without first monitoring overarching KPIs can lead to fragmented and unclear insights about your product’s health.

Result Metrics for Specific Objectives

In addition to your ongoing KPI tracking, result metrics are used to measure the success of specific objectives or initiatives. These metrics reflect the focus of your work in a given period and are often tied to the outcomes the product team is striving to achieve.

For teams using the OKR (Objectives and Key Results) methodology, result metrics are the key results—the measurable outcomes that indicate progress toward the objective. Even if you are not using OKRs, you will still have important metrics that measure the success of initiatives, representing the results you aim to achieve.

For example, in Nubank’s Customer Experience team, an objective might be to improve customer experience efficiently. The result metrics would include satisfaction scores like HSAT (human satisfaction) and MSAT (automated satisfaction), as well as efficiency metrics such as the deflection rate (the percentage of inquiries resolved by automated systems rather than human agents) and unit effort (the average effort required per interaction, measured in human hours). These result metrics align with the team’s priorities and objectives for that specific quarter or cycle.

Metrics for Initiatives

Each product initiative or project you undertake should have its own set of metrics. These metrics help measure the effectiveness of the initiative and guide decisions on whether to scale, adjust, or halt the initiative.

Let’s look at an example from Nubank, where the team aimed to improve the efficiency of customer service by changing how chat requests were distributed among agents. The previous system allowed agents to pull chat requests as needed, while the new system automatically pushed chats to agents, based on their chosen capacity to handle multiple chats at once.

In this case, the key metrics for the initiative included:

After running an experiment with a test group of customer service agents, the team found that the initiative improved efficiency without negatively impacting satisfaction or response time. As a result, the initiative was rolled out to the entire team, and new metrics were introduced for the rollout phase. These included unit effort, which could only be measured once the system was deployed across the entire customer service team, and the satisfaction of the operations leaders overseeing the various service areas (e.g., credit cards, banking, insurance).

Post-Rollout Monitoring

Post-rollout monitoring is a critical step in measuring the long-term success of an initiative. After the initial experiment phase, when a feature or change is implemented for all users, it’s important to continue tracking the relevant metrics to ensure the change has the desired effect at scale. In some cases, new metrics may be introduced to assess additional impacts or to address concerns that arose during the experiment phase.

For example, after the successful rollout of Nubank's chat distribution system, the team continued to monitor both efficiency and customer satisfaction, as well as the satisfaction levels of the operational leaders. If any of these metrics showed unexpected results, the team could make further adjustments or improvements in a subsequent version of the initiative.

Applying Metrics to Daily Work

To effectively use metrics in your day-to-day work as a PM, it’s essential to balance both ongoing monitoring of KPIs and focused result metrics for specific initiatives. Regular KPI tracking provides a general understanding of product performance, while initiative-based metrics help you gauge the success of targeted improvements and experiments.

By continuously measuring and refining these metrics, you can ensure that your product is meeting user needs, driving business outcomes, and evolving in the right direction.

Final Reflection: “You Can’t Improve What You Don’t Measure”

As Peter Drucker famously said, “You can’t improve what you don’t measure.” This statement highlights the importance of metrics in product management. By consistently measuring the right metrics, you gain the insights needed to make informed decisions, drive product improvements, and ensure that your product delivers value to both users and the business.

In summary, understanding how to use metrics effectively in your daily work is key to being a successful product manager. By integrating metrics into your everyday decision-making, you can ensure continuous improvement and alignment with your product’s strategic goals.