---

Whether you’re data-inspired, data-driven, or data-informed, there’s one truth that applies to all product roadmaps: they’re deeply connected to qualitative and quantitative data that, when used the right way, steer the product strategy in the right direction.

Quantitative data tells product teams the what of the product. Things like engagement stats, user flows and feature usage reflect what’s going on with the product—how people are behaving and interacting with it. Qualitative data—like user insight, market insight and team insight—then tell product teams the why of these quantitative data results. Qualitative methods like surveys, questionnaires, focus groups and observational testing, for example, help product teams understand why their users act the way they do.

Data is crucial to the product development process, and product managers are never short on sources of data. The hard part comes when it’s time to pick which data to focus on that will then inform the product strategy, which in turn informs the product roadmap.

Learn everything you need to know about product roadmaps (what they are, how to plan and build 'em, how to present 'em) in our 2021 guide to product roadmaps

Start by defining the right product metrics

The product roadmap is a tool that exists to communicate the product strategy and get everyone aligned on the company-wide efforts towards building, maintaining and growing a product. Without a solid strategy, a set of goals and the right metrics, it’s hard to know which data points to pay attention to.

When product leaders define what matters early on (at the business, market and product level), they can then narrow the overwhelming influx of data down to the most actionable, informative and valuable data points that will inform and steer all product decisions.

Product metrics, the measurements that tell product teams about the progress towards achieving the goals of the product, act as a funnel and a filter for narrowing down the focus of this data. Individual product metrics depend very heavily on the type of product, the industry, and where the product sits along the lifecycle. But all metrics should have this in common:

• They show accurate measures and health checks of the product strategy and business goals
• They provide the reasoning for why the initiatives the team works on will yield the highest impact
• They help teams identify strengths and weaknesses, and the progress made over time on both
• They show data that helps identify problems in a timely manner

And if you want to be completely sure that you’re not following any vanity metrics, follow Eric Ries’ three criteria for good product metrics:

1. Actionable. Your product metrics should have a clear cause and effect relationship.
2. Accessible. Everyone in the team should understand how a metric is measured, what the results mean, and why they matter towards the overall goals
3. Auditable. Product metrics should be unbiased. This means that anyone in the team can reach the same conclusion from looking at the source data.

If your product metrics fulfill all of these parameters, you can be confident that you’re looking at the right data. In this case, the right data is the information that helps you keep track of and make decisions about the product strategy and goals. And at a more granular level, it informs which strategic themes you’ll focus on when it’s time to plan the product roadmap.

Pro tip: use Dave McClure’s Pirate Metrics framework to define general product and business metrics, and Google’s HEART framework to define customer satisfaction metrics.

What’s the most important data you need to build a product roadmap?

In product development, your quantitative metrics measure the analytics data. This is the data that tells product managers what’s going on with the product, what users are doing exactly, what they’re clicking on, what they’re not clicking on, etc.

This data doesn’t show you the why (that’s what qualitative data is for) but it can certainly show you where the product needs to be improved using empirical, objective data. Here are some examples of the kind of data you might want to look at when formulating the themes that will make it to your product roadmap:

• Feature usage: This is the percentage of users that are engaging with your product. This data should tell you how often your users are engaging. How sticky/attractive/valuable are the features?
• Funnel stats: These are the steps a user undertakes to convert. Looking at funnel stats can tell you where drop-offs are happening. You can then formulate initiatives to address a way to optimize the funnel and fix whatever it is that’s keeping users from converting to customers.
• User flows: This data should show you what your users are clicking on and in what order. You can then formulate a hypothesis around why they’re not engaging in the order you’d like them to.

On the other hand, you have the qualitative data that tries to answer the question of ‘Why?’ —these questions are posed based on the quantitative data before you. Here are some examples of the answers qualitative data might provide when it’s time to formulate those themes and initiatives that will make it on the final roadmap:

• Funnel optimization: Why are users dropping off where they are along the funnel? Why are users not interacting with this feature? Are they confused/annoyed/frustrated with something?
• Usability: What are users’ pain points, current and future? Does the existing product fail to address any of these?
• Feature usage: What is seeing engagement and what is being ignored? Why is something being ignored?
• External competitors: Where is the product falling short compared to the external market? How can we get ahead? What are some features that you might be able to offer?
• Support feedback: What are the most common issues users are complaining about? What can you do to mitigate those?
• Cost reduction: What are the departments (CS, Sales, Operations) doing manually that can be automated to save time/money?

After doing some qualitative research, you can then take that data, and the metrics they inform and create themes on your roadmap. Themes are the overarching priorities you’ve collected from the quantitative and qualitative data. The initiatives the team will work on should be guided by those themes.

Run the initiatives by a prioritization framework

It’s important to quantify the impact of any given initiative whenever possible. Different products will measure impact differently, with some putting more emphasis on increased usage and revenue, while others will put more emphasis on better retention rates and revenue.

The metrics you focus on must directly correlate to this impact you hope to achieve with the product. On top of that, the initiatives that you do decide to plot along the product roadmap should also have clear impact measurements.

This is where product manager-approved prioritization frameworks can help. The best prioritization framework is one that allows you to measure the estimated impact for each initiative before it goes on the roadmap. Here at Roadmunk, we’re fans of the RICE method and value vs. effort—so much so, that we offer those two frameworks as templates in our feedback and idea management suite.

The RICE method originated at Intercom, where the product team needed a structured way to compare multiple product initiatives (updates, projects, features) in terms of their potential impact on any given goal.

RICE stands for Reach, Impact, Confidence and Effort:

• Reach: How many people will this impact? How many people can we see (from analytics data, customer requests) will be impacted by this initiative? Reach is measured in number of people/events per time period.
• Impact: How much will this impact each user? Will it increase the results of the metrics we care about (e.g. more users, longer sessions per user, less churn)? You can use a multiple-choice scale from 0-3 (0.25 being the lowest, and 3 the highest).
• Confidence: How confident are you in your estimates? Confidence is measured using a percentage, and anything below 50% should be reconsidered.
• Effort: How much time will this initiative take away from each individual team? You can measure this in “person-months”

The result of this formula lets you see the “total impact per time worked”, which then lets you make decisions on what you should prioritize for the roadmap.

To wrap things up

Using the right qualitative and quantitative data when it’s time to formulate roadmap themes is crucial. The idea isn’t that product leaders need to look at every single source of data, it’s how they can narrow down all those inputs to the ones that truly will inform the decisions that will move the product forward.

This is why nailing metrics down to a science is important. Without the right metrics, and without the right organized system for presenting these metrics, it can be too easy to get distracted by data that seems important (See: vanity metrics) but isn’t.

And when you form teams that are on the same page about the product vision, strategy and goals, you can feel confident that you’re listening to the right data.

Ready to start building your own product roadmap? Try our ready-to-use product roadmap template.