Why we have this Process
Roadmapping is the single most important Product Management process and ensures we are building the right thing. In this process, stakeholders are aligned on what new product features to build based on identified user needs and supporting data. I highly recommend reading the extensive introduction to roadmapping in Appendix A: Introduction to Roadmapping.
Context and Scope
Roadmapping is an ongoing process and part of the Discovery Process.
Note that Roadmapping is only user needs require a bigger solution that requires several weeks of software development (typically 1.5 - 3 months). For small improvements (<3 days of development) there is a lighter process with its own time bucket (see Appendix A: Introduction to Roadmapping)
When should the process be performed?
- Data collection through Learning and experimentation to validate/support roadmap themes are ongoing processes (See: User Interviewing, Surveying, User Testing, Analysis of Analytics and Support tickets, NPS surveys, fake page tests).
- The Product Council comes together monthly to discuss new themes and approve changes (to the top 5 theme's). Product Managers. In case of any new entrants in the top 5, the PM should prepare a short presentation that he also discusses with the team in the weeks before the meeting.
Who performs the process
The PM is responsible for the process.
The Product Council participates in the process as the decision maker.
The Head of Product monitors the correct execution of the process and acts as a coach and document reviewer.
- 2-3 weeks before the Product Council meeting analyze data from Learning and experimentation from last month (i.e. outcomes of User Interviewing, Surveying, User Testing, Analysis of Analytics and Support tickets, NPS surveys, fake page tests)
- Put relevant data in the column "Supporting Data" for the relevant existing or new Roadmap theme's (see Roadmap template for columns). Sidenote: For learnings that just need small product improvements (max 3 days development time) I recommend adding 'small improvement' type user stories into the Jira Backlog or add the findings to existing stories. Update the scoring for these theme's when relevant.
- Based on new data and scoring update the priority order of the theme's (including putting new theme's somewhere in the list).
- In case theme's entered the top 5 for the first time:
- Create a short presentation (max 5 slides) about the new theme (slides: High-level user need and supporting data, Scoring and business case, high-level solution direction, similar competitor features, high-level estimate)
- Prepare a slide about why you scored this relatively high, and how you see this will contribute to a higher LTV?
- Will it increase revenues per order? (basket size)?
Will it increase our profit margin? (reduce cost per order / reduce CAC)?
Will it increase orders per year (purchase cycle)?
Will it increase the customer growth?
Will it reduce churn/increase customer lifespan?
- Take a look at how competitors are addressing this user need, or how non-competitors solved a similar problem (you can also ask the UX designer to help with this). (screenshots don't count as extra slides)
- After you created the presentation so far, plan a meeting with the team, plus Head of Product and CTO, to present the new theme('s) and get their input and feedback. Also, let the team estimate the theme in terms of 'project points' by comparing the theme to the themes from the past. Ideally, you want to identify at least 2 theme's that were a little bit smaller than the current theme, and 2 theme's that were a bit bigger, and then more or less average the lower and higher theme to come to a rough project points estimate.
- Based on how long it took the team to complete theme's of a certain number of project points in the past, you can now calculate a rough end date for the Design and Delivery phase of the project and put this on the last slide.
- Present the new themes to the product council using the presentation you made. (I recommend presenting the highest ranked 3 or so new ones, not to overload the Product Council with information). Ask the Product Council for feedback.
- In case you don't have any next theme approved by the Product Council already:
- Ask the product council to vote blindly on the priority order (i.e. give 3 points to their number 1 theme, 2 points to 2, 1 point to 3, on a piece of paper). Summarize the votes and discuss with the Product council until you at least achieve agreement on the next theme.
- In case you still have approved theme's on the roadmap or don't need a new one for the next month:
- Per theme ask the product council members, per member, if they feel they have enough data to prioritize this theme, or if they want further data collection during the next month. Majority vote goes.
- In case the product council has enough data to make their decision for one or more new theme's launch a voting round (same as above. You can also do this for i.e. 5 theme's). Since there are already other approved theme's on your roadmap, they also need to score points to those too.
- Now you have a newly approved roadmap theme that you can start off in the Design process as soon as the designers in the team are done with the previous theme.
One time process setup
- Explain the process and the role of the Product Council to the CEO
- Discuss Product Council composition with the CEO
- Explain the purpose and role of the Product Council to the council in a meeting.
- Plan monthly Product Council meetings
What to do if a stakeholder declares an emergency and wants the Roadmap to be changed ASAP
Plan an emergency Product Council meeting, and do your best effort do the normal preparation steps. Usually, this will end in the Product Council talking sense into the stakeholder to finish the current theme and follow the normal process for the new theme (i.e. first investigate if there is really a user need). You can help shoot down bad ideas by doing a quick poll (i.e. using hotjar popup, or emailing a few users), ask support about it or a quick experiment (i.e. place a fake button for this functionality and see if anyone clicks it).
How to handle a CEO/Sales/Investor/Important Stakeholder who comes in with a Theme that was not yet on the Roadmap?
Put it on the Roadmap, and do some quick testing on it i.e. by doing a quick poll (i.e. using hotjar popup, or emailing a few users), ask support about it or do a quick experiment (i.e. place a fake button for this functionality and see if anyone clicks it). Then discuss these outcomes in the next Product Council. The other Product Council members will help you put this idea (or further data collection on it) in the fridge, assuming the user need wasn't really there. For how to handle sales coming in with a 'must have' feature to sell a to a big customer, see the next question.
How to handle sales coming in with a 'must have' feature to sell a to a big customer, see the next question.
Same as the previous question, but find additional arguments against (bad) ideas in this page: Bad idea: Specials / Customer specific Product (features)
Why 1.5-3 month long theme's? Isn't that too waterfall? Why not smaller steps to reduce risk and develop more incrementially?
A certain amount of capacity is allocated to small improvements (i.e. 25%), however, to really move a product forward effectively and efficiently you need to make bigger steps too. Several successful companies (i.e. intercom) stick to 6 weeks because to keep the planning effort low. This is why themes are important next too small improvements:
- Working on a theme basis allows the team to think through and design the whole user journey (i.e. UX design, architecture) and also do user tests on the whole journey.
- It's more efficient to work theme based
- As the theme is an MVP the whole thing is needed to validate if the solution addresses the user need.
- Discussing user stories and screens with a Product Council would take too much time and doesn't align with their strategic, not operational, role in the process.
- After working out the strategy and high-level priorities, the team should take care of finding the best tactics to get there.
- It's too time-consuming to prioritize 200 features vs 20 themes.
- Themes provide a coherent goal and thereby give direction and help with prioritization. Even if not every feature within the theme is done the goal can still be reached.
Isn't an experiment backlog better than a roadmap, focussed on learning every sprint and improving core KPI's?
Generally speaking, the Roadmap is an experiment backlog. By implementing a (MVP) solution you learn if it works. On the other hand, you constantly do collect data (i.e. experiments, surveys, interviews) to collect data and see which Theme should go first. I often see some companies read a growth hacking book and then try to be more 'experiment & data-driven' but overshoot by doing too many experiments and concentrate on too small improvements and trying to be too sure of things before moving forward. On one hand, you shouldn't start a roadmap theme for which there is insufficient supporting data, on the other hand, you shouldn't endlessly keep experimenting and collecting data until you are 'sure' a theme will work out. For the 'Fruit Basket subscription' example customer interviews followed by a larger online survey gives you the impression that circa 20% of your customers will order it. You could further proof this by offering it via mail to some of your existing customers and see how many subscribe. However, at some point you are spending too much time on collecting data on one theme without gaining much more certainty. To quote a famous Amazon principle: If you're 70% sure of something, go for it!
I suggested allocating about 60% of the team time to roadmap themes and about 25% to small improvements. You could use the same ratio to allocate experimentation/learning time. In case experiments need some minor development work I recommend to take that from the 25% small improvements time or to allocate some experiment building time.
Learning should be prioritized too just like everything else. Your roadmap themes are your highest return big bets and their priority order (and your certainty per item) will guide you on where more learning might help. One startup I worked for had as 'most important' learning experiment if a different background picture for the homepage would increase conversion. Another one was experimenting with button colors. These are a clear example of bad learning prioritization. Note that roadmap theme's should be MVP's and thus cannot be broken into smaller parts and still address the user need (i.e. one-time fruit basket orders, or single fruit piece recurring orders are different user needs, not smaller themes that would help you validate your bigger theme). Note that as part of the Design phase you create Clickable prototypes that you can use for user testing. If nobody uses or understands the Design a smart PM does not proceed with development. I suggest prioritizing based on what your riskiest assumptions are (also know as RAT Riskiest Assumption Testing)
A final point is 'smoke test'/'concierge model' experiments, whereby you only design and develop the first step in a process , and from there on the handle it manually. I.e. in the fruit subscription example one could create an order form in wufoo.com (and accept only payment by invoice). The question often is if this test is really needed. I.e. you could also call 10 customers to check if they would order, and the design of the page will later be tested with a user test. In many situations such a test could also require quite some 'backoffice'/concierge staff that you actually don't have.
I don't really own the roadmap, what now?
You should own it, meaning the process and preparation for the product council, although the whole council is in the end the decision maker on the next theme. Read: https://medium.com/@kathmatsumoto/having-trouble-with-owning-your-product-6fdd9b885c3c
Isn't a more gradual check/approve roadmap process better? (i.e. Stage gate process)
This is more of 'company preference' nature. The Product Council could give more gradual, step by step approval of a Roadmap theme. As a first agenda point the council might select a few theme's they want more data to be collected on. As a second point they might approve the start of design of one or two Roadmap theme's (based on the collected data). And a third agenda point they might approve the start of development of a Roadmap theme (based on the Design and User testing that has been done on the design). This more gradual approach prevents theme's from proceeding when the user test results clearly say users don't like the solution. It also helps to prioritize the data gathering (learning/validation/experimentation) efforts by PM, as the Product Council will ask for the the most important data they need for their decision making. It could however be agrued that a good Head of Product / CPO / PM could does not really need the direction from the council for this. An example from "How to create Products that customers will love":
For each product effort (except minor updates or fixes) there are five major milestones ("gates") for product council review and decision making:
- Milestone 1: Review proposed product strategies and product roadmaps, and initiate opportunity assessments for specific product releases. That is, select the product opportunities to be investigated.
- Milestone 2: Review opportunity assessments and recommendations, and issue go/no-go decisions to begin discovering a solution (=design prototype, do estimate).
- Milestone 3: Review product prototypes, user testing results and detailed cost estimates, and issue go/no-go decision to begin engineering.
- Milestone 4: Review final product, QA status, launch plans, and community impact assessments, and issue go/no-go decision to launch.
- Post launch assessment: It can be useful for the product council to review the business performance of the products that have launched. The product council may request a presentation on the business results of the product launch, typically 3-6 months post-launch. This sort of accountability will help the council better understand which investments and decisions they made were good ones, and why.
- Intercom - Why 6 week theme's
- Also note that this page have several sub pages that dig deeper into subtopics (see left side menu).
Appendix A: Introduction to Roadmapping
Roadmap Stakeholder Alignment - The Impossible Dream?
One of the biggest challenges of Product Management is to get all stakeholders to agree on a roadmap that is best for the company. What often happens in B2B product companies is that Sales forces feature onto the roadmap to close the sale with a big new customer, or that the CEO has a pet project for which actually there is very little customer demand. Even Support can unintentionally derail a product if they force too much time allocation for bugs and minor improvements. Also, various stakeholders (i.e. CEO, investors) can blow up the roadmap by demanding that 'version 2.0' of a certain way too many features before launch. Finally, disagreements between stakeholders can also lead to feature bloat, by trying to address too many personas or too many user needs.
Many PM's see stakeholders setting the roadmap as inevitable and accept this or give every stakeholder a set of points to spend on features which is just as bad. Doing this is signing the death warrant of Product as only products with the highest business and customer value will survive.
Taking charge of the Roadmap process is one of the core duties of a PM, and it's totally learnable and doable and does not require stellar negotiation or presentation skills and certainly does not require a company off-site (those are great for team building and brainstorming, not for roadmapping).
Foundations of Roadmapping Success
First of all, you need to accept that you cannot force a Roadmap on your stakeholders, even if it's the best roadmap in the world. Stakeholders like a CEO or CPO have end responsibility for the Product and to need to buy into your ideas. However, with the right process, you will be able to get your stakeholders to accept 90% of your roadmap proposals within 2 hour Product Council meeting. The Product Council makes all Roadmap decisions and typically consists of the Product Manager, Head of Product and most of the C-suite and typically meets once a month.
The Product Council decides on the priority order of the Roadmap Themes. A roadmap theme based on a single high-level user need i.e. "B2B Fruit Basket subscription MVP", and typically one to three months of development work. The Product Council does not go into detailed requirements and does not provide design feedback, but instead trusts the underlying processes and experts for that.
Roadmaps should be managed using the same Agile principles used to manage a scrum backlog, not as waterfall project. The only difference is that on the Roadmap the user stories are bigger. Most companies have learned and embraced scrum, yet when it comes to the Roadmap they fall back to Waterfall thinking (see table below).
|Waterfall thinking Roadmap||Agile Roadmap|
|Roadmap work items are feature-based (i.e. payment provider x integration).||Roadmap work items are user need-based (One time payment by Creditcard or Paypal).|
|Estimated by 1 or 2 people, not always developers.||Estimated by team including designers and testers, comparing it to past projects.|
|If development is delayed the deadline gets postponed.||If development is delayed the scope is reduced, but the deadline is fixed.|
|Often big projects / large scope||Small as possible theme's (MVP's)|
|Planning doesn't get updated until the first deadline is missed||Constant re-prioritization of the roadmap|
|One big release planned, i.e. 1.0 release on 1st of April.||Intermediate Release every (2 week) sprint.|
|Planning 6 months or more ahead, with details even far ahead||Don't plan too far ahead. Theme's who are at the top of the backlog get more detail.|
|A team working on several projects in parallel.||Just one theme per team at a time (until design is done and tested, then the designers can already start on the next theme)|
Companies that need to transition from a more waterfall approach could start with just some of these practices and instead of a 12 month roadmap do a 6-month roadmap.
Every roadmap theme is an MVP (Minimal Viable Product), meaning that only that only the bare minimum features are developed that are needed to address the user need and provide the proof if the user will actually buy or use the solution as designed. For the fruit basket example, this means that customer can only change his subscriptions by calling customer support and that the customer cannot (yet) change the contents of the actual basket. Every theme will have a short financial business case (i.e. 10% of users will order it) which can be tested against.
However not all software development is managed through the Roadmap, as this would blow up the process with bugs and minor improvements which are of less strategic importance. Only themes are managed through the Roadmap. For these strategic themes, a time bucket is allocated, i.e. 60% of the development time. For bug fixing there is a separate bucket (i.e. 15%) and for Minor Improvements (a.k.a. Business Blockers) there is another bucket (i.e. 25%). These percentages are set by the Product Council for a longer term. The PM has full authority to manage the priorities of bugs/stories in those buckets without consulting the Product Council, however, he/she might have to report on those retrospectively. Prioritizing Bugs and Minor improvements is outside the scope of this article, but typically improvements are ranked by financial business value and bugs by the number of people impacted and the severity (i.e. is there a workaround or not).
The decision the Product Council has to make is the order of roadmap theme's or at least what Roadmap theme goes next. The hard way of doing this is asking everybody for their opinion and arguments, who will mostly be subjective. This will lead to endless discussions that are not grounded in data on what the customer wants and therefore lead to bad compromises. The easier way of doing is that A) the PM convinces the group of the importance of data-driven decision making. B) The PM convinces the group based on what data points themes are prioritized C) The PM collects that data D) The PM presents a top 3-5 of next roadmap theme's supported by he collected E) The Product Council orders the list.
Prioritization should be driven for 95% by a scoring formula that the Product council agrees on. Every company will create its own formula and on the page Product Roadmap Template, you can find an example. In general I recommend a formula that accounts for the strength of the user need(0-100%), the additional profit this feature would bring per customer (0-100%), the number of customers that will use this feature (reach)(0-100%) , the alignment with the product vision or business goals(0-100%) and divide all the above by a rough effort estimation. Since all elements in the formula are based on a score (often (0-100%) you can literally calculate a score per roadmap theme. When you first place a new theme on the roadmap you do a quick estimate of all factors. And if the theme has a chance of making it into the top 5 or so, you start improving those estimates by collecting more data. Just like you don't make detailed plans for a far away future theme, you don't collect too much data on them either. For the current version of my formula, see the product Roadmap template.
Example formula: (((Impact - Satisfaction)+ ProfitPerCustomer + BusinessGoalAlignment) * Reach * Confidence) / Effort
I have seen too many PM's don't even start collecting supporting data for any roadmap theme because they think the effort is too big. This is often based on a limiting belief that data needs to be statistically significant and therefore a large deal of data points is needed. However, do you really believe that decisions made based on some data are worse than just relying on gut feeling? What I like to do is play Sherlock Holmes and put all 'clues' that I collected into my roadmap themes. This can for instance by things like "Place 3 in top 10 customer pains reported by Support", "In January survey this 75% agreed or strongly agreed with the statement ...", "23% of non-customers leave the funnel at stage y". The ultimate data points are often user behavior and survey answers, but often some imperfect data can be perfectly right for prioritizing the roadmap. Collecting data is ongoing work as part of the Discovery process (i.e. quarterly surveys, bi-weekly user tests, weekly analysis of support tickets, NPS score, analytics, etc).
Feeling you don't really own the Roadmap? You should, meaning the process and preparation for the product council, although the whole council is in the end the decision maker on the next theme. Read: https://medium.com/@kathmatsumoto/having-trouble-with-owning-your-product-6fdd9b885c3c
Improvement ideas for this Process:
- Short presentation template
- Separate Learning & experimentation page.
- Separate article with further LTV elements explanation & how to score on this.
- Better seperate article on several prioritization techniques