Product Discovery or Product Delivery: How do you Decide? [Mind the Product]

What’s the fundamental difference between product discovery and delivery or execution? The degree of uncertainty. The degree of uncertainty should determine whether you need to run product discovery or whether you can begin to deliver a solution to your customer.

Why is it Important to Know the Degree of Uncertainty Upfront?

Let’s walk through three hypothetical scenarios. In each scenario, you and your team are asked to set up a tracking system to better understand how customers sign up on your website and ultimately to improve customer acquisition. Let’s see what happens when we vary the degree of uncertainty and when we change our approach to dealing with uncertainty.

Scenario 1: Execution In An Environment Of Certainty

One of the founders at an early-stage startup asks you to analyse customer behavior to improve sign-up conversion. You set up a sign-up tracking system in a previous job so you have a lot of experience with this. You even have experience of analysing tracking data and coming up with valuable insights from it. Moreover, your team members have lots of experience of data-informed approaches to building solutions.

Based on this prior experience, you and your team are relatively certain about the steps involved and have a pretty good grasp of how long the task will take. The founders say they expect sign-up tracking to be implemented in two weeks. You and your team lay out a fairly detailed plan and estimate quite precisely that you need three weeks. You justify why you need the extra week with the founders. You and the founders commit to a certain deadline and, because there was low uncertainty throughout this project, you and your team can begin work immediately. The tracking system is implemented and you perform the sign-up funnel analyses by the deadline.

You’ve accomplished the goal and created value for the company. You built trust with the management team, not only because you reached the goal, but also in the way you reached it. You were realistic about the timeline and you created a plan that served as a reliable anchor for the management team to track the team’s progress in the right direction.

In general, predicting the future is difficult – but not in the case above. Why? Because there was a low degree of uncertainty; the team knew all the dependent variables and how they determined success. They could make correct predictions about what needed to be delivered to achieve the goal, and when it could be achieved. As a result, the team was able to start work without spending much time on discovering what needed to be done.

It’s easy to progress towards a goal in your daily life when you face a low degree of uncertainty, and it’s simple to make and keep your commitments.

Scenario 2: Execution in an Environment of Uncertainty

What could happen when you have little or no experience of the task you’ve been given and you’re unwilling to acknowledge what you don’t know? In this scenario, you and your team are unable to say with certainty what steps are involved and how long they will take.

When the founders ask you to complete the tracking system in two weeks, you feel pressured to commit to this deadline. You start by writing the high-level requirements and then hand them over to engineers who ultimately struggle to understand what is required of them. As the deadline approaches, you realise you won’t make it. Everyone stays late to try to salvage the project. Morale drops as people become frustrated and angry with what is now an impossible task. In the end, not only is the tracking system and sign-up conversion analysis incomplete, but the founders are upset and you have lost the trust of your team.

The team was confronted with a high degree of uncertainty, probably due to a lack of technical knowledge. But the fatal flaw was not that the team lacked experience, rather that the knowledge gap wasn’t acknowledged and commitments were made to hit a rushed deadline. Instead, the team should have run product discovery in advance, at least to check the technical feasibility, rather than starting directly with execution.

Setting up a tracking system is a simple scenario, even if an arbitrary deadline has been set. But what happens if the goal is much more complex, like improving a search engine so that customers will get more relevant and personalised results? In this case perhaps nobody really knows what relevance means from the customer’s point of view (there’s high uncertainty about whether you can create customer value). Or nobody has a clue how to build a search engine that does personalisation for millions of customers (high uncertainty about technical feasibility). Such an endeavour can take months, even years, and be expensive and frustrating if not done right.

What does it mean for your daily life? When you face many unknowns, don’t commit to hard deadlines. Instead, promise to find out what you don’t know so that you can deliver a solution that has a higher chance of success.

Scenario 3: Discovery in an Environment of Uncertainty

Let’s explore what could have happened had you chosen product discovery in the face of uncertainty before the execution phase.

You go back to your team and lead a discussion about the tracking and data analysis expected of you. Everyone on the team has a chance to think about what they know (from experience) and what they don’t know (uncertainties). Then you start to write down all the assumptions you think are important to reach the goal. Next, you evaluate the assumptions according to importance and degree of uncertainty. You will quickly find that technical feasibility is the biggest unknown, but important for success. Unfortunately, you know relatively little about how event tracking can be implemented for mobile platforms. So you decide to do a technical spike, which will take about a week. To prepare for the spike, the team has formulated all the questions that need to be clarified to be able to make a statement about achieving the goal. After four days, the team has gained such a good understanding of the technical feasibility that you estimate the goal can be reached in three or four weeks. You communicate this to the founders and the team is able to implement the tracking system as promised. It has taken four or five weeks, but everyone was very happy with how you handled it, both the leadership and your team.

What does it mean for your daily life? The tracking system is a simple example, but you can apply the same model of thinking to all your goals. If you are confronted with too much uncertainty, you should not start directly building a solution. Instead, take a step back to understand where the uncertainty is coming from and look into activities which give you evidence to pursue your goal. In product management we summarise these activities as product discovery. In the best case you reduce the uncertainty to a level that allows you to start building a solution that hopefully brings you closer to achieving your goal. Or you can’t find any evidence to justify continuing to work on your goal, so you can stop early enough without large investments. Both cases are a success and save the team valuable time and reduce frustration.

In summary then, the degree of uncertainty defines whether you can start directly with building a product (execution) that creates customer and business value, or whether you need to take a step back first to create more clarity through different types of activities (discovery). Being able to recognize the circumstances in which you find yourself – low or high uncertainty – makes a significant difference in terms of how successful and efficient you can be in achieving your goals.

Assessing Uncertainty and Taking the Right Actions

Let’s look at some practical strategies for assessing the degree of uncertainty and strategies for mitigating risk.

The Uncertainty Rubric

The goal of the Uncertainty Rubric is to help you assess the areas where you face the most uncertainty. In his book Inspired: How to Create Tech Products Customers Love, Marty Cagan writes about four different types of risk:

1. Value Risk – Will people buy/use it?
2. Usability Risk – Can people figure out how to use it?
3. Feasibility Risk – Can we build what we need with the time, skills, and technology we have?
4. Business Viability Risk – Does the solution create business value?

I use these four questions as a mental checklist to help me gauge how much I know. For example: if I don’t know whether a proposed solution is valuable to a customer, then I need to do product discovery to better understand the problems that customers face and what their true desires lie. If I don’t know whether a proposed solution is usable from a customer point of view, this also creates uncertainty, which I should tackle upfront through a usability test, before I start building a solution.

In practice, thinking about these four types of risk has led to more detailed questions that help me to quantify the degree of uncertainty my team and I face when building new digital products. These detailed questions form the rubric below. Just keep in mind that these questions are context dependent and they may vary. Use them to kick-start your thinking. Feel free to adapt them to your needs and go through them with input from members of your team.

Try to answer each question and score your degree of certainty on a scale of 1 to 10, with 1 meaning “I’m 100% certain about it” and 10 meaning “I’m just guessing”. You may never be 100% certain about anything, but you will have collected a significant amount of qualitative and/or quantitative data to support your claim. I also recommend that you don’t just to score the questions, but you also write down a detailed description – describe the customer or customer segment, the exact problem you want to solve. This will help you to realise how much you really know.

 

	Degree of Uncertainty
How certain are you that …	1 to 10 (least to most)
you know who the user/customer is?
you understand the user’s problem/desire?
you understand when the problem/desire is happening (context)?
the problem/desire is real (and not just made up by you)?
you know the number of users who face the problem/feel the desire?
you understand the most important customer benefit?
you know what the solution (user experience) should look like?
the customer can understand and use the solution?
the solution will solve the problem/fulfill the desire?
you can build the solution in a reasonable time or within a budget?
the proposed problem/solution-fit will create business value?

 

What do you do with your scored rubric?

If you and your team have marked every answer a 3 or below, it confirms that you have a high degree of certainty about your path, and you can start building. You will probably still discover new information but you can just adapt to it – this is why we work with Agile methodologies like scrum or Kanban.

Answers marked with 4 or above indicate areas where you should invest and do product discovery to reduce your uncertainty. At this stage, it’s better to figure out the right thing to do rather than commit to an outcome.

Once you have identified areas which require more understanding you can use an effective technique called Assumption Mapping.

Assumption Mapping

Assumption mapping is a technique to identify the most important thing you need to learn to be able to build a great product. How does it work in combination with the rubric?

By scoring and writing down what you know for each question you have started to identify assumptions and facts. Every question from the rubric that you scored as 4 or above can be treated as an assumption.

If you have identified only one assumption then you can skip the assumption mapping step and directly conduct an experiment (more below).

But if you have identified that you face more than just one assumption, you must prioritise the most goal-critical assumption. Normally this is the assumption that comes first in the product-creation process: Customer -> Solution -> Money -> Scale. To express the importance of an assumption, we add another dimension to the rubric – degree of impact.

Let’s walk through an example. For simplicity, let’s assume you have identified two areas where you have low certainty about doing the right thing – the most important customer benefit and the usability of your proposed solution. You are more sure of the customer benefit than the usability. But when it comes to the impact of your goal you wisely have decided that being sure about the customer benefit is more important than usability, at least at this stage of the product creation process. And this is right, because defining the impact score is time-dependent. It doesn’t matter if you solve the usability problem if you don’t create enough customer value in the first place. Let’s look at the scores:

 

	Degree of Uncertainty	Degree of Impact
How certain are you that … How much impact would it make if …	1 to 10 (least to most)	1 to 10 (least to most)
you understand the most important customer benefit?	5	9
the customer can understand and use the solution?	7	4

Next, you plot your assumptions on a grid where the x-axis describes the impact on your goal and the y-axis describes the amount of uncertainty you face. Our example is shown below:

The grid reveals that you should focus on proving your assumption about the customer benefit before you focus on your usability assumption. In general, you should start where the uncertainty and the impact is high. I’ve created a template [Google Sheet], so you can easily start mapping assumptions yourself.

Now you have a prioritised list of assumptions that you can use as a starting point to design and conduct experiments.

Design and Conduct Experiments

The next step is to convert your assumptions into hypotheses. The experiments can be quite different, depending on the nature of your assumption and business context. There is no silver bullet but a lot of best practices, ranging from generative user research, jobs-to-be-done, and demand/smoke tests for exploring the problem space, through to using low-fidelity and high-fidelity prototypes, technical spikes, and usability tests for the solution space.

If we go back to our example, the goal is now to design a cost-effective experiment to de-risk your assumption. The experiment should increase your confidence by collecting data points and enable you clearly to formulate the most important customer benefit. Then you can focus on the usability question and how to create a new cost-effective experiment to de-risk your assumption around it.

I recommend you set yourself concrete breakpoints – for example, after two weeks, you want to have collected enough evidence to continue the effort. This is especially helpful in the early stages, when you could do a lot of generative research. Set yourself boundaries to know how much time and effort you want to invest into a specific idea.

You can and should repeat all three steps as often as necessary, especially when you have new information. But as soon as you have created enough clarity on all your important factors you should start delivering the solution. In the end, you will just know if your customers will use your product when you release it. This is the real moment of truth…

In Conclusion

Innovation means uncertainty, but managing it properly makes a big difference. Being able to recognize the level of uncertainty you’re dealing with makes you more successful and efficient in achieving your goals. If we now reduce the whole story to one product principle, then it would be this one:

If the uncertainty you face is too high, start with product discovery and not with product delivery.

This approach is not an exact science, but it’s about recognizing uncertainty and addressing it. I hope you find this structured approach helpful and that it provides you with some guidance at the moment you need to decide whether to start building your product or to take a step back to find out what is right for your customers and business.

This is my first ever article. I’d be very happy to see your comments below or connect with you on LinkedIn. Feel free to share tools, ideas, and stories from products where you had to deal with a lot of uncertainty.

The post Product Discovery or Product Delivery: How do you Decide? appeared first on Mind the Product.

Source: Mind the Product https://www.mindtheproduct.com/2019/07/product-discovery-or-product-delivery-how-do-you-decide/