Evaluating Product Teams: Mind What You Measure [Mind the Product]

It is well accepted that outcomes  – such as product adoption, customer satisfaction, user experience, and product performance  – rather than outputs and tactical tasks should be used in evaluating product managers. However, in practice, these outcomes are at least influenced by – and often determined by – external factors.

What are these external factors and how do we account for them when evaluating product managers? Furthermore, in order to ensure that product teams don’t become too risk averse to learn from their failures and iterate their way to success, a few cautions are warranted in evaluating product teams.

Watch What You Measure

I was in a coaching session with our president of technology at Salesforce when he quoted Pearson’s Law – “that which is measured, improves”. He cited an exaggerated example  – that if you use bugs per line of code to measure the performance of a developer, then they’re unlikely to write any code at all so that they can improve this metric. Similarly, blindly measuring product managers by outcomes and product success can backfire. In fact, it can make them extremely risk averse as these outcomes are highly dependent on factors beyond their sphere of influence. Ideally, we would like product managers to take risks, learn from failures, iterate to improvise, and stubbornly persist to succeed.

Provide a Secure Environment

Product managers are deemed effective when their product succeeds in the marketplace. This is irrespective of how success is defined – often a form of revenue, adoption, market share, user engagement, customer satisfaction, or any other metric. In the real world product managers never succeed or fail all by themselves. They work at the intersection of a complex ecosystem, interacting with developers, sales, customers, channel partners, operations, and many other stakeholders. The effectiveness of the ecosystem, industry trends, and competitive forces has a disproportionate impact on the effectiveness of product managers.

The last thing that product teams need is a leader who tells them: “You either succeed or you’re out.” In my experience, product managers perform at their best when they’re led by a trusted leader with strong ability to guide and listen, and who can encourage them in the face of failures. Such leadership often increases the credibility and effectiveness of product managers in the ecosystem.

Implementation in Practice

How do we evaluate the performance of product managers amidst these uncertainties? Here are some thoughts, based on my experience in managing product management teams.

Define the Role and Success Criteria

Product managers play different roles in different contexts. Often a product manager’s role depends on variables such as product lifecycle, type of product, and organization structure. Additionally, the role of the product management teams should be strongly aligned with organizational goals. It’s important for leaders of product management teams to define the expected contribution of product managers and to communicate this clearly.

A framework I have successfully used to define a product management role by product lifecycle is depicted by the picture below.

Irrespective of the structure you use, it’s important to acknowledge that the role of product managers varies across several dimensions. This means that deliberate attention needs to be paid to this thought exercise beforehand and communicated to product management teams, as well as applied consistently in evaluating product managers.

Establish Baselines for Success Criteria

Once the role of product management and the desired outcomes are established, it’s important to establish appropriate baseline benchmarks for measuring the success of product management teams. For example: if you task a product manager with improving the availability of a system, help them to define the historical availability and expected results. This context is important in order to set realistic expectations on success criteria.

When you establish the baseline, account for the market dynamics such as competitive solutions. If you’re an online retailer with a top-notch product manager and are attacked by Amazon, your product manager is unlikely to be successful unless you account for this competitive situation in measuring their performance.

Account for Strengths and Weaknesses of the Ecosystem

Product managers seldom work in isolation. Even though they play the role of individual contributors, product managers manage people cross-functionally. Their degree of success is determined by the capabilities of their ecosystem. As an example, if you want to measure the success of your product manager by velocity of iterations, don’t ignore the development team’s velocity.

Obtain Buy-in

As in any leadership situation, you need to inspire commitment by getting buy-in. Although this seems like an obvious step, it requires significant rigor to clearly articulate and negotiate metrics leading up to buy-in. But, investing this time in people always pays off.

Closing Thoughts

In both my roles of managing product managers and evaluated by managers, I have seen the natural tendency of managers to fall back to evaluation by output rather than outcomes. It requires strong organizational commitment reinforced by executive leadership to stick to measuring product management effectiveness by outcomes. I realize that many thoughts expressed in this blog are also applicable to evaluating other functions in an organization. Thus, organizational commitment, culture, and processes play a disproportionate role in how product teams are evaluated. Organizations that are mindful of this tend to have committed, high-performance employees in general.

 

The post Evaluating Product Teams: Mind What You Measure appeared first on Mind the Product.

Source: Mind the Product http://ift.tt/2hzAgx0