Software Product Management Metrics

1. Software Product Management Metrics Why measure? What should we measure? What should we do with the data?

2. Format & Goals • Full disclosures: • I wrote this deck during the lunch break • My thoughts are based on my education & experience…but they’re not the whole picture so please share yours too! • Format: • Introduction • Quick presentation • Share stories & workshop solutions together • Goals • Learn Something New • Continue the Conversation

3. About Me (Dan Heinig) • Over 20 years of professional experience in technology, marketing/advertising, and software product management • MBA & multiple degrees in marketing, advertising management, information systems management (and cognitive psychology in progress) • Also university instructor of business & technology courses

4. Why Measure? • To Learn Something New • To Test Hypothesis • To Understand our “Current State” • To Understand the Impact of Change • To Predict Something About the Future • Rules of Thumb: • If you can't measure it, it’s not worth doing. • The data has to be worth more than the cost to get it.

5. What Should Product Management Measure? • Demand Validation (customer & stakeholder wants & needs) • Execution (product development & delivery) • Utilization & Engagement • Value: “How Much Is This Product/Feature Worth?” • Customer “value” (should reflect delivery against customer wants/needs) • Business “value” (should reflect positive impact on goals & objectives)

6. Key Performance Metrics: Demand Validation • Customer “Payment” for products/features • Leads (“notify me”) • Subscriptions (payment/pre-payment) • Time (customer research, beta testing, etc…) • Stakeholder “Payment” for products/features • Willingness to allocate time (writing business cases, estimating value, defining measurement, participation in planning activities) • Willingness to allocate real dollars (budget) • Willingness to allocate resources (trade-offs on prioritization & execution)

7. Examples: Demand Validation • Customers/stakeholders have to be willing to “pay” to demonstrate demand • Bad Demand Validation: • A survey/micropoll that asks “Would you use this feature if we built it?” • A survey that asks “How important is personalization to you?” • Good Demand Validation: • Ask users to “sign up to be notified…” (turbotax) • Ask stakeholders to write a “one page business case”

8. Key Performance Metrics: Execution • Lead Time to Development (in days/weeks) • a.k.a “ready for grooming/dev” • Lead Time to Deploy (in days/weeks) • a.k.a. “ready to merge” • Lead Time to Release (in days/weeks) • a.k.a “business readiness” • Cost of Change/Delivery (in $$$’s)

9. Examples: Execution Metrics • It takes the Product Management team an average of 3 weeks to get a new feature into the backlog (“ready for dev”) • It takes the Software Development team an average of 1 sprint (2 weeks) to develop a new feature (ready to merge/deploy) • It takes the rest of the organization (sales, marketing, training) an average of 2 weeks to release a new feature (“business readiness”)

10. Key Performance Metrics: Utilization/Engagement • % of target customer adoption • Number of customers we build the product/feature for, divided by • Number of customers that actually try the product/feature • Are they using it as often as we expected them to? • Active daily users • Are they using it the way we thought they would? • Feature analytics

11. Examples: Utilization/Engagement Metrics • We currently have 4,000 active users of a product: • We built a new features and 1,000 of the users tried it in the first 30 days • We achieved 25% adoption • We released a new feature in Q1: • 90 days later, we have 1,300 active daily users (they use the features at least once per day) and on average, customers use the features 1.7 times/day. • We released a new feature in Q3: • 80% of customers that use the feature use it the way we expected • 20% of customers that use the feature appear to be using in a different way, or for a different reason

12. Key Performance Metrics: Value • Return on Investment (basic cost/benefit in $$$) • Should be able to measure against business goals & objectives • What impact did we think this would have? • What as the actual vs. estimated impact? • Explain the gap. • Should impact customer revenue generating activities • Acquisition • Retention • “Consumption” (purchases if that’s part of your revenue model)

13. Examples: Value Metrics • We built a new product for 2019. • It “cost” $1.2M • It generated $2.4M in new customer revenue • We achieved 100% return on our investment. • We have a business goal of “increasing new customer acquisitions by 25% in 2019” • The new product/feature resulted in a 12% increase in new customer acquisitions. • The new product/feature also resulted in a 2% increase in customer retention • The new product/features also reduced COA and increased CLV

14. What Should We Do With The Data? • Validation Data: • Informs our product/feature roadmaps • Execution Data: • Informs prioritization “How soon can we start to benefit from this product/feature?” (cost of delay) • Utilization Data: • Informs “go-to-market” release activities • Informs “sunsetting” of products/features • Value Data: • Informs “weight” of insights • Helps us predict/innovate • Ultimately helps us “keep doing what’s working” and “stop doing what’s not working”

15. Quick Q&A • Any questions so far?

16. Share Stores & Workshop Solutions • Spend a moment thinking about some of these scenarios you may have encountered: • “We want to build a product/feature but we don’t know how to validate whether it’s worth it or not” • We want to build a product/feature but don’t know how much it will cost or when it will be done • We built a product/feature and have no idea if it mattered/made a difference… • Others? • Please share your examples and we’ll see if we can have a discussion about how measurement can help us make better decisions.

17. Thank you! • Thank you for participating. • If you’re not already involved in local PDX Product Management Meetups, please give it a shot. • Let’s keep growing the PDX Product Community…it attracts investors, creates jobs, and improves the quality of life for all of us. • Feel free to connect with me on LinkedIn.