Telling your product story with Metrics

1. Telling your product story with Metrics Govind Ramanathan

2. We communicate as a Product Manager • Stakeholders • Current Status • Roadblocks • Timeline • Partner Issues • Executive Team • Product Vision • Roadmap • Revenue • Resource Utilization • Customers • New Features • Feature Improvements • Performance • Business Value • - • Marketing • Vision • Customer Impact • Innovations • Target Markets • Competitive Advantage • Product Team • Overall Vision • Feature Priority • Overall experience • User story • Roadmap

3. We run as fast as possible Market Requirement Document Power Point Slides Vision Feature Specifications Spreadsheets User Stories Scrum Mission Timeline Kanban Backlog Go to Market Strategy Roadmaps Waterfall

4. Think Words to Numbers With Documents With Metrics Vision Increase conversions to 30% Increase quality leads to 55% Reduce cost per acquisition to 20% Status Conversions are at 15% Quality of leads increased to 40% Cost per acquisition is 45% • Vision • Build a platform that creates quality leads to our customers at low cost • Status • The Feature to screen quality leads is 50% done • The APIs needed to measure conversion is defined

5. Metrics • It is not… • Replacement to a Vision statement • Metrics are derived from your vision statement. It helps to enforce the vision. • Replacement to Scrum or the engineering progress • Replacement to Roadmaps • It is… • Language to communicate progress accurately • Goal for the teams to focus on • Improve it and feel good about the work • Method to filter the vision into numbers • Means to force us to think about actual benefit/value we are creating • To measure and communicate success pre and post product launch • Tool to reduce chatter on details and focus on the bigger picture

6. Types of Metrics • Feature Metrics • For large feature measure the progress • Example • Product: Inventory management system. Own fulfilling orders and replenishing inventory. • Metric: • Average time to fulfill order is less than or equal to 2 days • Orders fulfilled greater than 2 days is less than 10% • Buffer inventory <= 30%

7. Types of Metrics • Product Metrics • Directly measures product improvements • Measures the product in isolation • Example • Product: Sell Textbooks Online • Metric: • Decrease online ordering time to less than 2 minutes • Reduce shipping average shipping cost to less than $3

8. Types of Metrics • Company Metrics • Measures the entire suite • Reports on the improvement of the eco-system • Forces compromises on individual products for the overall Product Suite • Example • Company: WeSellLotOfStuffOnline.com • Metric: • Increase Revenue by 30% to $30M per quarter • Decrease Operating Cost by 10% to $20M per quarter • Increase customer base by 50% to 5M customers

9. How to define Metrics? • Identify the entity of value • Identify the audience for the metric • Avoid metrics that may be influenced by externalities • Metrics should be impacted only by what it measures • Look back at your vision • Would improving this metric satisfy your vision • If more than one metrics satisfies the vision then pick the most important • You can have more than one but make sure they are mutually exclusive • Look at the top-level metrics to determine if lower-level metrics are targeted reasonably

10. Things to watch out… • Always monitor over optimizing for a metric • Evaluate if counter metrics needs to be created • Evaluate if new metrics are needed • Define and evangelize how you obtain the data for your metrics • Including pre and post processing • Data is never perfect…but we can get close • Resist the temptation to add revenue as a metric • This is only useful at the company level. At the product level you lose focus • Never count product bugs or support calls as a way to monitor product improvements

11. Continue measuring the product pre and post launch

12. Thank You “In God we trust; all others must bring data.” W. Edwards Deming