SharePoint Moneyball … …a Practical Framework for Winning the SharePoint Metrics Game

1. SharePoint Moneyball……a Practical Framework for Winning the SharePoint Metrics Game July 24, 2012

2. Experience Main Event Find Me About Me • Knowledge Management • Information Architecture • Portals • Collaboration Solutions • Governance, User Adoption, Metrics • Led national Portals, Collaboration, and Content Management practice for Dell • Director of Knowledge Management at American Management Systems • President, Susan Hanley LLC Expertise • www.susanhanley.com • @susanhanley • sue@susanhanley.com • 301-442-0127

3. Why Measure? – The Four “F” Words Feedback Good measurement is good management Funding and metrics should be used to optimize and improve the process Follow-on Focus

4. Before During After Make the business case Develop benchmarks Develop lessons learned • Provide a target • Make tradeoffs • Tune the implementation process Measurement throughout the life-cycle

5. Measurement Process 4. What are the METRICS and how should we PRESENT them? 1. What are the BUSINESS OBJECTIVES? Modify the measures 2. How should the SOLUTION be DESIGNED to meet the objectives? 5. How can we COLLECT the metrics? Modify the process or tool 3. Who are the metrics STAKEHOLDERS? Aid decision making 6. What do the metrics TELL US about how we need to CHANGE?

6. 1. What are the BUSINESS OBJECTIVES? Without a critical business initiative … …“Career limiting move”

7. It’s easy to go for the “motherhood” objectives … More innovative products and services More effective marketing Better access to knowledge Lower cost of doing business – reduction in travel and other operational costs Higher revenues Improved employee, customer, and partner satisfaction

8. It’s better to Get SMART! Specific (concrete and well-defined) • Measurable (quantifiable, comparable) • Achievable (feasible, actionable) • Realistic (consider resources) • Time-bound (deadline driven)

9. SMART objective for a proposal library Reduce the average amount of time it takes to produce complex proposals by 10% in the next year Reduce the average amount of time it takes to produce complex proposals by 10% in the next year Reduce the average amount of time it takes to produce complex proposals by 10% in the next year Reduce the average amount of time it takes to produce complex proposals by 10% in the next year Reduce the average amount of time it takes to produce complex proposals by 10% in the next year • Specific • Measurable • Achievable • Realistic • Time-bound

10. 2. How should the solution be DESIGNED to meet these objectives? Site Architecture Technical Infrastructure Features Customization Security Governance Roles and Responsibilities Training and Communications

11. 3. Who are the metrics STAKEHOLDERS? • They’re at all levels - especially in the middle • They care about different things

12. For each key stakeholder, ask … • What counts? • What keeps you up at night? • What do you already use? • What do I need to tell you? Focus on the outcomes, then work backwards to figure out how you will measure that outcome

13. 4. What are the METRICS and how should we PRESENT them? • Identify the type • Quantitative • Qualitative Consider the life-cycle Establish a baseline Gain commitment about targets Decide the best way to communicate

14. Good metrics come in multiple types … plan on both • Quantitative • Performance between points • Spot trends • Qualitative • Provide context • Used when numbers aren’t easy (storytelling) • Used at early project stages (future scenarios) • Richer (“serious anecdotes”)

15. Return on Investment Platform investments: • Traditional ROI (costs and benefits) • Ongoing or trade-offs • ROI-lite LINK

16. ROI is only PART of the story Good metrics are: Related to outcome Relevant to stakeholders Collected at low cost Balanced

17. Metrics Example: Best Practice Library • Business Goal: • Save time, improve consistency by not re-inventing the wheel Approach Options: • OK: Number of “Hits” • Better: Ratings (with instructions) • Best: Survey to seek out specific re-use cases

18. Metrics Example: Prioritizing Projects (ROI-lite) Business Goal: Approach: Allocate limited SharePoint Resources Baseline -> Target -> Measure x xx • T = time on task (in minutes) • E = number of employees performing that task • N = number of times per year a typical employee performs that task • S = average employee’s loaded salary per minute

19. Consider two types of quantitative metrics System Metrics Business Metrics

20. Example of a System Metric Top Queries (from Search) Indicator for “Trending” Content Promotable MORE

21. Sample Business Metrics • Hours per week to execute a process • Number of Proposals/Contracts per year • Number of “[My Organization]-All” emails • Number of email attachments • Average application training costs • Cost savings to retire a legacy application

22. Qualitative Metrics – the stories that drive it home Keep it real In the storyteller’s words Serious Anecdotes

23. Serious Anecdote | Consulting I joined the organization on March 16 without previous experience. After one week of training, I joined a project team. After one day of training on the project, I was assigned a task to learn a particular technology that was new to everyone on the team. I was given a bunch of books and told that I had three days to learn how to create a project using this technology. In my first week of training, I learned about the company’s intranet where people described their expertise. I sent an email to four people I found with a search for that technology asking for their help. One of them sent me a document containing exactly what I wanted. Benefit: Instead of three days, my task was completed in 4 hours.

24. Serious Anecdote | Pharma – The Need • A scientist with Thrombotic & Joint Diseases in Germany began a project to isolate and culture macrophages and needed some help. • Meanwhile, two scientists in the US had deep experience in protocols for this area.

25. Serious Anecdote | Pharma– The Result • The German scientist consulted the expertise directory to find that expertise existed within the company and contacted the two US scientists he found in his search. • Both scientists quickly responded with assistance. One helped him with culturing protocols and the other helped him with information on magnetic cell sorting. Benefit: The German scientist was able to leverage existing internal expertise and, in the process, reduce his research effort by four weeks.

26. Different Measures for Different Stages Enterprise Growth Phase Use mixture of metrics to show value across the organization. Pilot Project Phase Use definitive metrics to show real value for business objectives. Solution Maturity Startup Phase Use scenarios and simulations to explore projected results and effects. Pre-Planning Phase Use scenarios and simulations to explore projected results and effects. EXAMPLES Time

27. How do you spell success? Have a Baseline and Target

28. Presenting Metrics Balanced Scorecard Dashboard “Report Card”

29. Balanced Scorecard Dimensions Health Capabilities Business Value

30. Balanced Scorecard Example | Expertise Location Metric Pilot Outcome Target Business Value: Health: Capabilities & Culture:

31. 5. How can we collect the metrics? Try not to over-achieve – balance counting with “doing” Automate where possible Get creative when it comes to qualitative metrics Ask • Survey • Usability Testing • Active Listening Seek • Send out a “journalist” Track • Classify by type • Keep storyteller value metrics – what was the benefit to you?

32. Example Survey Questions Don’t Take It Away “User-Friendliness” Rating “Intuitiveness” Rating EXAMPLE If given the choice, would you KEEP it? How does this COMPARE? How EASY was it to …?

33. Tip: Collect Qualitative Metrics with Ratings Was it helpful? Were you able to get value? Trigger a survey at 4-5 Can we call you? Follow up conversation

34. 6. What do the metrics tell us about how we need to change? • Are we doing the right thing? • What areas are most successful? • What areas should we be promoting? • In which areas should we be investing? • Which initiatives should we discontinue?

35. Keep in mind Metrics alone won’t make your program successful A person who is accountable for making changes based on analysis A person whose job it is to monitor them It’s as important to have a plan for acting on metrics as it is to have a plan for collecting them!

36. Call to Action • Make sure metrics are part of someone’s job. • Develop a plan to capture quantitative and qualitative metrics. Identify baseline measures – and gain commitment on targets – before you start!! • Develop a library or list to capture and categorize qualitative metrics. • Develop an approach to produce and promote metrics.

37. Contact Information Susan Hanley President, Susan Hanley LLC www.susanhanley.com sue@susanhanley.com 301-469-0770 (o) 301-442-0127 (m) Blog: http://www.networkworld.com/community/sharepoint Twitter: @susanhanley

38. Extras

39. Download the ROI Model http://download.microsoft.com/documents/uk/enterprise/75_The_Total_Economic_ImpactTM_Of_Microsoft_SharePoint_Server.pdf BACK

40. White Paper For a white paper that explains the concepts in this presentation in more detail – with lots more examples, check out http://www.susanhanley.com.

41. Sample System Metrics (“out of the box”)

42. Additional Useful System Metrics (third-party) BACK

43. One sample metric for each stage (more in White Paper) BACK

44. Balanced Scorecard Framework Solution Health Capabilities Business Value BACK

45. Survey Example: Post-Pilot Usability BACK

46. Other Resources How to Measure Anything: Finding the Value of Intangibles in Business by Douglas Hubbard JakobNielen’s Alert Box - Current Issues in Web Usability: http://www.useit.com/alertbox/ Determining the Value of Social Business ROI: Myths, Facts, and Potentially High Returns: http://www.idc.com/getdoc.jsp?containerId=225497 Social Software for Business Performance: The missing link in social software: Measureable business process performance: http://www.deloitte.com/assets/Dcom-UnitedStates/Local%20Assets/Documents/TMT_us_tmt/us_tmt_socialsoftwareexecsummary_021411.pdf SharePoint Lifecycle Management Solution with Project Server 2010: http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=17058

47. Examples of Intranet Measures