Metrics that Matter Advanced Report Training Session

1. Poll • Welcome to the Metrics that Matter Advanced Report training session! Have you previously attended an MTM Basic Report training session? • Yes • No, but I know the basics of running reports • No, I have no previous MTM experience © 2009, KnowledgeAdvisors

2. Metrics that Matter® MTM Advanced Report Training

3. Purpose • Your organization is usingMetrics that Matter (MTM) to: • Collect evaluations / tests for training • Improve learning effectiveness and impact • Communicate results • Learn to use MTM optimally • Advanced Reporting class • Basic Reporting is a pre-requisite © 2009, KnowledgeAdvisors

4. Poll • Which of the following best describes your role? • Instructor • Courseware Developer • Program Manager • Operations Manager • Analyst • Other © 2009, KnowledgeAdvisors

5. Expectations for Training • Interactive • Ask questions, communicate needs • Participate in hands-on exercises • Close email and other applications • Focused • 90 minutes + 30 minutes Q&A • Off-topic questions may need to wait • Evaluation at end of class © 2009, KnowledgeAdvisors

6. Agenda • Quick review • Reduce scrap learning • Measure overall performance • Analyze impact of strategicprograms • Additional Reports • Wrap-up © 2009, KnowledgeAdvisors

7. Poll • Which topic are you most interested in? • Reduce scrap learning • Measure overall performance • Analyze impact of strategic programs • None of the above © 2009, KnowledgeAdvisors

8. Logging into MTM www.metricsthatmatter.com/mtm • View announcements • Register for workshops and events • Register for MTM training sessions © 2009, KnowledgeAdvisors

9. Hands-On Activities Hide pane • Click Hand buttonto raise hand • Switch to web browser • Perform tasks • Switch back to GoToWebinar • Click Hand buttonto lower hand Hand button Submit questions © 2009, KnowledgeAdvisors

10. Hands-On: Log into MTM • Open your web browser • Go to www.metricsthatmatter.com/mtm • Bookmark this page • Login © 2009, KnowledgeAdvisors

11. Poll • Do you need a review of any of the following from the MTM Basic Report training session? • Auto-email class reports • Manually run class reports • Compare instructor or course performance • None of the above © 2009, KnowledgeAdvisors

12. MTM Report Categories © 2009, KnowledgeAdvisors

13. Poll • Where would you find reports to compare your overall performance for last quarter against your goals? • ROI Tools • Executive Tools • Aggregate Tools • Tactical Tools © 2009, KnowledgeAdvisors

14. Reduce Scrap Learning • Analyze courses for highest scrap • Schedule reports © 2009, KnowledgeAdvisors

15. Scrap Learning • 50% of every $ spent on training is wasted • Already know the content • Don’t learn the content • Content is not relevant to jobs • Lack of support • Ensure the right people are attending the right training at the right time © 2009, KnowledgeAdvisors

16. Poll • Where would you find reports to compare the amount of scrap learning for each of your courses? • ROI Tools • Executive Tools • Aggregate Tools • Tactical Tools © 2009, KnowledgeAdvisors

17. Compare Scrap for Courses • Focus on courses with highest scrap (red/yellow) • Analyze using Quick Question • Revise or retire courses, or change learners Variance © 2009, KnowledgeAdvisors

18. Hands-On: Variance by Course • Click Aggregate Tools > Variance Analysis • Select Courses as the Report Type • Set date parameters • Select Question Category: Job Impact filter • Click Generate Report © 2009, KnowledgeAdvisors

19. Schedule Custom Reports • Aim for 70% scheduled, 30% manual reports • Deliver info at time of need Date range options Delivery options © 2009, KnowledgeAdvisors

20. Schedule Custom Reports • Aim for 70% scheduled, 30% manual reports • Deliver info at time of need Email options Recipient options © 2009, KnowledgeAdvisors

21. Hands-On: Schedule Variance Report • Click Schedule button • Select Last Full Month from Dynamic Dates box • Select Monthly in Frequency box • Select 5 in Date box and click Add link • Click Add button • Check Email this report to me • Click Save © 2009, KnowledgeAdvisors

22. Measure Overall Performance • Trend performance over time • Compare performance vs. benchmark © 2009, KnowledgeAdvisors

23. Measuring Overall Performance • Provide context for the metrics • Trends, Benchmarks, Goals • Focus on communication • Within L&D and to customers • Establish accountability for improvement © 2009, KnowledgeAdvisors

24. Trend by Learning Methodology Trend Scale Average • View performance trend by learning methodology • View trend of evals collected • Analyze downward trends © 2009, KnowledgeAdvisors

25. Hands-On: Trend Scale Average • Click Aggregate Tools > Trend Analysis > Scale Average • Select Learning Methodologies in Report Type box • Select Performance and Activity • Select Monthly in Frequency box • Set date parameters • Click Generate Report © 2009, KnowledgeAdvisors

26. Group Challenge Situation • Performance of your instructor-led training has trended downward for 6 months • VP wants to see this go up next quarter What reports could help you understand how to improve? © 2009, KnowledgeAdvisors

27. Use Benchmarks • Compare performance against internal or external reference • Context for quantitative metrics • Insight into above and below average areas Report Card © 2009, KnowledgeAdvisors

28. Select a Benchmark • Identify the primary default benchmark • Use more targeted benchmarks selectively • Region • Industry • Audience • Course Type © 2009, KnowledgeAdvisors

29. Hands-On: Report Card • Go to My MTM > Set Preferences • Select Everything Except My Company benchmark • Click Update Preferences • Go to Executive Tools > Report Card • Set date parameters • Click Generate Report © 2009, KnowledgeAdvisors

30. Analyze Impact of Strategic Programs • Determine impact of key programs • Communicate value to customers © 2009, KnowledgeAdvisors

31. Measure Impact of Key Programs • Select strategic, costly, and visible programs • Collect Post Event, Follow Up, Manager evals • Estimate fully loaded costs and average salary % of work time requiring knowledge and skills * % overall improvement in this area * % improvement due to training * 65% Improvement= (Salary*Improvement) –Cost) ROI = Cost © 2009, KnowledgeAdvisors

32. Determine Impact of Key Program Human Capital ROI Card • Filter to program courses • Focus on key metrics • Time to job impact • Barriers to use • % improvement due to training • Increases in business results and ROI • Compare vs. benchmark © 2009, KnowledgeAdvisors

33. Hands-On: Human Capital ROI Card • Go to ROI Tools > Human Capital Methodology > Human Capital ROI Card • Set date parameters • Click Create New Filter • Name filter and click Add Filter • Click Courses, select course(s) • Click Add to Filter and click Done • Click Close Window and click Next • Click Generate Report © 2009, KnowledgeAdvisors

34. Communicate Value to Clients • Focus on needs of clients • Increase productivity of workforce • Achievement of goals • NOT highly rated instructors and courseware • Set goals with clients during design phase • Review metrics as program progresses • Use metrics as tool for discussion © 2009, KnowledgeAdvisors

35. Additional Reports • Crosstab • Curriculum Hierarchy • Performance to Goals © 2009, KnowledgeAdvisors

36. Wrap-Up • Summary • Q&A • Evaluation © 2009, KnowledgeAdvisors

37. Summary Objectives • Reduce scrap learning • Measure overall performance • Analyze impact of strategicprograms Tools • Variance • Schedule reports • Trend Scale Average • Report Card • Human Capital ROI Card © 2009, KnowledgeAdvisors

38. Q&A • Any questions? • Would you like to review anything? • Any urgent needs for other reports? ? © 2009, KnowledgeAdvisors

39. Evaluation © 2009, KnowledgeAdvisors