Prepared by : Andrew Somiah Systems Training Manager Date 18th July 2011 Version 2.0

1. Practical strategies for Learning Evaluation – Metrics That Matter in Identity & Passport Service (IPS) Prepared by : Andrew Somiah Systems Training Manager Date 18th July 2011 Version 2.0 Not protectively marked

2. Overview of IPS Identity and Passport Service (IPS) is a Public Sector Organisation • Approximately 3800 employees spread across the UK • Core Business Products: • Passports & Identity Management • Birth, Marriage, Death & Civil Partnership Certificates

3. Overview of Learning & Development Function in IPS • Approximately 589.6 days of training delivered for 2009-2010 financial year • Blended learning solutions managed via the Learning Management System (LMS). Centralised Learning & Development function with a wide geographical remit. • E-learning delivered by Kallidus Learning Management System (LMS) • Classroom trainer-led events

4. 5 Levels of Evaluation Model used in IPS (Jack Philips Model)

5. Learning Measurement & MTM Strategy • MTM is integrated with the IPS LMS • Estimate, Isolate & Adjust technique used by MTM • All training courses including some e-learning are evaluated • Internal targets and an Organisational Development Score Card has been set to drive strategy • Three key indicators that are monitored monthly: • Trainer Performance • ROI: Cost Benefit Ratio • Learning Impact & Value (business results) • IPS targets are incorporated into KPOs for L&D staff who are performance managed with these

6. Learning Measurement & MTM Strategy 5 Optimized IPS-Strategic 4 Effectiveness Comprehensive 3 Tactical 2 1 Implementing Time • Collecting evaluation data and measure business results • LMS dashboards presentation to business • Presenting monthly analyses to business

7. 1st Quarter of 2010 Figures

8. Impact on Business • To improve impact and value to business: information is displayed on the LMS dashboard

9. Practical Approach to Data Analysis Evaluation Data Analysis 9

10. Review & Analysis Cycle 10

11. Reports Used for Practical Analysis • Human Capital ROI Card • Business Results ROI Card • Instructor Performance Summary Report • Training Improvement Summary Report by Trainers • Project Initiation Documents/Action plan by Training Managers • Course Effectiveness Summary Report (Director & board Level) • Talent Development Executive Report (Director & board Level) 11

12. Challenges & Tips • There was initial resistance when the system was implemented as it was a cultural change • IPS has put in place an Organisational Development balance score card accepted at board level • The internal benchmark targets have been incorporated into the organisations Key Performance Indicators • Align evaluation strategy to business objectives • Note: Its is a Process not a project expect results over time. • Simple continuous marketing of the benefits to business units helps improve evaluation return rates

13. Summary • Aspirations- Evaluation to become second nature and part of the culture • MTM is providing an invaluable resource for: • demonstrating the worth of training to organisation • provides performance information for making crucial strategic decisions • Could have a positive impact on the bottom line of any organisation

14. Questions

15. Contact and Further Information Stuart Halliday EMEA Sales Director KnowledgeAdvisors P: +44 (0) 203 371 1165 M: +44 (0) 771 7507 945 shalliday@knowledgeadvisors.com Andrew Somiah Systems Training Manager Identity and Passport Service HQ Learning & Development 4th Floor Mailpoint B NW Peel Building 2 Marsham Street London SW1P 4DF T: 0207 0356 798 E: andrew.somiah@ips.gsi.gov.uk