Six Lessons for Monitoring and Evaluation into Delivery

1. Six Lessons for Turning Monitoring and Evaluation into Delivery Lima July 2012 Ray Shostak, CBE

2. Remember who makes a difference to the quality of services………..

3. Setting and Meeting Expectations Engage Empower Co-Design Co-Produce Results

4. Government Ambition to Delivery Government Priorities CITIZENS

5. Outcomes Driven Performance ACCOUNTABILITY with consequences Government Strategy Goals/Ambitions For parliamentary period BUDGETING FOR RESULTS (multiple years) MONITORING, CAPACITY BUILDING and INTERVENTION To Parliament • CULTURE • Outcomes focus • Data rich • Can do • Real time monitoring • Reform • SKILLS • Problem solving • Cost benefit analysis • Engagement • Evaluation • ACTION • When off track To President/Prime Minister Non Government Government Programme budgets against results Unit costing Monitoring Efficiency Joint procurement ……. By Institution By Individual Aligned as Government Plan Delivery Partners Programmes, Objectives, Milestones, Deliverables Institutions and Individuals By Institution By Individual Outcomes for and with citizens

6. Lesson 1: Start from Citizens

9. Detailed Journey Maps

11. Lesson 2: Make sure Priorities are Priorities

12. USA Performance.gov Taxonomy Agency Theme Legend Submitted by agencies – Phase I Agency-DefinedGroupings Budget Functions Planned for Phase II Priority Goal Sub-Goal (tag-only) Other Information Performance Data Action Plan Contributing Programs Measures Milestones Other Agency Measures Other Agency Programs Explanatory Narratives Explanatory Narratives Indicators Financial Resources Explanatory Narratives USASpending Data.gov or agency site Budgetary Resources Granular Data MAX

13. 4 6 4 8 8 6 2 3 5 3 5 2 2 8 1 3 15 10 7 4 8 3 4 4 3 2 3 8 6 9 • A better quality of life • Improve the health and wellbeing of children and young people • Improve children and young people’s safety • Tackle poverty and promote greater independence and wellbeing in later life • Promote better health and well-being for all • Ensure better care for all • Deliver a successful Olympic Games and Paralympic Games with a sustainable legacy and get more children and young people taking part in high quality PE and sport • Stronger communities • Build more cohesive, empowered and active communities • Make communities safer • Deliver a more effective, transparent and responsive Criminal Justice System for victims and the public • Reduce the harm caused by alcohol and drugs • Reduce the risk to the UK and its interests overseas from international terrorism • Fairness and Opportunity for All • Halve the number of children in poverty by 2010-11, on the way to eradicating child poverty by 2020 • Raise the educational achievement of all children and young people • Narrow the gap in educational achievement between children from low income and disadvantaged backgrounds and their peers • Increase the number of children and young people on the path to success • Address the disadvantage that individuals experience because of their gender, race, disability, age, sexual orientation, religion or belief • Increase the proportion of socially excluded adults in settled accommodation and employment, education or training • A more secure, fair and • environmentally sustainable • world • Lead the global effort to avoid dangerous climate change • Secure a healthy natural environment for today and the future • Reduce poverty in poorer countries through quicker progress towards the Millennium Development Goals • Reduce the impact of conflict through enhanced UK and international efforts • Help people and businesses come through the downturn sooner and stronger, supporting long-term • economic growth and prosperity • Deliver commitments by the National Economic Council to help people and businesses come through the downturn sooner and stronger, • including packages on repossessions, apprenticeships, business access to finance and help for the unemployed. • Raise the productivity of the UK economy • Improve the skills of the population, on the way to ensuring a world-class skills base by 2020 • Ensure controlled, fair migration that protects the public and contributes to economic growth • Promote world class science and innovation in the UK • Deliver reliable and efficient transport networks that support economic growth • Deliver the conditions for business success in the UK • Improve the economic performance of all English regions and reduce the gap in economic growth rates between regions • Maximise employment opportunity for all • Improve long term housing supply and affordability Numbers of departments contributing to the PSAs

14. Lesson 3: Get Policy Right First Time – Components of Open Policy Making

15. How……. “Crowdsourcing” questions to shape the definition of the problem, not just consulting on solutions. Using ‘Policy Labs’ which draw in expertise Involving delivery experts early Creating cross-departmental teams Using web based tools to widen the policy debate Making more source data avialable

16. Lesson 4: Understand the system

17. Moving from a ‘delivery chain’… Ministry Regional Government Local Government Service provider Citizen

18. … to a ‘delivery system’ …………’Delivery Systems’ EDUCATION SPORT HEALTH Youth Sport Trust School Sport Partnerships Government Office Health Authority Local Authority Primary Care Trust School Hospital School Citizen Family

19. Better Health For All Delivery System for PSA 18 – Better Health For All – DH strand Legislation (incs EU legislation); health prevention & promotion advertising Public Health Observatories provides support Patients, People who need and use social care, Citizens, Socially Excluded / Disadvantaged Groups, Carers…. Workforce advertising, influencing recruitment & retention; NSFs Competition, pricing, vfm Funding, LDPs Primary Care Trusts (inc PBC) DH Performance Management Strategic Health Authorities Influence Performance Management of delivery agreements Acute / MH Trusts and FTs Equitable outcomes LAAs, LSPs JSNA Choice & Voice influencing provision and commissioning e.g. LINks GPs & Practices Specialist Units / Trusts joint commissioning National Regional Agencies Delivery Agencies Delivery mechanisms Local Community = national levers and covers HCC, CSCI, NICE, Audit Commission and wider agencies e.g. CQC etc including ‘clinicians’ e.g. GMC, NMC, HPC, etc Key : = Department / Agency Footnote 1, 07/8 programme to co-locate public health and social care presence in the regions. Footnote 2, consider in light of wider regulatory review. = Working jointly and in partnership = joint local working relationship

20. Better Health For All Delivery System for PSA 18 – Better Health For All – DH/CLG strand Legislation (incs EU legislation); health prevention & promotion advertising Public Health Observatories provides support Patients, People who need and use social care, Citizens, Socially Excluded / Disadvantaged Groups, Carers…. Workforce advertising, influencing recruitment & retention; NSFs Competition, pricing, vfm Funding, LDPs Primary Care Trusts (inc PBC) DH Performance Management Strategic Health Authorities Influence Performance Management of delivery agreements Acute / MH Trusts and FTs DCLG Equitable outcomes LAAs, LSPs JSNA Choice & Voice influencing provision and commissioning e.g. LINks GPs & Practices Government Offices (incs PH1) Local Authorities inc OSCs Specialist Units / Trusts Performance Management, funding Commissioning incs joint commissioning environment, housing, roads, schools, benefits, etc LA Teams and Specialist LA Units Social Care providers Independent providers e.g. ISTCs Voluntary providers Regulators & Inspectorates2 Inspect, Reviews & Reports National Regional Agencies Delivery Agencies Delivery mechanisms Local Community = national levers and covers HCC, CSCI, NICE, Audit Commission and wider agencies e.g. CQC etc including ‘clinicians’ e.g. GMC, NMC, HPC, etc Key : = Department / Agency Footnote 1, 07/8 programme to co-locate public health and social care presence in the regions. Footnote 2, consider in light of wider regulatory review. = Working jointly and in partnership = joint local working relationship

21. Better Health For All Delivery System for PSA 18 – Better Health For All – Complete system Legislation (incs EU legislation); health prevention & promotion advertising Public Health Observatories provides support Patients, People who need and use social care, Citizens, Socially Excluded / Disadvantaged Groups, Carers, etc Workforce advertising, influencing recruitment & retention; NSFs Competition, pricing, vfm Funding, LDPs Primary Care Trusts (inc PBC) DH Performance Management Strategic Health Authorities Influence Performance Management of delivery agreements Acute / MH Trusts and FTs DCLG Equitable outcomes LAAs, LSPs JSNA DCSF Choice & Voice influencing provision and commissioning e.g. LINks GPs & Practices DWP Government Offices (incs PH1) Local Authorities inc OSCs Specialist Units / Trusts Performance Management, funding Cross-government working Commissioning incs joint commissioning HO environment, housing, roads, schools, benefits, etc LA Teams and Specialist LA Units DCMS influence influence Defra OGD influence RDAs, Regional Assemblies, etc Social Care providers DIUS regeneration Independent providers e.g. ISTCs Business community DfT OGD influence Voluntary providers Regulators & Inspectorates2 Inspect, Reviews & Reports National Regional Agencies Delivery Agencies Delivery mechanisms Local Community = national levers and covers HCC, CSCI, NICE, Audit Commission and wider agencies e.g. CQC etc including ‘clinicians’ e.g. GMC, NMC, HPC, etc Key : = Department / Agency Footnote 1, 07/8 programme to co-locate public health and social care presence in the regions. Footnote 2, consider in light of wider regulatory review. = Working jointly and in partnership = joint local working relationship

22. Lesson 5: Let Data Drive Delivery and Cost

23. One Version of the Truth

24. Monitor the Impact Know the Impact of Policy on Practice

25. Strategic Reviews Efficiency Key Review Concepts Appropriateness Effectiveness Efficiency Integration Performance Assessment Strategic Policy Alignment Policy Alignment Effectiveness

26. The Program Activity Architecture • Policy requirements for Strategic Outcomes incl: • Performance Measures Strategic Outcome* Accountability levels to Parliament (Estimates & Public Accounts) *require TB approval, incl. major & minor changes • Policy requirements for each program activity element include: • Program title & description • Expected results • Performance measures • Planned & actual spending • Target & actual results • Governance Program Activities* Departmental PAA: reflect the inventory of all the programs of a department depicted in their logical relationship to each other and to the SO(s) to which they contribute **require TBS approval Sub-Activity Level** Sub-Sub Activity Level** Lowest Level programs

27. Monitor the Impact Lesson 6: Support to deliver improved, more efficient and effective services for citizens

28. A Delivery Unit Performance Policy (ie. Performance Framework, Excellence, cross departmental working) Reporting Prime Minister Secretary of State Senior Officials Unblocking Delivery Obstacles (ie. progress reviews, problem solving, follow-up work with and brokering between departments) Capacity Building & Cross Govt Learning on Delivery (ie. knowledge mgmt, deptl delivery units, networks & training) Performance Monitoring (ie. Data tracking and reporting)

29. Some final observations Outcomes Matter The skills and dynamics of delivery can be analysed and understood The basics of good delivery are key Consistency and alignment enhances outcomes Systematic intervention can improve systems and outcomes Collaboration pays dividends at all levels Citizens can co-produce services

30. Continuing the dialogue….. Ray Shostak, CBE ray@shostak.uk.com