Monitoring & Evaluation Aspects for Round 6 "Targets & Indicators Table" + M&E Self Assessment

1. Monitoring & Evaluation Aspects for Round 6 "Targets & Indicators Table" + M&E Self Assessment Checklist

2. Presentation outline • Performance Based Funding (PBF) • Round 6 Documents: • M+E requirements for complete proposal • The M+E framework & relevance to R6 • Possible Tools & Lessons Learned • Some tools for R6 M&E guidance • Introduction of "M&E Self Assessment checklist" • Looking back at some common challenges Appendix - Moving to a Grant Agreement

3. 1(a). PBF - Importance of M&E We adhere to Performance Based Funding model Some key intended outcomes Long term Ensure investments are made where impact on the diseases can be achieved Strategy level Provide incentives to focus on results and timely implementation Implementation level Help countries identify: • Effective efforts for early replication & scale-up; or • Take early corrective actions

4. 1(b). PBF Diagrammatically – Link between Funding & Performance Performance Based Funding Programmatic monitoring Financial monitoring  Regular disbursements of funding depend on programmatic and financial performance PBF/050106/1

5. 1(c). PBF - Key Principles • Country ownership &leadership • Harmonization with existing systems • Coordination with technical partners • Simplicity - small set of relevant indicators for reporting on performance

6. 2(a). Round 6 – M&E Requirements Very similar framework to Round 5 Number of key sections to be completed: Sections: 4.3.1- Component executive summary 4.6 - Component Strategy 4.9 - Description of the plan for M&E of program & how it fits in with the national M&E framework 5.6 - M&E budget summary (functional split of budget) + Attachment A – mandatory on per-component basis Why? Attachment A is the Applicant’s statement of the expected performance and impact over proposal term

7. 2(b). Round 6 – M&E Framework Focusing a little more on section 4.9  The proposal should: • Form an integral part of the national strategic plan • Build on and strengthen the existing M&E system • Showconsistency between goals → objectives → strategies (SDAs) → selected indicators Indicators/milestones selected for reporting should be supported by a detailed Work Plan and Budget

8. 2(c). Building Measurement Framework Not a Global Fund system – as Indicators and measurement tools have been developed with relevant partners for all levels Impact System effects Grant performance Operational performance Expected overall impact of contribution of Global Fund resources if performance meets outcomes anticipated from grant agreements

9. 2(d). Round 6 – Indicator selection Guidelines suggest • Simplified set of indicators Focus is multi-agency M&E Toolkit – Jan 06 version Around 12 to 18 indicators (depends on circumstances) • No 'Global Fund Indicators' – even the 'Top 10' • Use a simple M&E framework • On short-term: indicators focusing on SDAs for the diseases • On long-term: behavior change or decrease in disease morbidity and mortality • Good balance between routine data collection & surveys

10. 2(e). Round 6 – Indicator selection:Lessons from earlier Rounds • Reporting on performance, important to have: • Someindicators that can be reported on quarterly and/or 6 monthly(depends on existing measurement framework);and • Many that can be reported by months15-18 month of Phase 1 (Important for Phase 2 process) • Select standard impact and outcome indicators to monitor longer-term achievements and performance outcomes • Avoid duplication – use existing in-contry indicators, data collection systems and planned surveys(e.g. from earlier GF grants or national systems)

11. 2(f). Round 6 – Mapping Goals, Objectives, SDAs and Main Activities(TB hypothetical example) Impact indicator TB prevalence rate Goals To reduce morbidity and mortality due to TB Outcome indicator Case detection rate Treatment success rate Objectives To increase case detection & treatment success rate To intensify effective NTP partnership and collaboration Service delivery areas Timely detection and quality treatment of cases MDR-TB PPM DOTS Number of new smear positive TB cases detected Main activities Detection and treatment of TB cases Number & % of new smear positive TB cases successfully treated Maintain adequate drug and laboratory supply Number of health facilities strengthened to provide TB diagnosis and treatment Number of lab technicians trained Capacity building of service providers Number of health center staff trained in DOTS

12. 3(a). Some Possible Tools for M&E for Round 6 A number of tools are available to assist Applicants, including… • Multi-agency M&E Toolkit (Jan 2006) • Proposal Form & Guidelines • FAQ on Global Fund Website (see "Tools")For example, includes, a tool from "Health Metrics Network" for M&E & HIS gaps which is of particular relevance to HSS Also – from 2 June 2006, country-owned "M&E Self Assessment Checklist"

13. 3(b). Possible Tools – M&E Self Assessment Checklist Goal enhance country ownership of gapassessment & solution identification Objectives  and Relevance to R 6 • Better identify M&E capacity gaps • Guide investments in M&E before grant signature • Help identify key strengths required of an effective PR • Ensurenew resources contribute to strengthening of the National systems(avoid parallel reporting systems) • Improve quality of programmatic data to enhance planning & support performance based funding

14. M&E Checklist 1.M&E Plan 2.Data managementcapacities of the PR 3.Data collection & reporting systems per Program Area (e.g., ARV) 3(b). Possible Tools – M&E Self Assessment Checklist The M&E self-assessment Checklist assesses 3 areas: • To assess the strength of the M&E Plan (for the program for Global Fund grant – but not Global Fund only)  • To assess the capabilities of the PR to manage data related to the implementation of the Grant.  • To assess the strengths of the data-collection and reporting systems per Program Area, including the ability to report valid, accurate and high quality data related to implementation. 

15. 3(b). The M&E Self Assessment Checklist – Timing for Round 6 Expected "go live" date is Friday 2 June 2006 • Will be on R6 documents page • Not a required part of proposal However • may help find & effectively plan for capacity gaps • Will be required for Grant signature (PR obligation) And  Could be "annual model" for all programs from all donors – to better achieve harmonization

16. 3(c). Recap – some common pitfalls from earlier Rounds • Lack of consistency between the goal → objectives → service delivery areas → indicators • Poor indicator selection - Main objectives of proposal not monitored • Too many indicators – especially low level • Discrepency between targets set at the time of grant signing & performance reported • Not built on existing M&E and HIS systems, therefore difficult & too costly to monitor indicators

17. Questions?

18. Appendix Moving to a Grant Agreement • Submission of a M&E plan • Role of LFA in Assessment of M&E capacity • Implementing the Grant Performance system • Key actors inside the Global Fund

19. 4. Moving to Grant Agreement (a) Submission of M&E Plan (b) Assessment of M&E capacity by LFA For Round 6 – M&E Self Assessment is 1st step Harmonization with existing GF grants (if any): • Use standard indicators among grants • Combine M&E plans if possible/required • Match new baselines for impact and outcome indicators with existing grant(s) • Align disbursement requests and progress updates

20. 4(a). Moving to a Grant Agreement Submission of M&E Plan When PRs are at this stage - there are some “key messages” • There are no Global Fund specific indicators • Recommended indicators: standard global indicators approved by partners • Adapted to fit in the GF framework • Stick to targets in the proposal • Only use indicators relevant to proposed activities - standard definitions and increased use of GF Top 10 • Countries can use national indicators if they better capture program activities • Move from output towards outcome and impact indicators • Strengthen network and collaboration with partners: broker role in filling M&E demand

21. 4(b). Moving to Grant AgreementRole of LFA The role of the Local Fund Agent includes: Before Grant Agreement: • Assessing PR/Repeat PR’s M&E capacity and Plan • Ensuring that selected indicators/targets can be routinely reported and are supported by the Work Plan and Budget After Grant Agreement: • Carrying out on-site data verification • Critical analysis of performance and recommendations

22. 4(c). Implementing the Grant Performance System Grant lifecycle 6 Mth 12 Mth 16 Mth 18 Mth 2 Year Phase 1 end Grant Start 10mth 24mth 3 Mth 9 Mth 15 Mth Performance tools Annual Review Grant Agreement CCM Request for continued funding Grant Scorecard Grant Proposal Form Disbursement Request/ Progress Update Annex A Attachment 3 Attachment 2 Attachment 1 Grant Performance Report Performance data in real time

23. 4(d). Key actors Inside the Global Fund Key actors: FPM, SIE and Legal Support operations team in ensuring regular reporting-selection of indicators Review the M&E Plan and assess PR M&E capacity based on LFA assessment report Analyse performance of key indicators (people reached by services, coverage, outcome, impact) SIE Role