Changes in and Goals of USAID M&E Reporting Requirements

1. Changes in and Goals of USAID M&E Reporting Requirements February 6, 2007 Jackie Doremus, CARPE M&E Consultant

2. Outline of Presentation • Success of M&E to date – Broadly and the Operational Plan case study • Overall goals of CARPE M&E – why do we want this information? • Brief overview of current M&E system • Proposed CARPE M&E Calendar • Feedback from Partners’ Phase IIA final reports

3. Outline of Presentation • Proposed updates to CARPE M&E system • SO level indicator • Updated matrix • MOVs • Narrative structure for LS consortia • GIS data • Feedback and distribution of tools

4. Success of CARPE M&E System: Broadly • M&E system developed to satisfy all CARPE USAID reporting requirements, finalized Feb 2005 • Effectively communicated results to Congress, helping ensure a fourth year of funding

5. Success of CARPE M&E System: A case study • December 2006 saw major changes in USAID’s reporting structure • New, comprehensive reporting tool, the Operational Plan, required for each Operating Unit • Entered into a web-based system, FACTS: Foreign Assistance Coordination and Tracking System • Data requirements much greater than previous USAID reports • Very little ‘warning’ or collaboration with the field on the content of the report

6. Success of CARPE M&E System: A case study • CARPE team was only SO in USAID/DRC to not return to partners for repeated information requests to satisfy FACTS* • Operational Plan design mimicked CARPE database design • Information entered into FACTS was auditable and robust due to MOVs and matrices

7. FACTS stats from Matrix • FACTS required activity budgeting • Average % of the total LS budget spent on “Training/Capacity Building” is 14% • CARPE funds led to at least 2,163 people receiving training with FY06 funds • FY06 LS funds were used to improve management on 32.6 M ha

8. Goals of CARPE M&E • FUNDING • Determine project warrants funding • Use all available resources to communicate results and ensure funding • Regular conduit of communication between partners and CTO • Better to have structured and well-anticipated report submission than ad hoc requests and changing requirements • Must efficiently use time and resources of partners during reporting submission and of CARPE team during processing of reports • Matrix requires an initial investment from partners to learn format, then an annual large investment when consortia create the workplan • Landscape leaders will need to help new subpartners

9. Goals of CARPE M&E

10. Overview of CARPE M&E • Semi-Annual Reporting Cycle: • 1. A workplan matrix and narrative • 2. A Semi-Annual Report matrix and narrative • 3. An Annual Report matrix and narrative • MOVs • A one-time GIS data submission

11. Overview of CARPE M&E • 1. Partners submit a workplan matrix and proposed budget • Analyzed by CARPE using rubric (ex ) • FY06 Workplans were submitted in CARPE IIB RFA proposals

12. Overview of CARPE M&E • Workplan data is entered into a MS Access database • The database links Landscape financial and performance data • Financial: Activity budgeting by zone type, partner, and country, semi-annual expenditure rate • Performance: Each zone’s name, benchmark, size, country location, and type • Allows CARPE rough comparisons between landscapes and a way to quickly scale up the regional program • Eventually data will be represented spatially using zone and landscape polygons

13. Overview of CARPE M&E

14. Overview of CARPE M&E • 2. Partners submit Semi-Annual Report (SAR) matrix and narrative • SAR is a tool for partners to communicate to the CARPE Operating Unit on progress toward benchmarks and any proposed changes • Adaptive management—flexible • Required in Cooperative Agreements • Level of Effort: • No major changes made to matrix submitted as workplan unless changes in benchmarks • Percentage accomplished on right-side is updated • CARPE uses SAR performance (financial and programmatic) to influence budgeting of LS funds

15. Overview of CARPE M&E • 3. Partners submit an Annual Report matrix, narrative, and MOVs • Matrices: Final results are updated into database for reporting to Washington • FACTS, Global Climate Change, 118/119, Performance Report, etc • Narratives: Success stories and reporting is created from accomplishments listed here • MOVs: catalogued, cross-checked, and evaluated • Important as a check to show that results were achieved • Required by USAID data quality assessment • Esp important due to limitation of field visits • Some published on CARPE website

16. Proposed CARPE Reporting Calendar Try to continue approximately this cycle for length of agreements to help planning, though flexibility is necessary

17. Updating of M&E System • All proposed updates are small and have been balanced with need for stability in reporting • Changes will: • Better measure CARPE results as the program matures • Ensure CARPE receives complete information from bundled consortia • Segments reporting was automatically disaggregated by country • Respond to Partners’ feedback in Final Reports • Improve data quality • Help CARPE reporting burden with reduction in team size

18. Summary of Partners’ Feedback • Thank you! Feedback was very helpful • Across the board notes: • Request limitation in future changes to format – has stabilized since Feb 2005 workshop • Matrix is resource and time intensive • Request CARPE distribute a clear reporting calendar to help partners’ plan meeting logistics and time management • Most partners find the matrix an important and useful tool • Some have incorporated it into their own LS monitoring plans • Useful for team building and will be important for guiding new consortia and managing sub-grants • Smaller segments and segments with many donors found it burdensome • Request for French translation of reporting tools • Matrix available in French, guidance currently is not • Recommendation for more site visits by CTO

19. Proposed Updates to M&E System:SO level indicator • SO Level indicator from PMP: • “Population status for selected biodiversity “indicator” species such as: wide-ranging “landscape” species and/or ecological keystone species (e.g. elephants, large predators) and/or globally threatened species (such as, mountain gorillas, bonobos, etc.)” • Program maturity • Unit of measurement described in PMP: No. of individuals of indicator species • Timing: Biennial/triennial • Compatible with SOFR Indicators • Will be separated and not linked to tasks

20. Updated Matrix • People trained summarized on separate page • Reported by “training event” • Disaggregated by gender • No. of days trained • On-going tasks link to MOVs and LUP progress should be described as best as possible • Percentages now given semi-annually on workplanning (right) side • Changes from Workplan or SAR marked in RED font and explained in narrative • no “comparison percentages” (since changes highlighted in red) • Zone names • Add zone ID (assigned by CARPE, see future handout) • Add country where zone is found in parentheses, below name

21. Updated Matrix • Break down budgets by country whenever possible, even within the same NGO • Example: WWF in TNS Cameroon and CAR • Linked to Transformational Diplomacy reporting requirements • Annual Reports and Workplans will be posted on the CARPE website (without financial data) • Feedback form

22. Updates: Means of Verification (MOVs) • Hyperlinking has helped control the MOVs, however some were missing or mis-linked for FY05 • If MOVs are missing without explanation, CARPE will report the benchmark as not achieved in Washington reporting • MOVs are sifted through by CARPE team and appropriate material will be published on the CARPE website

23. Narrative Updates • Narrative format will continue in the same format as before, with increased page limits to accommodate LS consortia • Suggestions welcome in feedback form • Annual Report will include “success stories” with photos, like the Final Reports of Phase IIA

24. GIS Data • CARPE partners are active in LUP in more than 150 zones • Shapefile data received for Phase IIA zones • Quality was uneven, few partners sent in metadata, required great time and resources from UMD/NASA team to clean data • Destination for shapefiles • CARPE website • CARPE management tool

25. GIS next steps • Landscapes will give CARPE the names and contact information for their GIS experts so that they can be contacted directly • The polygons for the remaining zones for Phase IIB will be sent in to CARPE by a date agreed at this workshop • Suggestion: one month from now?

26. Feedback and distribution of tools • A form has been created and feedback on all of these issues is welcome and appreciated • Feedback will be compiled and a consensus distributed by end of workshop. Most importantly: • Reporting Calendar • GIS point of contact and polygon due date

27. Feedback and distribution of tools • The updated tools (Workplan, SAR, and AR matrices and narratives), zone ids, summary of changes, and this presentation will be available on the CARPE website and can be transferred electronically by flashdisk at this workshop after feedback is compiled and incorporated • The summary of the decisions made here on CARPE M&E will also be distributed in paper after the feedback has been compiled