DFID s Results Framework focus on Multilateral Results EU Expert Meeting on Results, Brussels 16 November 2011

1. DFID�s Results Framework � focus on Multilateral ResultsEU Expert Meeting on Results, Brussels 16 November 2011 Welcome to the presentation today. Aim of presentation: This presentation describes the main elements of DFID�s new Results Framework with a focus on the results achieved through our contributions to multilateral organisations, including the European Commission. A version of this presentation was recently (8 November 2011) made to DFID�s Directors and Heads of Departments, to enable them to understand how the monitoring and reporting of results achieved �on the ground� is used as part of DFID�s internal management and decision-making and also in terms of meeting our statutory obligations to report on the use of public funds that DFID manages. Welcome to the presentation today. Aim of presentation: This presentation describes the main elements of DFID�s new Results Framework with a focus on the results achieved through our contributions to multilateral organisations, including the European Commission. A version of this presentation was recently (8 November 2011) made to DFID�s Directors and Heads of Departments, to enable them to understand how the monitoring and reporting of results achieved �on the ground� is used as part of DFID�s internal management and decision-making and also in terms of meeting our statutory obligations to report on the use of public funds that DFID manages.

2. Outline DFID corporate architecture Reporting requirements Principles of Multilateral Organisation (MO) reporting Data used in DFID Results Framework EU Results TWG - objectives

3. Corporate architecture DFID Results Framework (DRF) Tier 1 � Development outcomes Tier 2 � Output results Tier 3 � Operational effectiveness Tier 4 � Organisational effectiveness It is helpful to think of the corporate architecture in a bottom-up, chain fashion. So for the multilateral (MO) side, we have MOs� own results frameworks feeding in to the DFID Results Framework � through a UK attributed share of results at Tier 2. These results will also feed �right up the chain� to the 3 reporting publications at the top. These top-level publications are: DFID Annual Report � from a legal perspective this is the most important document; the presentation to Parliament of an Annual Report is a statutory requirement under International Development (Reporting and Transparency) Act 2006. Required data include assessment of bilateral and multilateral aid of meeting MDGs. MO specific data � amount of MO aid, broken down by MO; imputed share of aggregate MO spend including breakdown by country. DFID Business Plan � Government wide format. As such, the format of the document and, to an extent, the data requirements have been imposed upon DFID. The Business Plan contains the Structural Reform Plan, 8 input indicators and 7 impact indicators. Of these input indicators, 6 have a multilateral component; for the impact indicators, meanwhile, 4 have a multilateral element. Peter will outline the individual indicators shortly. DFID Changing Lives � essentially this documents reflects the Government�s emphasis on results and transparency. SoS notes in the foreword: above all, our new approach to development will be defined by our determination to deliver the greatest possible return on investment, both for the world�s poorest and for the British taxpayer. This document sets out the results that we expect from that investment.� Below these documents, the newly named DFID Results Framework � DRF for short � will provide the consolidated overview of headline results and lower-level corporate performance. The DRF follows international best practice established by Asian Development Bank and followed by other multilaterals. Tier 1 � what progress is there on development in the countries we work in? Tier 2 � what output results has DFID financed? Tier 3 � how well does DFID manage its operations? Tier 4 � does DFID manage itself effectively? Owing to the externally imposed structure, the Business Plan may convey the impression of sitting rather awkwardly against DRF. However, it is important to note that the input indicators are captured by tier 3 and the impact indicators included in tier 2 of the DRF. Finally, the bottom level shows the source of DFID�s data � made up of country offices� operational plans on the bilateral side and the MOs� results frameworks serving the multilateral side. It is helpful to think of the corporate architecture in a bottom-up, chain fashion. So for the multilateral (MO) side, we have MOs� own results frameworks feeding in to the DFID Results Framework � through a UK attributed share of results at Tier 2. These results will also feed �right up the chain� to the 3 reporting publications at the top. These top-level publications are: DFID Annual Report � from a legal perspective this is the most important document; the presentation to Parliament of an Annual Report is a statutory requirement under International Development (Reporting and Transparency) Act 2006. Required data include assessment of bilateral and multilateral aid of meeting MDGs. MO specific data � amount of MO aid, broken down by MO; imputed share of aggregate MO spend including breakdown by country. DFID Business Plan � Government wide format. As such, the format of the document and, to an extent, the data requirements have been imposed upon DFID. The Business Plan contains the Structural Reform Plan, 8 input indicators and 7 impact indicators. Of these input indicators, 6 have a multilateral component; for the impact indicators, meanwhile, 4 have a multilateral element. Peter will outline the individual indicators shortly. DFID Changing Lives � essentially this documents reflects the Government�s emphasis on results and transparency. SoS notes in the foreword: above all, our new approach to development will be defined by our determination to deliver the greatest possible return on investment, both for the world�s poorest and for the British taxpayer. This document sets out the results that we expect from that investment.� Below these documents, the newly named DFID Results Framework � DRF for short � will provide the consolidated overview of headline results and lower-level corporate performance. The DRF follows international best practice established by Asian Development Bank and followed by other multilaterals. Tier 1 � what progress is there on development in the countries we work in? Tier 2 � what output results has DFID financed? Tier 3 � how well does DFID manage its operations? Tier 4 � does DFID manage itself effectively? Owing to the externally imposed structure, the Business Plan may convey the impression of sitting rather awkwardly against DRF. However, it is important to note that the input indicators are captured by tier 3 and the impact indicators included in tier 2 of the DRF. Finally, the bottom level shows the source of DFID�s data � made up of country offices� operational plans on the bilateral side and the MOs� results frameworks serving the multilateral side.

4. DFID Results Framework This slide presents the newly named DFID Results Framework (DRF) in more detail. This will be the overall conceptual framework for future DFID monitoring, measuring and managing for results and is based on emerging international best practice in this area. We would encourage delegates to look at the model pioneered by the Asian Development Bank (ADB) and set out in their Annual Development Effectiveness reports for further details on methodology and approach. In our view, this basic model is one that any effective aid agency which cares about results should be looking to adopt - in a format tailored to meeting the needs of their organisation and its main stakeholders.This slide presents the newly named DFID Results Framework (DRF) in more detail. This will be the overall conceptual framework for future DFID monitoring, measuring and managing for results and is based on emerging international best practice in this area. We would encourage delegates to look at the model pioneered by the Asian Development Bank (ADB) and set out in their Annual Development Effectiveness reports for further details on methodology and approach. In our view, this basic model is one that any effective aid agency which cares about results should be looking to adopt - in a format tailored to meeting the needs of their organisation and its main stakeholders.

5. Reporting requirements Fortnightly - the Structural Reform Plan covers an array of commitments: Progress on 0.7% target and MDGs; building stronger accountability; developing UK�s �soft� power; creation of International Citizen Service (ICS); establishment of ICAI; publication of project information; strengthening evaluation function; citizen empowerment; focus on wealth creation, fragility & conflict, gender and climate change. Updates are commissioned every two weeks and are reported to number 10. Quarterly - the Quarterly Data Summary is a one page Excel-based snapshot showing progress on the input and impact indicators within the Business Plan, as well as latest programme spend and admin spend (estates, IT, procurement etc). Data on our workforce (e.g. on gender, disability; lost working days) are also reported. Quarterly � QMR report to the MB � levels 2-4 of the DFID Performance Framework (output results, operational effectiveness and organisational efficiency). It is similar to the QDS but is more comprehensive. Six monthly and yearly � Changing Lives. The SoS requires that he is updated every six months on the output results that DFID achieves. On an annual basis, it is likely that these updates will be reflected in a published Changing Lives update. Yearly � Annual Report will continue to be the principal document reporting DFID�s results. Its broad format is largely set owing to statutory requirements on the data that should be presented. Fortnightly - the Structural Reform Plan covers an array of commitments: Progress on 0.7% target and MDGs; building stronger accountability; developing UK�s �soft� power; creation of International Citizen Service (ICS); establishment of ICAI; publication of project information; strengthening evaluation function; citizen empowerment; focus on wealth creation, fragility & conflict, gender and climate change. Updates are commissioned every two weeks and are reported to number 10. Quarterly - the Quarterly Data Summary is a one page Excel-based snapshot showing progress on the input and impact indicators within the Business Plan, as well as latest programme spend and admin spend (estates, IT, procurement etc). Data on our workforce (e.g. on gender, disability; lost working days) are also reported. Quarterly � QMR report to the MB � levels 2-4 of the DFID Performance Framework (output results, operational effectiveness and organisational efficiency). It is similar to the QDS but is more comprehensive. Six monthly and yearly � Changing Lives. The SoS requires that he is updated every six months on the output results that DFID achieves. On an annual basis, it is likely that these updates will be reflected in a published Changing Lives update. Yearly � Annual Report will continue to be the principal document reporting DFID�s results. Its broad format is largely set owing to statutory requirements on the data that should be presented.

6. Principles of MO results reporting DFID�s Management Board agreed in May: MO results will be reported separately from the bilateral programme MO results will be given equal weight to bilateral results DFID will not claim a share of MO results but will present its funding contribution alongside the results

7. MO output resultsWhat results has DFID contributed to? Table of multilateral outputs published for first time in 2011 Annual Report Shows the MO�s own results Covers MDBs, Global Funds, some UN agencies and PIDG � but not EC in 2011, as relevant data not available To complete the presentation, I�ll spend a little time setting out the individual indicators. These are given in the handout that you have in front of you; this handout is taken from the 2010/11 DFID Annual Report. Firstly, let�s take a look at the table of MO output results. These results equate to and form a basis of level 2 of the DFID Results Framework (and in many cases the results are drawn from the MO�s own four-tier results framework). This table is the basis of our reporting MOs� own results. The key question here is �what results has DFID contributed to as a result of its funding of the MO?� (So again, it reflects the principle of contribution not attribution.) Yet, we do want to outline the funding contribution as this is important to transparency. Now, one might legitimately ask why we do not aggregate results across MOs themselves. The reason why we don�t do this is that definitions can vary � so aggregation may be misleading. Having said that, we are obviously keen to avoid double counting and avoid providing two indicators if the result of one of the indicators show up in the result of another indicator. An illustration of this is our decision not to show the insecticide treated bednet figure for UNICEF, where we believed that some of the UNICEF result would have been reflected in the GFATM total � which we did show. It includes results for MDBs (IDA, AsDB, AfDB), Global Funds (GAVI, GFATM), some UN agencies (IFAD, UNICEF, UNFPA) and PIDG. It does not include any EC results because there are no recent numbers to report. In 2010 the EC published their �MDG progress� report which contains quite a few results, but they are aggregated across 5 or more years, and we were not confident that if we included them in the table this year, there would be updated numbers to report in subsequent years. Other gaps � GPE, humanitarian, climate and environment, WB cash transfers. IDO is working with the teams to fill the gaps and will provide regular updates on the results agenda to HoDs. The basis for inclusion of the results is the set of 23 commitments included within the MO page of the Changing Lives document. We will formally track progress towards these targets.To complete the presentation, I�ll spend a little time setting out the individual indicators. These are given in the handout that you have in front of you; this handout is taken from the 2010/11 DFID Annual Report. Firstly, let�s take a look at the table of MO output results. These results equate to and form a basis of level 2 of the DFID Results Framework (and in many cases the results are drawn from the MO�s own four-tier results framework). This table is the basis of our reporting MOs� own results. The key question here is �what results has DFID contributed to as a result of its funding of the MO?� (So again, it reflects the principle of contribution not attribution.) Yet, we do want to outline the funding contribution as this is important to transparency. Now, one might legitimately ask why we do not aggregate results across MOs themselves. The reason why we don�t do this is that definitions can vary � so aggregation may be misleading. Having said that, we are obviously keen to avoid double counting and avoid providing two indicators if the result of one of the indicators show up in the result of another indicator. An illustration of this is our decision not to show the insecticide treated bednet figure for UNICEF, where we believed that some of the UNICEF result would have been reflected in the GFATM total � which we did show. It includes results for MDBs (IDA, AsDB, AfDB), Global Funds (GAVI, GFATM), some UN agencies (IFAD, UNICEF, UNFPA) and PIDG. It does not include any EC results because there are no recent numbers to report. In 2010 the EC published their �MDG progress� report which contains quite a few results, but they are aggregated across 5 or more years, and we were not confident that if we included them in the table this year, there would be updated numbers to report in subsequent years. Other gaps � GPE, humanitarian, climate and environment, WB cash transfers. IDO is working with the teams to fill the gaps and will provide regular updates on the results agenda to HoDs. The basis for inclusion of the results is the set of 23 commitments included within the MO page of the Changing Lives document. We will formally track progress towards these targets.

8. Changing Lives, delivering results Key policy document in terms of results 24 �we will� output commitments 3 we wills have an MO element � people out of poverty (composite of MO and bilateral); teacher training (MO only); immunisations (MO only) Attribution will apply � so exception to the principle that DFID will not claim share of result 23 output commitments on MO page This is the foundation of the MO output results table, where 18 Changing Lives commitments feature Work has begun to assess progress against Changing Lives; input of international teams will be crucial DFID Changing Lives � essentially this document reflects the Government�s emphasis on results and transparency. SoS notes in the foreword: above all, our new approach to development will be defined by our determination to deliver the greatest possible return on investment, both for the world�s poorest and for the British taxpayer. This document sets out the results that we expect from that investment.� Because of the document�s focus on results, we are required to provide updates on a six monthly basis. Precisely, these three we wills are as follows: We will provide more than 50 million people with the means to help work their way out of poverty; We will train more than 190,000 teachers and improve the quality of education and children�s learning; We will help immunise more than 55 million children against preventable diseases. IFAD (259,200); AsDB (134,558); IFC (4,480,000); remainder is bilateral (Private Sector). WB (110,000); AsDB (79,695) GAVI (55,000,000 � now revised up to 80,000,000 although extension of reporting year by one year) On attribution, we will need to be clear about the methodology. Further, with the composite we will on prosperity, aggregation will also need to be made � and again, we will need to be clear on the methodology. The need to avoid double counting presents a real methodological challenge. The biggest concern is the absence of output results for the EC results. Changing Lives mentions 5: People helped to cope with volatile food prices 50,000,000 People given access to safe water 14,500,000 People provided with individual or collective energy services 6,700,000 Children not yet in school supported millions Election observation missions 40 It is difficult to say definitively on the basis of a quick check that the targets will be achieved: Timeframes for results not clear from Changing Lives document; Need to understand the impact that significant replenishment scale-ups will have in terms of results. IDO will meet with teams over the coming months for a detailed discussion on progress on Changing Lives commitments and again we will keep HoDs informed. DFID Changing Lives � essentially this document reflects the Government�s emphasis on results and transparency. SoS notes in the foreword: above all, our new approach to development will be defined by our determination to deliver the greatest possible return on investment, both for the world�s poorest and for the British taxpayer. This document sets out the results that we expect from that investment.� Because of the document�s focus on results, we are required to provide updates on a six monthly basis. Precisely, these three we wills are as follows: We will provide more than 50 million people with the means to help work their way out of poverty; We will train more than 190,000 teachers and improve the quality of education and children�s learning; We will help immunise more than 55 million children against preventable diseases. IFAD (259,200); AsDB (134,558); IFC (4,480,000); remainder is bilateral (Private Sector). WB (110,000); AsDB (79,695) GAVI (55,000,000 � now revised up to 80,000,000 although extension of reporting year by one year) On attribution, we will need to be clear about the methodology. Further, with the composite we will on prosperity, aggregation will also need to be made � and again, we will need to be clear on the methodology. The need to avoid double counting presents a real methodological challenge. The biggest concern is the absence of output results for the EC results. Changing Lives mentions 5: People helped to cope with volatile food prices 50,000,000 People given access to safe water 14,500,000 People provided with individual or collective energy services 6,700,000 Children not yet in school supported millions Election observation missions 40 It is difficult to say definitively on the basis of a quick check that the targets will be achieved: Timeframes for results not clear from Changing Lives document; Need to understand the impact that significant replenishment scale-ups will have in terms of results. IDO will meet with teams over the coming months for a detailed discussion on progress on Changing Lives commitments and again we will keep HoDs informed.

9. Input & impact indicators (DFID Business Plan)Operational performance Input indicators Cost/child in primary school Bednets average unit price Cost/person of sanitation DFID spend on elections Cost/birth by birth attendant Cost per person of improving access to financial services DFID spend through MOs DFID spend on climate change DFID Business Plan � this shows the 8 input and 7 impact indicators. They are supposed to be related � and some of ours are but not all. The indicators in blue are those which have a multilateral element in addition to a bilateral � except for DFID spend through MOs, which is obviously exclusively multilateral. The underlined data are those where no data have yet been published. Again, the principle that the MO result is reported alongside but not aggregated with the bilateral result holds. As with the MO output results, we have already published results for the input and impact indicators. For the input indicators, 4 of the 6 MO indicators now have a published result. The 2 that don�t are 1) cost per person of improving access to financial services and 2) cost per person of providing sustainable access to sanitation with DFID support. It is envisaged that data for these will be available next year. On the impact side, 2 of the 4 MO indicators have a published result. The 2 that don�t are 1) number of people DFID supports to cope with the impact of climate change (where there is also no bilateral result) and 2) number of people with sustainable access to an improved sanitation facility with DFID support. Again, results are expected next year. As with the �we will� commitments, there is an exception to the principle of no attribution. Attribution will be applied to the impact indicators with a multilateral element; again an important point will be that the methodology is made explicit. However, aggregation will not be made. Further, a footnote will be provided to advise users not to aggregate the data due to the likelihood of making a double counting error. Double counting would arise because country offices will count multi-bi results in their results; however, these results also feature in MOs� own results frameworks. If we think of the four tier framework, the input indicators represent the tier 3 measures of organisational effectiveness. How DFID performs here will feed in to the higher level, tier 2 output results (which, confusingly, are labelled impact indicators in the Business Plan!). For example, we need to put sufficient resources into our climate change programmes in order to ensure that enough people are supported in dealing with the effects of climate change. Similarly, if the unit cost of delivering a baby is unnecessarily high then the number of births that will be supported by a qualifed attendant will fall. Finally, I haven�t mentioned tier 4 of the DFID Results framework. At this level it is more about DFID staff members (e.g. absence rates; Chapter delivery; air miles travelled) � standard corporate management functions � and so international teams are no different from teams working on the bilateral side. However, it is important that institutional teams are cognisant of their own MO�s four tier results framework reflects principle of learning from others.DFID Business Plan � this shows the 8 input and 7 impact indicators. They are supposed to be related � and some of ours are but not all. The indicators in blue are those which have a multilateral element in addition to a bilateral � except for DFID spend through MOs, which is obviously exclusively multilateral. The underlined data are those where no data have yet been published. Again, the principle that the MO result is reported alongside but not aggregated with the bilateral result holds. As with the MO output results, we have already published results for the input and impact indicators. For the input indicators, 4 of the 6 MO indicators now have a published result. The 2 that don�t are 1) cost per person of improving access to financial services and 2) cost per person of providing sustainable access to sanitation with DFID support. It is envisaged that data for these will be available next year. On the impact side, 2 of the 4 MO indicators have a published result. The 2 that don�t are 1) number of people DFID supports to cope with the impact of climate change (where there is also no bilateral result) and 2) number of people with sustainable access to an improved sanitation facility with DFID support. Again, results are expected next year. As with the �we will� commitments, there is an exception to the principle of no attribution. Attribution will be applied to the impact indicators with a multilateral element; again an important point will be that the methodology is made explicit. However, aggregation will not be made. Further, a footnote will be provided to advise users not to aggregate the data due to the likelihood of making a double counting error. Double counting would arise because country offices will count multi-bi results in their results; however, these results also feature in MOs� own results frameworks. If we think of the four tier framework, the input indicators represent the tier 3 measures of organisational effectiveness. How DFID performs here will feed in to the higher level, tier 2 output results (which, confusingly, are labelled impact indicators in the Business Plan!). For example, we need to put sufficient resources into our climate change programmes in order to ensure that enough people are supported in dealing with the effects of climate change. Similarly, if the unit cost of delivering a baby is unnecessarily high then the number of births that will be supported by a qualifed attendant will fall. Finally, I haven�t mentioned tier 4 of the DFID Results framework. At this level it is more about DFID staff members (e.g. absence rates; Chapter delivery; air miles travelled) � standard corporate management functions � and so international teams are no different from teams working on the bilateral side. However, it is important that institutional teams are cognisant of their own MO�s four tier results framework reflects principle of learning from others.

10. Objective of EU Results TWG How can EC and MS further strengthen our management and reporting of results? Other multilaterals including the World Bank and ADB now able to regularly report aggregated results Agenda for Change and proposals for EU Common Results Framework are a good first step UK wants to see clear action plan, with targets and milestones, for how the EC will strengthen its own results framework This will be critical in forthcoming budget discussions for next MFF 2014-20