Conference on Public Sector Management in Support of the MDGs 13-15 June 2012, Bangkok

1. Conference on Public Sector Management in Support of the MDGs 13-15 June 2012, Bangkok Monitoring and Evaluation of the MDGs at the Regional Level Jan Smit ESCAP Statistics Division

2. Overview • Asia-Pacific MDG reports • Context and history • Tools • Strengths and weaknesses of the MDGs as a development framework • Lessons learned for the Post-2015 development agenda • Importance of timely and reliable data 2 Monitoring and Evaluation of the MDGs at the Regional Level

3. Levels of MDG progress tracking / reports • Global: “The Millennium Goals Report” • Issued annually by UN Secretary General since 2005 • Compare (sub-)regional progress • http://mdgs.un.org/unsd/mdg/ • Regional • Asia-Pacific, Africa, Arab Region, Latin America & the Caribbean, Europe & Central Asia • Sub-regional • E.g. Pacific 2004 • National reports • United Nations Development Group website • http://www.undg.org/index.cfm?P=87 Monitoring and Evaluation of the MDGs at the Regional Level

4. Asia-Pacific MDG reports • ESCAP, ADB and UNDP Partnership “Supporting the achievement of the Millennium Development Goals in Asia and the Pacific” • Initiated in 2002; now Phase 3: 2010-12 • Data and policy-analytical work through production of Asia-Pacific MDG Reports • Strengthen capacity of national statistical systems to produce, analyze and disseminate high-quality MDG-related data • Strengthen national capacity to engage at regional and sub-regional cooperation and formulate solutions to achieving the MDGs Monitoring and Evaluation of the MDGs at the Regional Level

5. Asia-Pacific MDG Reports history • Full-blown reports • 2003, 2005, 2008, 2009/10, 2010/11 and 2011/12 • 2 more scheduled: 2013 and 2015 • Include progress assessments (1st chapter) • Standalone progress assessment reports • 2006 and 2007 • Series of technical background papers, e.g. • Achieving the MDGs in Asia: A case for more aid? • Available at: • http://www.unescap.org/publications/ • http://www.adb.org/Publications/ Monitoring and Evaluation of the MDGs at the Regional Level

6. Asia-Pacific MDG Reports • Try to gauge “Who are left behind?” by answering the questions: • How are Asia-Pacific countries doing vis-à-vis each other and (sub-)regional averages? • How is Asia-Pacific and its sub-regions doing compared to the rest of the world? What are the Targets of main concern? • Where has progress been too slow? What is the change in speed required to hit the target? How many people would benefit? • Which population groups within countries are more vulnerable than others? • Where are data lacking and consequently needs statistical capacity to be strengthened? Monitoring and Evaluation of the MDGs at the Regional Level

7. Tools needed • Selection criteria • For indicators, countries, (sub-)regions and other relevant country groupings • Data • For indicators and reference populations • Survey microdata for within-country disparities • Models and classification rules • To measure and classify progress of countries / regions • To gauge within-country disparities • Communication • Visualization of results to communicate messages as simple but powerful as possible Monitoring and Evaluation of the MDGs at the Regional Level

8. Selection criteria: Indicators … • Directly related to a Target • For which progress is measured for developing (and not developed) countries • That have quantitative targets to be reached by 2015 • Explicit target values for 2015 • Relative (reduce by ½, 2/3, ¾) • Absolute (full enrolment, gender parity) • Reversal of trends • “Have halted and begun to reverse…” (Targets 6A & 6C) • “Reduce biodiversity loss, achieving, by 2010, a significant reduction in loss” (Target 7B) • 22 in 2011/12 report Monitoring and Evaluation of the MDGs at the Regional Level

9. Selection criteria: Countries & regions • 55 countries of Asia and the Pacific • All developing ESCAP member countries • I.e. all members except Australia, New Zealand and Japan • Groups of countries: • Asia-Pacific and groups of countries therein • Sub-regions • Including excluding dominant countries • China and India • PNG in the Pacific • LDCs, low income and middle income • Sub-Saharan Africa and Latin America & Caribbean • To compare with Asia-Pacific Monitoring and Evaluation of the MDGs at the Regional Level

10. Data sources • Indicators: MDG indicator global database • http://mdgs.un.org/unsd/mdg • Product of Inter-Agency Group and Expert Group on MDG Indicators (IAEG), maintained by UNSD • Reference populations: World Population Prospects • http://esa.un.org/unpd/wpp/ • United Nations Population Division • Survey microdata • Demographic and Health Surveys (DHS) • http://www.measuredhs.com • Multiple Indicator Cluster Surveys (MICS) • http://www.unicef.org/statistics/index_24302.html Monitoring and Evaluation of the MDGs at the Regional Level

11. Data issues: MDG indicators • Discrepancies with national data, possibly resulting in differences in assessing progress • Global data typically drawn from official statistics provided by ministries and national statistical offices provided to international agencies • However, internationally agencies may: • Adjust data to ensure comparability across countries • Estimate data (for various reasons) • Also, countries may have more recent data • Backward revisions • Not only new, more recent data, added to database, but historical numbers are also revised • Progress results not comparable between MDG reports Monitoring and Evaluation of the MDGs at the Regional Level

12. 2011/12 report:Coverage & timeliness of Goal 1 data of particular concern Monitoring and Evaluation of the MDGs at the Regional Level

13. Modelling progress • Extrapolation of trend since 1990 to 2015 • Most series are transformed, depending on indicator • Linear least squares for most indicators • With quadratic term for indicators with possible turning points • All data points in country-indicator series used • I.e., rather than only first and last • For series of only 2 points, need to be ≥ 3 years apart • What if no data for 1990? • Target to be achieved from first year with data • Rather than backcasting series to 1990 or adjusting target for years left to 2015 Monitoring and Evaluation of the MDGs at the Regional Level

14. Monitoring and Evaluation of the MDGs at the Regional Level

15. Progress classification Four categories: ●Early achiever: Already achieved the 2015 target On track: Expected to meet the target by 2015 Off track-Slow: Expected to meet the target, but after 2015 Off track-No progress/regressing: Stagnating or slipping backwards Monitoring and Evaluation of the MDGs at the Regional Level

16. Early achiever Nepal Monitoring and Evaluation of the MDGs at the Regional Level

17. Early achiever Nepal Turning point achieved! Monitoring and Evaluation of the MDGs at the Regional Level

18. On track Nepal Monitoring and Evaluation of the MDGs at the Regional Level

19. Slow Nepal Monitoring and Evaluation of the MDGs at the Regional Level

20. Regressing / No progress Federated States of Micronesia Monitoring and Evaluation of the MDGs at the Regional Level

21. Strengths and weaknesses of the MDGs Not easy to do, since counterfactual is missing, i.e. we don’t know what would have happened without the MDGs However, it is clear that no previous development agenda has provided a common worldwide cause to address poverty and put human progress at the forefront Following slides borrow from the discussion in the UN Task Team to support the preparation of the Post-2015 UN Development Agenda 21 Monitoring and Evaluation of the MDGs at the Regional Level

22. Strengths of the MDGs Focus on a limited set of concrete, time-bound, common human development goals and targets Simplicity and transparency helped rally broad support Multi-dimensionality and emphasis on human development shifted policy attention well beyond economic growth objectives Helped to set priorities for national and international development priorities Strengthened global partnership for development Linked ODA, a fair multilateral trading system, debt relief and affordable access to new technologies and essential medicines to the human development agenda 22 Monitoring and Evaluation of the MDGs at the Regional Level

23. Strengths of the MDGs (cont.) Focus on Results Concept of concrete and time-bound goals and targets helped to identify set of indicators to track progress This in turn highlighted the need for the strengthening of national statistical systems and the use of timely and reliable statistics for the planning, monitoring and evaluation of policies Much has been achieved, but a lot remains to be done! 23 Monitoring and Evaluation of the MDGs at the Regional Level

24. Weaknesses of the MDGs Some targets weren’t adequately formulated, or didn’t adequately account for, issues of: Demographic dynamics Productive employment and decent work Violence against women Social protection Peace and security Governance and the rule of law Human rights Persisting inequalities No provision of vision on how to address root causes of poverty and unmet basic needs 24 Monitoring and Evaluation of the MDGs at the Regional Level

25. Weaknesses of the MDGs (cont.) Special case of MDG 8: Defined rather imprecisely, thereby weakening accountability for promised international support Insufficient access to predictable development finance, export markets, technologies and medicines remain important constraints to sustainable and inclusive development for many developing countries Shortcomings in the global partnership to harness development efforts against shocks caused by volatile world markets remain 25 Monitoring and Evaluation of the MDGs at the Regional Level

26. Lessons learned for post-2015 agenda Format of concrete and time-bound goals, targets and indicators should be retained MDG targets were realistic, which made them credible, but realism needs to be balanced against the need to be ambitious in, for example, combating climate change and preserving environmental sustainability Focus on ends should be retained Every country is different in terms of geography, demography, urbanization, etc., so there can be no blueprints for national development strategies However, general guidance on policies that foster productive investment and decent work and growth-oriented macroeconomic policies could be considered 26 Monitoring and Evaluation of the MDGs at the Regional Level

27. Lessons learned for post-2015 agenda (cont.) One-size-fits-all approach should be avoided Targets need to be tailored to regional, national and sub-national realities Country and regional consultations critical in formulating the agenda and adaptation of targets to country / regional contexts The extent of within-country disparities in many Asia-Pacific countries is large See especially Asia-Pacific MDG Report 2011/12 http://www.unescap.org/publications/detail.asp?id=1482 Need to track sub-national progress along lines of administrative regions, urban-rural locations, sex, age, education, ethnicity, caste, language, etc. Need for sub-national data! 27 Monitoring and Evaluation of the MDGs at the Regional Level

28. Lessons learned for post-2015 agenda (cont.) Need to respond to challenges not explicitly, or inadequately, addressed in MDGs: Reducing inequalities Addressing climate change Ensuring environmental sustainability Increasing resilience to natural hazards Accounting for demographic and epidemiological dynamics Dealing with urban growth Guaranteeing peace and security Improving governance Respecting human rights 28 Monitoring and Evaluation of the MDGs at the Regional Level

29. Importance of timely and reliable data Emphasis on results in MDG agenda increased demand for data and recognition of the need to develop statistical capacity Countries and development partners worked intensively to strengthen statistical programmes Discrepancies between international and national data Cause of friction, but also led to discussion about data quality according to international standards & recommendations Increased availability of quality data and their enhanced use in policy making improved policy implementation and monitoring Improved coordination of statistical system and collaboration among ministries / government agencies 29 Monitoring and Evaluation of the MDGs at the Regional Level

30. However: Statistical capacity varies widely across developing countries and remains insufficient in many small states, fragile situations and LDCs Many still not able to conduct household surveys without external financial and technical support And derive most of their MDG data from internationally sponsored household survey programmes, such as those for the Multiple Cluster Indicator Surveys (MCIS) and Demographic and Health Surveys (DHS) Improvements in availability of survey data often unaccompanied by strengthening of administrative data systems, such as the civil registration and for social service provisioning 30 Monitoring and Evaluation of the MDGs at the Regional Level

31. Way forward on data & monitoring systems Underlying data needed to measure MDG progress derives largely from statistical systems Yet: No explicit target, effort or program under current framework to invest in strengthening and improving these systems Needs therefore consideration for post-2015 Also: many issues like migration and climate change are trans-boundary Therefore: Consider under new global partnership Finally: Systems critical for citizens to be able to hold institutions accountable 31 Monitoring and Evaluation of the MDGs at the Regional Level

32. Thank you Strengthening the ‘evidence’ for evidence-based policy making: http://www.unescap.org/stat/ 32 Monitoring and Evaluation of the MDGs at the Regional Level