Effective Evaluations: Process for Improving Public Policy

1. Effective Evaluations: Process for Improving Public Policy SINERGIA: Timely information for proper decision-making SINERGIA Technical Secretariat National Planning Department Republic of Colombia

2. In Colombia, we believe in effective evaluations

3. How effective are evaluations? Evidence • Importance of technical assistance to improve productivity of small producers • (only 10.8% had technical assistance) • Absence of a M&E System • (lack of continuous and relevant information • for decision making) Program redesign Result RDE • New program called “Rural Development with Equity” (RDE): • Targetingsmall and medium producers • Promoting cooperatives and economies of scale • Integral support • M&E mechanisms Agricultural Income Security (AIS) farm subsidies program Program Budget: $ 491,520 million year Impact Assessment (January 2008 - February 2011) Cost Assessment: $ 3.739 million

4. How effective are evaluations? Evidence Evolution to a more cost- effective alternative Additions resulting in the improvement of the initial conditions of the concession contracts (2.78 modifications per year) Need for the concession of highways not lanes Result Transition between generations of concessions has eliminated commercial risks worth $480 billion pesos Road concessions: infrasture program by concession Impact evaluation (June – December, 2010) Cost Evaluation: $380 million

5. How effective are evaluations? Evidence • Unjustified teacher absenteeism represented approximately 8% of GPS resources for education in 2007 • (770 billion) Better Performance of Public Policy Result The public perception survey showed that public school teacher attendance in the last twelve months has: General Participation System: resource transfers from the State to the territory (education, health and drinking water) Program Budget: $ 24 billion per year Institutional Assessment and Impact (December 2007 - September 2009)Assessment Cost: $ 2.5 billion

6. How effective are evaluations? Evidence Best intervention nomination Positive impacts on metalinguisticand spatialskills Result Incorporation of recommendations in program action plans and in extended Batuta councils Application of evaluation methodology to all children in the program Award nomination Juscelino Kubitschek Batuta: musical training program in vulnerable populations Program Budget: $ 9 billion per year Impact Assessment (June - December2010) Assessment Cost: $ 380 million

7. …in Colombia, an effective evaluation is one that is used in the decision-making process

8. What guarantees their effectiveness? DEPP Academic Training Standardized evaluation process The highly qualified SINERGIA team ISO 9001:2008 NTCG 1000:2009

9. Why are they effective? Evaluations are defined by government agreement (fase 0)… ..and respond to the strategic National Development Plan • 2011 Agenda with 55 thematic areas divided into: • Sustainable Growth and Competitiveness • Equal Opportunities • Peacebuilding • Cross-cutting Support • 2010 Agenda for 27 thematic areas divided into: • Equity • State Administration • Competitiveness

10. Why are they effective? • The participation of stakeholders facilitates and legitimizes evaluation results… … if program managers are involved, they receive continuous feedback, even before the final evaluation report

11. Why are they effective? The scope of the evaluation is defined by the understanding of the value chain… …which aims to identify bottlenecks Executive Evaluation Not only evaluates the impact Input Process Outputs Results Impacts There is a wide range of evalutions that can be tailored to the program to be assessed. Evaluation of Institutions Evaluation of Operations Evaluation of Results Evaluation of Impacts

12. Why are they effective? • The socialization seeks to: • Reach the largest number of actors (at high and medium-high decision-making levels) • Give clear and concise messages SINERGIA uses different communication channels for socialization

13. … In summary Evaluations are defined by government agreement (phase 0) Evaluations are designed by the government with high levels of participation (phase 1) • The contract guarantees a competitive process in which the best methodology and work team are selected (phase 2) • SINERGIA leadership during implementation raises the technical level of the evaluation (phase 3) • Socialization reaches decision makers (phase 4)

14. In Colombia SINERGIA evaluations have evolved… More evaluations over time… Transports ICTs Social protection Defense Presidency Education Planning Defense Statistics Trade, IndustryandTourism EnvironmentandHousing Farming … More sectors analyzed

15. … articulated with NDP monitoring Annual (January - December) Annual (August - July)

16. … in this way Territorial Supports the Government in staying the course and improving performance to achieve the public policy objectives

17. What is the next step??? SINERGIA continues to innovate: NATIONAL RADAR ASSESSMENTS Sociedad civil PD y LP Seguimiento 2009 Presupuesto participativo – Pasto* 2011 Asistencia técnica a emprendimientos productivos 2009 Plan Pacífico 2009 PAD + open data SINERGIA Synthesis of evaluations Descentralización de la Política de derechos Humanos 2011* Tolerancia Social Violencia basada en genero 2010 Generación de ingresos 2009 Política de Reintegración 2010 Promoción cultura del ahorro 2010 Obras para la Paz 2008 Plan Nacional de Consolidación 2010 Estado Evaluación en diseño Evaluación finalizada

18. Information is the currency of Democracy Thomas Jefferson

19. THANK YOU PUBLIC POLICY EVALUATION DIRECTORATE NATIONAL PLANNING DEPARTMENT REPUBLIC OF COLOMBIA Tel: (571) 381 5270 Email: sinergia-DNP@dnp.gov.co