Policy Analysis

1. Policy Analysis • Analyzing the Problem • Bardach, the eightfold path • Education Sector Assessments • Program Evaluation • Analyzing the Process of Policy Making (Politics and Interests) • Haddad (national politics) • Kemmerer on Sector Assessments (national politics) • World Bank, Heyneman (intra-bureaucratic politics) • EFA, Jones (inter-agency politics) • Alternative ways to think about the relationship between Problem Analysis (research) and Process • Informed Dialogue

2. The Eightfold Path • Define the Problem • Assemble some Evidence • Construct the Alternatives • Select the Criteria • Project the Outcomes • Confront the Tradeoffs • Decide • Tell your Story

3. Assessing the Performance of an Education System. Conducting Education Sector Assessments.

4. Summarize the state or condition of the sector • Sector Analysis • Sector Review • Sector Assessment

5. Identification of sector needs and constraints and opportunities for system improvement • Sector Analysis • Sector Review • Sector Assessment

6. Judgement of the success of programs • Sector Analysis • Sector Review • Sector Assessment

7. Education Sector Assessments. A comprehensive review of the performance of an education and human resource development system, with emphasis in the linkages across sub-sector components. Systems-approach. Integrated by sub-sector assessments.

8. “A sector assessment is a detailed analysis of the goals and objectives, status, plans, needs, constraints, and priority target areas with rank ordered recommendations for action in a national education system…all components analyzed interdependently…Objectives of the assessment are too encourage the use of relevant and current information for planning, policy formulation, and resource allocation that will improve efficiency throughout the entire education sector.” (Cieutat and Pigozzi, 2-2).

9. Differs from a needs assessment and from an evaluation in that it is multi-issue, multi-program in scope, not focused on a specific policy or program. Evaluation focuses on a single activity or set of activities. • Sector analysis: Summarize the state or condition of the sector • Sector review: Judgement of the success of programs • Sector assessment: Identification of sector needs and constraints and opportunities for system improvement.

10. Sector assessments have been in existence since educational planning began in the 1950s. They have been supported and popularized by donors. In the 1980s a USAID supported-project (Improving the Efficiency of Education Systems, which began in 1984) sponsored many sector assessments in developing countries, popularizing a particular approach to conducting sector assessments described in Pigozzi and Cieutat’s manual.

11. Early examples: • Ashby Commission Report, Nigeria.1959. • USAID’s sponsored Nigerian Human Resources Development and Utilization 1964. • Report of the Education Commission in India (1964-1966): Education and National Development. • AID-sponsored assessment of education in Korea in 1971: Systems Analysis for Educational Change: The Republic of Korea. • USAIDs Bureau for Latin America sponsored several sector assessments, social demand approach. Conducted in late 1960s and 1970s.

12. Beneficiaries/Audiences: Narrow view: Government officials and donors • Broader view: Stakeholders for education reform. • Purposes of a Sector Assessment: • Baseline Information • System Efficiency • Constraints • Subsector Priorities • Synthesis • Planning Tool • External Assistance Agency Coordination • Institutionalization of systems analysis

13. Process of Carrying out a Sector Assessment • Preparation • Data Collection • Data Analysis • Identification of Issues and Contraints • Conclusions • Recommendations • Review “Frequent and careful review of the assessment during its development is an important part of the process. Reviews should occur not only at the end of the assessment but also during the process. They provide an opportunity to ensure that data and their interpretations are correct.” • Revision

14. Preparation • Definition of the Scope of Work (SOW) • Specify: Purpose, task, approach, staffing (level of effort), duration, reporting. • Initial Outline • Terms of Reference for Consultants • Management, institutional coordination. • Logistics (transportation, support, access, review coordination) • Costs • Timeline and schedule

15. Structure of a Sector Assessment • Synthesis • Economic and Financial Analysis • Overall macroeconomic situation • Employment and job creation • Emerging needs to support economic growth (Manpower supply and demand) • Rates of return • Fiscal capacity • History, structure and management capacity • Pre-Primary Education • Primary • Secondary • Tertiary • Technical/Vocational • NonFormal • Teacher Education • Special Studies

16. Each section of the assessment has a comprehensive descriptive section and an analytical section. • Data Sources: • Quantitative records • Policy documents • Existing studies • Interviews • Special, ad-hoc, studies

17. The descriptive section must address: • Historical setting and cultural context • Relevant national goals and objectives • Organization and Institutional Structures • Existing Programs: teachers, curriculum, materials, evaluation practices, facilities and equipment and costs.

18. Administration in each subsector: • Organization Chart, functions and staffing • Supervisory structure • Management personnel education and capabilities • Supply and demand of management personnel • Communication links through subsectors • Education and skills of principals • Teacher appointment, review, salary and promotion.

19. Students in each subsector: • Enrollments by grade • Age distribution • Access and capacity (gross and net enrollment) • Initial access and criteria • Cohort flow • Enrollment projections

20. Teachers in each subsector • Numbers by grade and location • Distribution by training, experience and specialization • Salary schedule • Age distribution • Student/teacher ratios • Teachers teaching

21. Curriculum by subsector • Purposes and content (Intended Curriculum) • Scope (national, regional, institutional) • Source • Instructional materials required and available • Teaching methods (Implemented Curriculum) • Language of instruction • Relevance of the curriculum • Student learning (Achieved Curriculum)

22. Assessment systems and research • Facilities • Costs and Financing • Analysis of Various Sub-sectors: • Identify Needs • Discuss Plans • Examine constraints • Highlight Isues • Summarize conclusions • Formulate recommendations

23. Analytic themes: • External efficiency • Internal efficiency • Access and equity • Administration and supervision • Costs and financing

24. Utilization of a Sector Assessment Purposes: answer whether the system meet its goals and if it does so efficiently. “Because equity is a stated goal in almost every educational system, the efficiency analysis must include an analysis of the system’s equity effects since, by definition such systems cannot be efficient if they are not also equitable” (Kemmerer). “The technical component [of sector analysis] is simply one step in an alternative approach to strategic planning; an approach built on inter- and intra-sectoral collaboration and on the collection and analysis of reliable and timely information on system efficiency”

25. Identification of key stakeholders Purposes of Sector Analysis to Encourage National Dialogue on Education:  • Promote national discussion of key educational issues • Expand the focus of educational ministries and programs • Promote collaborative problem solving • Clarify goals • Increase accountability • Win support for reform • Build a forum

26. Steps: • Initiation • Forming the National Steering Committee • Determining the Scope of the Analysis (Scope of Work) • Form the subsector teams • Form the technical analysis team • Collect data • Discuss preliminary findings • Collect additional data • Formulate draft recommendations • Discuss draft recommendations • Revise and prioritize recommendation

31. Government • Government is the most central and essential political actor because the other political actors are “mapped” (or categorized) in terms of their opposition or support to government. • External Actors • They include: multinational corporations, foreign embassies, international NGOs, and bilateral and multilateral donor agencies. External actors are often the economic fuel for many policy decisions. • Social Sectors • Social sectors are defined by a general interest or motive that brings individuals together through mass mechanisms. Examples are small farmers, agro-export farmers, urban working class, large landholders, industrialists, or minority groups. • Political Parties • Their main objective is to shape public policy via selecting candidates to run for office and appointing party loyalists to government positions. • Pressure Groups • Pressure groups can be labor unions, teacher coalitions, church organizations, business associations, NGOs, or other professional organizations that function to influence public policy.

32. Exercise in small groups 10 mins. • You are working as an education specialist for USAID in Pakistan. A recent education sector assessment has identified the low quality of science education in K-12 as a serious challenge. As you contemplate the design of a project to improve K-12 science instruction in the country, develop a political map of the stakeholders that are likely to support or oppose this project.

33. The utilization of sector analysis to improve education management information systems (EMIS). • The use of sector analysis as a strategic planning tool. • Going beyond the sector analysis • Reformulating mission statements • Developing detailed implementation plans • Scheduling implementation • Adjusting resource allocation systems • Assessing the implementation and impact of system changes • Adjusting policy

34. Reconciling Multiple Objectives. Logical Framework Analysis. • How do we turn broad education strategies and policies into workable programs and projects? How do we reconcile rational analysis with the politics of negotiating differences? How do we reconcile tradeoffs between the goals of equity and effectiveness?

36. Examples of Logical Frameworks • http://www.iadb.org/SDS/SOC/eccd/6example.html#ex1

37. The Logical Framework Approach is a project design methodology. Also know by the initials of the method in German ZOPP (Ziel Orientierte Projek Planung, or Objectives Oriented Project Planning or OOPP). This methodology, the Logical Framework Approach, calls for a structured sequence of activities that leads to the production of a Logical Framework (a Document).

38. The main steps in this process include: Analysis of the Present Situation • Analysis of Stakeholder Groups and Institutions Affected by the Problem • Problems Analysis • Objectives Analysis • Alternatives Analysis • Activities Planning • Logical Framework

39. The Logical Framework was developed and popularized in the design of USAID projects in the end of the 1960s and has been adopted widely by development organizations. • Key themes: • Objectives Oriented • Target Group Oriented • Participatory

40. Universal Primary Education = Poverty Reduced • Quality of Education Increased  National Productivity Increased • Confict Resolution Program Implemented  Social Violence Reduced

41. Distinguishing between Immediate and Development Objectives: • Both outside project control. Project hypothesis. • Cause-Effect.

42. Increased productivity of the labor force • Increased literacy of the population Increased healthy practices in population Higher educational attainment

43. Increased value added in information technology industries • Greater scientific knowledge of college graduates • Greater peace and security • More sophisticated international relations • More knowledge and awareness of global issues

44. Projects transform inputs into outputs. Activities are the processes that create the outputs using inputs. These three are within the project control. • Outputs are results than can be guaranteed by the project as a consequence of activities • Activities are actions necessary to transform given inputs into planned outputs within a specified period of time • Inputs are the resources of a project necessary to produce the intended outputs

45. Outputs • 100,000 graduates of literacy campaign Activities • 10,000 literacy workshops conducted in 2001 Inputs • 100,000 literacy booklets • 2,000 literacy promoters • 200 literacy coordinators • 500 all purpose vehicles • 2,000 sites for training

46. External Factors. Necessary and Sufficient Conditions. • External factors are necessary for project success but outside the control of project management. • Outside and inside the project. • Project Environment.

47. Examples: • Participants engage in training opportunities • Trained participants exhibit skills gained in training • Complementary resources (e.g. textbooks) are available. • Jobs are available • Parents send their children to new schools.

48. Critical factors need to be integrated as project outputs, when possible. Alternatively they need to be monitored carefully. • Indicators and Means of Verification. • Should specify: • Target group • Quantity • Quality • Time • Location

49. Examples: • Increased literacy, defined as ability to read and understand a short text. • Ability to design a project using a logical framework approach

50. The Logical Framework Matrix is a useful tool to systematize the hypotheses implicit in the project. It can be used by individuals or project teams, and often teams design their projects first and then retrofit them into the Logical Framework Matrix. This approach, obviously, misses the most important benefits of the approach, the possibility to organize thinking and to facilitate communication and dialogue among a range of stakeholders about problems and options for intervention.