14 July 2008 - SPA508 Policy Analysis & Design

1. 14 July 2008 - SPA508 Policy Analysis & Design Fleshing out problem definition From problem definition to a report – what’s ahead Quantitative Evidence Sources Indicators of quality

2. Help me know you On card put Name Name you wish to be called Where you are working (if working) Undergraduate Major

3. Public Problem Definitions • Let’s consider a policy issue– should math & science be taught in English or in Bahasa? • Why is math & science a particular concern? • What are the benefits/drawbacks of having math & science taught in English? • What are the benefits/drawbacks of having math & science taught in Bahasa?

4. A possible problem definition • To meet 21st century demands Malaysian students need to be more competent in math & science; they will be more competent if these subjects are taught in the appropriate language • Alternatively lack of competence may be due to • Method of instruction, drill vs problem solving • Quality of instructors or materials • Availability of capable instructors • Access to needed technology

5. Points to remember • Policy analysis alone will not solve policy disagreements • Supporters & opponents of a problem definition can use policy analysis findings to support their position • Policy analysts work for a client. Thus they limited by • The client’s problem definition • The client’s time frame

6. From Problem Definition to a Report: Overview of Analysis I • Develop evidence to support and refine problem definition (14 July) • Select policy instruments • Market vs Government (Week 3) • Possible policy instruments (Week 4) • Mandate all students be taught in given language? • Create inducements to encourage use of given language? • Provide information on resources? • Keep the status quo?

7. From Problem Definition to a Report: Overview of Analysis II • Select criteria (Week 5) • What do we want to achieve with the policy? • How do we know we achieved our objectives? • What do we mean by competence? How will measure it? Will efficiency? Equality? Or another value be important? • Conduct analysis • Cost-benefit analysis (Weeks 6 & 7) • How much will our policy cost? • What benefits do we expect? • Program evaluation (Week 11) • Was the policy implemented? • What did it achieve? • What factors served as barriers/facilitators or led to success or failure? • Performance monitoring (Week 13) • How did the achievement of outputs vary over time?

8. From Problem Definition to a Report: Overview of Analysis III • Criteria for designing policies • Logic models (Week 8) • What resources & activities are needed for a change in language of instruction to work? • Evidence based policies (Week 10) • What does sound research tell us about the effectiveness of similar program & policies? • How useful are pilot programs to test a policy? • Designing a policy objective: a measurable objective that states how much will be achieved by when • Writing the report (Week 12) • Policy analysis & the budget process (Week 14)

9. Quantitative evidence to support a policy definition I • Identify & collect data and other information to spell out the extent and nature of the problem • What data exist to indicate Malaysian students’ competence in math & science? • Do the data track changes over time? • Do the data track differences by language of instruction? Differences between states? Other relevant differences?

10. Research evidence • Research on the role of language • What are the benefits of English? For whom? What are the drawbacks? • What are the benefits of Malaysian (or other mother tongue)? For whom? What are the drawbacks? • What language-related policies have been developed? What have they achieved?

11. Evidence of feasibility • Possible evidence • Capacity to implement policy • Cost of implementing policy • Political factors, e.g., agency receptivity • Applications to math & science policy • Number of qualified math & science teachers in English? Malaysian? Both • Availability of texts in English? Malaysian? • Evidence of the quality of the texts?

12. Summary:Types of evidence sought • To document nature of problem • That it exists • How large it is • Whom it effects • To support problem definition • To identify plausible policy tools • To demonstrate feasibility and costs of a policy solution

13. Finding the evidence • Existing data preferred to original data b/c of cost and time constraints • Governments & research organizations provide the strongest statistical data. Go to http://www.statistics.gov.my • Research literature may provide evidence to support problem definition, e.g., problem is worth addressing

14. Assessing the Quality of Quantitataive Evidence • Measurement – assigning values (numbers or categories) to a phenomenon • Phenomenon – Employee productivity • Measure – Hours worked per day • Possible values Number of hours worked in a day • Possible values; Did not work a full day, worked a full day, worked overtime • Two basic criteria to assess quality • Reliability – relatively free of random error • Validity – measuring what we intend to measure

15. Evidence of reliability • Survey or similar questions are unambiguous. Ambiguous questions - how many hours do you work? How much do you earn? • If more than one person is collecting information they should be consistent, e.g., do data collectors count the number of work hours the same (lunch counted by some and not others) • Most official statistics are reliable

16. Evidence of validity • Consider do the number of hours worked indicate employee productivity? • Let’s assume that number of hours worked is what we intend to determine • We could ask employees • We could have an observer at different times of the day • We could examine employee records • Important: Deciding that a measure is valid is a matter of judgment

17. Measuring Poverty • Malaysia’s Poverty Line Income based on RM necessary to “keep body & soul together” absolute poverty amount considers • Family size & location • Food– cost of maintaining a balanced diet • Non-food: cost of clothing, housing, transport, other items • Alternative measure, relative poverty – • How income matches with average income • Precludes participating in accepted daily life

18. Examining Tables • The following two tables are from the Straits Times (13/7/08) • Write a sentence to summarize what each table indicates? • What do they suggest about poverty in Malaysia? • What other information would you like? Why?

19. Being Poor in Malaysia, PLI

20. Who is poor in Malaysia

21. What numbers to collect • Amount – number of students who are not competent, number of employees who do not work a full day, number of people living in poverty • Percent or rate • Average (mean) – average test score, average family income • Note variations – test scores may be clustered close to middle; family income may be spread out

22. Exercise: Penang Bridge • List all the data that might be useful as evidence that there is a problem with the Penang Bridge. • Which data would make the most convincing argument?

23. What you should know by now • Problem definitions evolve • Evidence helps us better understand the problem, its dimensions, the feasibility of potential solutions • There is no one way to present quantitative evidence • Depends on your purpose • What is available

24. What you should know by now II • By examining information on how the phenomenon was measured you should be able to decide that it is • Reliable – relatively free of random error • Valid – that you are measuring what you say it is measuring

25. Class on 21/7 • Topic: Policy Solutions – Markets v Government • Read Deborah Stone, “The Market & the Polis,” Policy Paradox: The Art of Political Decision Making. Handout • What lessons do you think that Stone’s chapter offers to a policy analyst? • Look for market-related terms in this chapter • Remember Informant Interview

26. Looking ahead • Syllabus will be ready by 21/7 • On your own policy assignment – this will be developed in more detail no later than 28/7 • Pick a Malaysian problem (if from another country you can use that country) • Develop problem definition with evidence of its dimension • Select and justify a policy tool • Develop a preliminary strategy to evaluate the policy

27. On Homework • Papers are not late if turned in at the beginning of class (6:30 p.m.) they are due • Hard copy is preferred; electronic copy is acceptable. Final paper should be submitted in hard copy • First assignment will be graded leniently, please pay attention to comments

28. Written Assignment 21 July • Interview a policy analyst or consumer (referred to as an informant) • 5% of final grade • Name and brief job description on your informant (please attach card) • Brief summary of the informants training • How does the informant decide what policy problems (or issues) to work on? • If the informant is a policy analyst • What steps does s/he take to study a problem • What tools does s/he commonly use? • What tools or skills does s/he consider most valuable? • What advice would s/he give a student interest in a career as a public administration

29. 21 July Assignment (Cont’d) • If the informant is a policy consumer • How does s/he go about researching a problem? • What resources does s/he find most valuable? • What types of analysis or research presentations does s/he find most useful? Why? • What types of analysis or research presentations does s/he consider of little or no value? Why? What advice would s/he give a student interest in a career as a public administration