Public Policy Analysis and Evaluation

1. Public Policy Analysis and Evaluation

2. Empirical and normative approaches to policy analysis • All approaches to policy analysis – seek to explain why certain policies are established, and the consequences they bring about: • Empirical approaches – draw principally from behavioral social sciences to explain why things occur and the significance we should attach to them. • Normative approaches – derived principally from analytical philosophy, try – to establish what makes a policy approach “good” or “bad,” and/or right or virtuous. • Approaches are usually employed in tandem!

3. Some major empirical approaches and their significance • Rational economic models – 18th Century (J.S. Mill, J. Bentham). Public policies are based on articulation of cumulative individual choices – from a normative standpoint, this approach serve as a justification (to some) of the efficacy of utilitarianism. • Public choice approaches – a modern variant of rational choice based on insights from political science. Assumes people act rationally, but do so within the context of groups, not as individuals. They join coalitions of like minded individuals in order to enhance both choices, and their power in obtaining them (motives may vary – from egoism/hedonism to justice). • State capacity approach– preferred policies and their achievement depend, to a large extent, on the ability of a polity to amass the resources, and administrative competence, to develop a set of policies over some social concern and then carry them out (David Apter, Samuel Huntington, Theda Skocpol).

4. Variants of public choice • Institutional rational choice – imperfect information, organizational impediments, and limits of our cognition constrain what we are able to achieve. Thus, everything we do is an effort to optimize or “satisfice” – achieving the best solutions given limits of time, money, other resources (e.g., Herbert Simon). • Common experience within public sector organizations: regulatory agencies, planning agencies, business corporations. • Advocacy coalition model – often, we find that practically speaking, we form coalitions not based on “like-minded” interest, but to thwart or counter-act a common opponent, and because we share a few practical aspirations in common: • Common experience in legislatures and citizen-action movements: taxpayer groups and environmental interests joining together to oppose dams, nuclear power plants, or subsidies for large energy companies) - to Paul Sabatier/Hank Jenkins-Smith).

5. Lessons from state capacity approaches • The more highly-developed a polity, the stronger and more adaptable tend to be the institutions charged with making & implementing policy – and more broadly-based and – usually – flexible, are policy outcomes: e.g., • To have a national health care system, you must first be sufficiently developed enough to have medical personnel who are highly-trained and equipped, and whose work can penetrate society and deliver basic needs. • Without these, you cannot organize and carry-out national programs for immunization, pre-natal care, primary health care delivery, etc. – numerous exponents.

6. Normative approaches to policy analysis • Long-standing debate regarding objective status of norms in public policy:  • Cognitivism values have a clear, knowable, defensible foundation & are universal. • Distributive justice – Society is obligated to equalize or make widely-distributed opportunities for everyone through politics: Plato, Aristotle, Montesquieu, J. Rawls. • Utilitarianism – whatever accrues to the greatest pleasure for the greatest number is good policy: J. Bentham, J.S. Mill. • Personalism or natural law approach – human needs, and the development of the whole person should be goal of policy: Plato, Aquinas, social psychologists such as A. Maslow and L. Kohlberg • Deontic views of justice– treat others as we would have them treat us; care for others as we would care for ourselves & put yourself in another’s place in making policy: I. Kant – other advocates of the “Golden Rule.” • Divine revelation/divine law – e.g., religion/so-called “eternal law” should be basis for policy (common theme in Judaism, Christianity, Islam, Buddhism Hinduism, etc.)

7. Normative approaches (con.) • Non-cognitivism: values are merely preferences, “social codes” based on convention or self-interest, not universal principles. • Ethical relativism – “what’s right for you is right for you.” Ethical principles that form basis of policy are culturally-shaped and not universal (various thinkers). • Egoism/hedonism – prudential self-interest based on maximizing pleasure is the only value public policy should pursue: T. Hobbes, B.F. Skinner. • Realism– politics and policy are the triumph of power, not justice. “The ends justify the means as viewed by the Prince:” N. Machiavelli. • Social Darwinism –fierce competition for survival produces social progress and prosperity; government’s role should be minimal (“law of the jungle” – e.g., Herbert Spencer).

8. Applying normative approaches Example – President Obama’s student financial aid policy proposal – end federal guaranteed loan program; students would borrow directly from federal government; making it easier and cheaper to borrow for college. Pell Grant maximum amounts would be indexed to inflation. Distributive justice- low interest government loans will equalize access by removing financial barriers – this benefits society by producing more college graduates. Personalism – the less the economic burden to get an education, the greater the full-development of human capital. Utilitarianism – the greater the number of college students educated, at lower economic cost, the greater the net benefit to society. Social Darwinism – life isn’t fair; some have to work and compete harder to get an education. Those who struggle on their own and succeed are better for it, and so is society. Relativism – if you want to go to college, that’s your choice. Some choose not to because they don’t value higher ed. Neither is a social benefit, merely an individual choice. Government should not promote one option over the other.

9. What is a public policy? • A course of action followed by a government in dealing with some problem or matter of concern. • Examples? • Problems of religious nature, economic nature, health, environment, education, political, safety, security, social, judicial, ruling….

10. What is public policy evaluation? • As much an art as science • Estimation • Appraisal • Assessment of a • Policy • Its content • Implementation • Goal attainment and other effects

11. Policy Impact • Policy Outputs • Things actually done by agencies in pursuance of policy decisions and statements • Eg. Amount of taxes collected, miles of highways built, welfare benefits paid, price-fixing agreements prosecuted, traffic fines collected, foreign-aid projects undertaken • Outputs can be readily counted, totaled and statistically analyzed

12. Limitations of Policy Outputs • ‘bean counting’ • Pressure on agencies from… • Legislators • Interest groups • To demonstrate results • Eg. HEC policies -> increased no. of publications (Quality??)

13. Policy Impact • Policy Outcomes • Also called policy ‘results’ • Consequences for society • Intended or unintended • Stemming from deliberate government action or inaction • E.g. Social welfare policies (BISP)

14. Before evaluation we need to • Define policy target population • Spillover effects or externalities on other groups than target population • Costs • Opportunity costs

15. Public Policy Evaluation Types • Impressionistic or intuitive • Based on anecdotal or fragmentary evidence • Influenced by ideological or partisan valuational criteria • Process • Operation • Administration

16. Public Policy Evaluation Types cont. • Systematic • Social science methodology • Specification of goals • Collection of information and data • Inputs, outputs and consequences • Rigorous analysis using quantitative or statistical techniques

17. Evaluation Designs • Experimental design • Classic method • 2 comparable groups • Experimental or treatment group • Control group • Randomly selected from target population • Pretests and posttests of both groups

18. Evaluation Designs cont. • Quasi experimental design • Pure experimental design not used due to cost, time and ethical considerations • Process of random selection not used • Treatment group compared with another group similar in many respects • Consequently, greater likelihood of being influenced by internal characteristics rather than treatment!

19. Fundamentals of Decision Making Terminology • Terms: • Alternative – a course of action or strategy that may be chosen by the decision maker • State of nature – an occurrence or a situation over which the decision maker has little or no control

20. The decision maker faces conditions of: Certainty Risk Uncertainty Level of ambiguity and chances of making a bad decision Lower Moderate Higher A View of Decision-Making Conditions

21. Decision-Making Conditions State of certainty: • A condition in which the decision maker knows with reasonable certainty what the alternatives are and what conditions are associated with each alternative. State of risk: • A condition in which the availability of each alternative and its potential payoffs and costs are all associated with probability estimates.

22. State of Uncertainty • A condition in which the decision maker does not know all the alternatives, the risks associated with each, or the likely consequences of each alternative.

23. Decision-Making Conditions • Certainty • An ideal situation in which a manager can make an accurate decision because the outcome of every alternative choice is known. • Risk • A situation in which the manager is able to estimate the likelihood (probability) of outcomes that result from the choice of particular alternatives.

24. Decision-Making Conditions • Uncertainty • Limited or no information prevents estimation of outcome probabilities for alternatives associated with the problem and may force managers to rely on intuition, hunches, and “gut feelings”.

25. Decision-Making Environments • Decision making under certainty • State of nature is known • Decision making under risk • Several states of nature may occur • Each has a probability of occurring • Decision making under uncertainty • Complete uncertainty as to which state of nature may occur

26. Decision MakingCertainty • Linear ProgrammingOne of best known tools of Management Science • Used to determine optimal allocation of an organization’s limited resources

27. Linear Programming • State the problem • Decision Variables • Independent variables • Objective function • Desired benefit (profit) as mathematical function • Constraints • Limits, expressed as equations

28. Example

31. Decision Making Under Risk • Expected Value • Decision Trees • Queuing • Simulation

32. Decision Making with Risk • Expected Value Approach • If probabilistic information regarding the states of nature is available, one may use the expected Monetary value (EMV) approach also known as Expected Value (EV). • Here the expected return for each decision is calculated by summing the products of the payoff under each state of nature and the probability of the respective state of nature occurring. • The decision yielding the best expected return is chosen.

33. Expected Value of a Decision Alternative • The expected value of a decision alternative is the sum of weighted payoffs for the decision alternative. • The expected value (EV) of decision alternative di is defined as:

34. where: N = the number of states of nature pj = the probability of state of nature Oij = the payoff corresponding to decision alternative and state of nature pj

35. Example

36. Example 1

37. Example 2

38. Decision Trees

39. Queuing Theory • The essence of a typical QT problem • Identifying the optimum number of servers to reduce overall cost • Most of the cases the serving facility (e.g. airport) is not paying for the time in the queue (airline) but wishes to avoid disgruntled customers

40. Queuing Theory

41. Simulations • Live • Real people, equipment, simulated environment • E.g. Military exercises • Virtual • Real people, simulated equipment • Flight or driving simulators • Constructive • Simulated people, simulated equipment • Real people can stimulate (provide input) • Model of factory production layout

42. Decision Making under Uncertainty • If the decision maker does not know with certainty which state of nature will occur, then: • the optimistic approach (Maximax) • the conservative approach (Maximin) • the minimax regret approach (Minimax regret) • Equally likely (Laplace criterion) • Criterion of realism with  (Hurwicz criterion)

43. Optimistic Approach (Maximax: maximum of the maximum) • The optimistic approach would be used by an optimistic decision maker. • The decision with the largest possible payoff is chosen. • If the payoff table was in terms of costs, the decision with the lowest cost would be chosen.

46. Conservative Approach (Maximin: maximum of minimum) • The conservative approach would be used by a conservative decision maker. • For each decision the minimum payoff is listed and then the decision corresponding to the maximum of these minimum payoffs is selected. (Hence, the minimum possible payoff is maximized.)

47. Regret Approach (Minimax) • The minimax regret approach requires the construction of a regret table or an opportunity loss table. • This is done by calculating for each state of nature the difference between each payoff and the largest payoff for that state of nature. • Then, using this regret table, the maximum regret for each possible decision is listed. • The decision chosen is the one corresponding to the minimum of the maximum regrets.

49. Criterion of Realizm (Hurwicz) • Often called weighted average, the criterion of realism (or Hurwicz) decision criterion is a compromise between optimistic and a pessimistic decision. • First, select coefficient of realism, a, with a value between 0 and 1. When a is close to 1, decision maker is optimistic about future, and when a is close to 0, decision maker is pessimistic about future. • Payoff = [a x (best outcome) + (1-a) x (worst outcome)]

50. Equally Likely (Laplace) Criterion Equally likely, also called Laplace, criterion finds decision alternative with highest average payoff. • First calculate average payoff for every alternative. • Then pick alternative with maximum average payoff.