Logic

1. Logic Karen Baehler, VUW ANZSOG Conference on Project Management and Organisational Change 22 February 2006

2. 2 Purposes Essence and examples Uses in policy cycle Strengths and weaknesses Good and bad practice

3. 3 Core message is about intentions A logic model explains How the policy, program, or intervention is meant to work The policy�s essential theory of change If X, then Y If outputs, then outcomes The chain of intended effects

4. 4 Compare to a decision tree, but here we don�t have probabilities Note the necessary, but sufficient relationships. Note that �fairness� may also be a desired outcome from such a programme, parallel to tax credits for health insurance Note where evidence might slot in Compare to a decision tree, but here we don�t have probabilities Note the necessary, but sufficient relationships. Note that �fairness� may also be a desired outcome from such a programme, parallel to tax credits for health insurance Note where evidence might slot in

5. 5 Compare to a decision tree, but here we don�t have probabilities Note the necessary, but sufficient relationships. Note that �fairness� may also be a desired outcome from such a programme, parallel to tax credits for health insurance Note where evidence might slot in Compare to a decision tree, but here we don�t have probabilities Note the necessary, but sufficient relationships. Note that �fairness� may also be a desired outcome from such a programme, parallel to tax credits for health insurance Note where evidence might slot in

6. 6 Good practice: A few points of �grammar� Outcomes Are achieved states, not policies themselves, processes or activities They happen to someone or something OUTSIDE of government The following are not outcomes The regulation of economic activity is effective and low cost. Regional development is actively encouraged.

7. 7

8. 8 Bad practice:Shoring up current policy IVL should be used to challenge the status quo, not to rationalise it Antidote is to specify assumptions and risks This needs to be done in consultation with other

9. 9

10. 10 On complexity (an aside) Effects of government activity on the human & physical world are often very complex (though not necessarily large) Hard to measure and even harder to anticipate Intervening in �wicked� problems is full of uncertainty The world is not linear But, things that governments can do are limited (carrots, sticks, sermons) Logic of government intervention is usually simple and easy to state

11. 11 Bad practice: Heroic leaps in logic Social policy: These often occur near the top of an outcomes hierarchy. Uses other than the statement of intent Regulatory version of this? Example of top down: End outcome: Reduce welfare dependency Who is dependent? (problem focus) divorced v never married, out of wedlock teen births those with low skills What contributes to these characteristics? Why are certain ethnic groups overrepresented? Why are there gaps? End outcome: Resilience in children Factors that contribute Mentors Psychological characteristics/protective factors Genetics Economic growth What makes an economy like NZ�s grow? Where in your dept�s generic hierarchy do you need to focus more attention? Where are the big unknowns? This is the place for your research agenda. And it�s probably not at the management levels where you have more control. The higher up the chain, the more external factors there are, the more research you need. Uses other than the statement of intent Regulatory version of this? Example of top down: End outcome: Reduce welfare dependency Who is dependent? (problem focus) divorced v never married, out of wedlock teen births those with low skills What contributes to these characteristics? Why are certain ethnic groups overrepresented? Why are there gaps? End outcome: Resilience in children Factors that contribute Mentors Psychological characteristics/protective factors Genetics Economic growth What makes an economy like NZ�s grow? Where in your dept�s generic hierarchy do you need to focus more attention? Where are the big unknowns? This is the place for your research agenda. And it�s probably not at the management levels where you have more control. The higher up the chain, the more external factors there are, the more research you need.

12. 12

13. 13 Antidote Don�t be afraid to include the obvious Get someone outside of your policy area to review the logic, to point out gaps

14. The Matrix (Funnell 1997) Just one among many soft systems approaches. Success criteria: Measuring outcomes What units? Targets? Huge step toward cost effectiveness analysis Factors in control (assumptions) Fewer as you move up & some will be in dispute Factors outside control (assumptions) Not necessarily risks, but potential risks; Can you assign probabilities to them? Are any of them in another dept�s control? Another institutions? Helps us pick collaborators Activ and Resources Should cover as much of column 3 as possible Intermediate outcome + columns 4 & 5 = next intermediate outcome (supply what is needed to be sufficient) Can include costings here, thus enabling cost effectiveness analysis Performance Track how you�re doing against columns 5 and 2 Can you write a contract? Which column could you use to deter creaming or cream-skimming of clients?Just one among many soft systems approaches. Success criteria: Measuring outcomes What units? Targets? Huge step toward cost effectiveness analysis Factors in control (assumptions) Fewer as you move up & some will be in dispute Factors outside control (assumptions) Not necessarily risks, but potential risks; Can you assign probabilities to them? Are any of them in another dept�s control? Another institutions? Helps us pick collaborators Activ and Resources Should cover as much of column 3 as possible Intermediate outcome + columns 4 & 5 = next intermediate outcome (supply what is needed to be sufficient) Can include costings here, thus enabling cost effectiveness analysis Performance Track how you�re doing against columns 5 and 2 Can you write a contract? Which column could you use to deter creaming or cream-skimming of clients?

15. 15 Use of logic models in New Zealand Personal preferences of analysts Policy unit good practice Links to operations are still weak Links to evaluation stronger Obstacles Official Information Act Treatment of skeptics Central agency positions

16. 16 Top-down logic models Central agencies� guidance to departments for Statements of Intent Prescriptive v softer approach Lessons learned Balancing control v influence Backlash is always a risk Impatience (hothouse) Need for options and recognition of variability

17. 17 Good practice: Thinking fresh about paths to outcomes Toward systems models of your patch Don�t try to make everything connect

18. 18

19. 19

20. 20 Bad practice: Unreadable diagrams (�copulating spiders�) These may be useful internally But simplify the diagram before you use it to engage Ministers, other departments, stakeholders, etc

21. 21

22. 22 Implementation matters When government intervention fails, it usually does so because of Bad policy choice, or Good policy, poorly implemented Logic models are meant to help prevent these types of failure by Scrutinising policy choice early Aligning implementation plans with policy intentions Organising and promoting learning Can it work? Two sources of accountabilityTwo sources of accountability