PATHFINDER

1. PATHFINDER Mission: Results for New Zealanders

2. Agenda for WG4 1. Introduction (Chair) - includes short website update (Greg) 2. WG / WS Process (Chair) - building effective relationships - ideas for improvements (human & output) - feedback on Q&A sessions 3. New WG Products (Roger / Greg) - 10 early step performance cycle (StW 1-3) - Intervention Logic (one-off presentation) - Building Block / Learning Point documents 4. Other matters (Chair)

3. Focus on Outcomes

4. Pathfinder ‘Building Blocks’ • Define & measure ‘mission critical’ outcomes (using ‘state’ or ‘situation’ outcome indicators) • Map causal logic linking outcomes to outputs • Assess impact of interventions / targeting • Assess cost-effectiveness (vs. outcomes achieved) • Management to maximise outcomes (e.g. core/pilots) • Benchmarking with outcomes (business units / nations) • Focus strategic / annual plans on improving o/c • Redesign planning & operations to maximise o/c (incl. feedback & continuous improvement) • Improving outcomes across agencies / sectors Steps are not sequential – If feasible, do steps #6-9 early

5. Observation vs. Action Outcome Type Measure Applications Status quo outcome (client /service groups) Situation Report Benchmarking Crude targeting Risk, Need & Likely Change (individuals / cases) Priority setting Demand analysis Risk managemnt Improvement in outcome (‘treated’ groups) Continue? Modify? Grow? Kill?

6. Improving Results – 10 Step Performance Cycle Identify Core Outcomes Define in Measurable Terms Measure as State Indicators Identify Areas for Change in State Define / Refine Intervention Logic Identify Intervention Options Prioritise Intervention Options Design & Deliver Interventions Impact (o/c) Evaluation Framework Cost Effectiveness Analysis

7. Intervention Logic Decision: Sufficient interest to arrange half-day session? • Aligning activities / outputs to outcomes • Defining a robust Intervention Logic (IL) • Using IL 'backbones' as a top-down (strategic) tool to scope intervention options • Testing intervention logic is robust, and produces best approach (ex ante, ex post) • Strengthening intervention logic using evidence-based approaches (R&D, eval) • Continuous evolution and testing of intervention logic / frameworks

8. Documenting Learning Points Decision: Approval of principles / generic process; First Priciple … Keep it short(ish), keep it pithy; • What’s best for NZ as generic guidance • Want strong support of documents (-> SG) • (Recognise there is no ‘one-size-fits-all’) • Seek consensus on any disputed points • Initial comments via e-mail • ‘Living document’ for discussion at WG • Send to SG (discussion / approval / -> web site)

9. Documenting Learning (Decision) Decision: Approval of format below as start point for writing; History: WG2 agreed focus on StW 1-3; no feedback on format; Proposal: ‘Trial by fire’ of the following format … • Executive summary • Purpose statement • Generic framework (10 step process) • Attributes of good outcome frameworks • Methods / steps / issues • Management and ownership • Best practice / good practice (integrated) • Worked examples (refer to supplementary docs?)

10. Documenting Learning (Contd.) Attributes of good outcome frameworks, e.g. … • Keep it tight (one or few outcomes, clear priorities for activities) • Researched (outcomes derived from institutional knowledge and, where feasible, comparison with ‘similar / like’ agencies) • Clear causal links (outcome-output-input links clear; other means of achieving outcome(s) visible; clear fit to your services / clients; covers significant activities of your / relevant agencies) • Measures (objective / quantitative as possible; non-measurables identified; built on robust data; confidence intervals known; directly attributable to agency activities; can ‘drill down’ for underlying trends; ‘impact’ and ‘state’ outcomes; decision-making / quality outcomes) • Clear fit w. prioritisation / management decisions (e.g. measures inform key mgmt decisions, fit resource information) • Clear fit with vision / purpose (e.g. strategy / legislation) • Known relation to outcomes of other agencies

11. Documenting Learning (Contd.) Generic processes, e.g. Building Block #1… • Identify Core Outcome(s) (e.g. from mission / purpose identify key outcomes; compare to o/c / missions used by similar agencies overseas; ensure good fit w. results you expect from services you deliver) • Define in Measurable Terms (translate verbal statement of o/c into precise definition that matches data available at reasonable cost; ID key demographic / service / etc groups for which o/c are required) • Measure State / Environment Indicators (document system; gather / process data; measure o/c for all groups w. confidence intervals) • Identify Desired Change in State (analyse state indicators for statistically significant ‘highs and lows’; identify improvements sought; identify where results are good / some resources could be shifted) • Build Results into Business (e.g. report internally and externally; input into strategic plan, SoI, broad priority setting, KPI, benchmarking, etc) • Confirm Fit w. Outcome Management Framework (e.g. how can results be used to benchmark, strategise, prioritise, etc; how do measures have to change or be analysed to perform different functions)

12. Documenting Learning (Decision) Decision: Approval of format below as start point for writing; History: WG2 agreed focus on StW 1-3; no feedback on format; Proposal: ‘Trial by fire’ of the following format … • Executive summary • Purpose statement • Generic framework (10 step process) • Attributes of good outcome frameworks • Methods / steps / issues • Management and ownership • Best practice / good practice (integrated) • Worked examples (refer to supplementary docs?)

13. Agenda for WG4 1. Introduction (Chair) - includes short website update (Greg) 2. WG / WS Process (Chair) - building effective relationships - ideas for improvements (human & output) - feedback on Q&A sessions 3. New WG Products (Roger / Greg) - 10 early step performance cycle (StW 1-3) - Intervention Logic (one-off presentation) - Building Block / Learning Point documents 4. Other matters (Chair)