Interagency Council on Housing and Homelessness

1. Interagency Council on Housing and Homelessness Framework for Learning & Evaluation May 20, 2009

2. Meeting Agenda • Agenda • Welcome and Introduction (One Family) • Overview • Outcome Evaluation • Innovation Evaluation • Network Evaluation • Roles, Resources and Timelines • Next Steps Purpose: Agree on an evaluation & learning framework for ICHH Networks.

3. Steering Committee • Bob Pulster, ICHH • Sue Beaton, Fireman Foundation • Deborah Fung, Fireman Foundation • Dennis Culhane, University of Pennsylvania • Emily Cohen, One Family • Joe Finn, MHSA • Madeleine Taylor, Innovation Network for Communities • John Cleveland, Innovation Network for Communities

4. Context Overview

5. The Stakes Are High • The ICHH funding is one-time funding designed to push a paradigm shift in the state approach to homelessness. • Innovation in the field will drive future state policy around know best practice. • We need a rigorous evaluation and learning processes to make a convincing case for systems reform.

6. The Paradigm Shift Emerging Model Prevailing Model Employment Assistance Employment Assistance Day Care Day Care Shelter Housing Stabilization Prevention, diversion & rapid re-housing Shelter MH/SA Services Housing Placement Family Supt Services MH/SA Services Family Supt Services

7. Systems Transformation • From “the Continuum” to “the Network” • Creating a New Community of Practice: Housing Stabilization • New Service Priorities: • Diversion • Tenancy Sustainment • Service Coordinators • Benefits Counselors • Housing Relocation Specialists • Family Mediation • Home Visitors

8. ICHH Evaluation Mandates in 5 Yr Plan • Improve statewide data gathering systems. • Create an information management system to monitor performance and outcomes. • Assess progress on key benchmarks and use information to make system adjustments.

9. Evaluation Overview

10. Three Areas of Evaluation Purpose: To understand what works and doesn’t work to reduce individual and family homelessness, and align state policy with this knowledge. Areas of Learning Outcomes What were the targeted outcomes and to what extent were they achieved? Innovations What innovations were used, and how did they affect the outcomes? Networks What is the structure of the networks, and what effect did it have on program outcomes?

11. Three Areas of Learning

12. Three Processes

13. Measuring Outcome Data

14. Collecting Outcome Data • ICHH’s goal is to implement an improved state-wide HMIS system. • In implementing HMIS, regions can utilize existing platforms if they meet appropriate data exchange standards. • However, outcome data planning and collection cannot wait for the completion of the HMIS system. • All regions will be asked to collect “common data elements” to evaluate outcomes.

15. Common Data Elements • HUD has issued new HMIS standards for the Homelessness Prevention and Rapid Rehousing Program (HPRP) • DHCD will adopt these new data elements for HPRP providers and regional networks • New elements include 1) tracking HPRP eligibility, 2) services provided, and 3) outcomes • Client tracking and recertification for assistance until exit • Quarterly reports due to HUD and DHCD starting January 2010

16. Outcome Evaluation Resources • MHSA will work with each network on implementation of the “Home and Healthy for Good” evaluation methodology • Dennis Culhane will serve as an overall consulting resource to ICHH on outcomes data collection and analysis • One Family will provide TA to networks on evaluation of family outcomes

17. Home and Healthy for Good Methodology • Secure informed consent • Data collection through one-on-one interviews: • Entry Interview, Follow-Up Interviews, and Exit Interview • Entry of data by MHSA into secure databank • Analysis of data: • Costs • Medicaid • Quality of life • Sub-analysis • Comparison of data across networks

18. Assessing Innovations

19. Common Types of Innovations Early Warning Systems Prevention and Diversion Assessment and Intake Case Planning & Management Access to Housing Supportive Services

20. Common Innovations

21. Things To Learn About Innovations • What specific practices did the innovation consist of? • How did the practice vary across regions? Why? • How did the practice vary across populations? Why? • How did the innovation affect outcomes? • Were networks used in the innovation? If so, what kind? • How critical was the network design to the innovation success? • What are the “best practices” in each category? • What are the economics of each innovation? How much do they cost and what do they save? • What are the policy implications?

22. Innovation Evaluation Process • One Family and MHSA site visits to document actual innovation practices • Debrief with network staff on details of implementation: • What practices were actually implemented? • What made a difference in outcomes? • Development of best practice examples for peer learning sessions

23. Network Evaluation

24. Network Evaluation

25. Innovations and Networks

26. Different Types of Networks CONNECTIVITY Information ALIGNMENT Identity PRODUCTION Initiative Connects people to allow easy flow of and access to information and relationships Aligns people to develop and spread an identity and collective value proposition Individuals come to share a set of ideas, language, or standards • Fosters joint action for specialized outcomes by aligned people • Mobilization • Advocacy • Learning and dissemination • Delivering products and services Increased levels of complexity

27. Different Networks for Different Purposes • Example: Early Warning System • Connecting – Shared alert system • Aligning – Common standards on risk factors • Producing – Joint interventions

28. Network Evaluation Process • Develop network baseline map • Material reviews • On-line survey • On-site interviews or focus groups • Develop electronic map • Develop baseline assessment • Develop network scorecard • Collect follow up data in 9 months and at completion

29. Network Evaluation Products • Graphic network maps that show baseline and evolving structure • Network scorecards • Case studies • Network evaluation for each region • Summary network evaluation

30. Peer Learning Sessions

31. Peer Learning Sessions • Quarterly meetings (Up to 6 over the 18 month time frame) • 1-4 people per region (up to 50 people total) • Half day; facilitated by the Innovation Network for Communities • Agenda will include: • Specific case studies on the focus topic • Presentation of outside points of view • Round-robin discussion on best practices • Identification of challenges and barriers • Policy options

32. Roles and Resources

33. Roles

34. Evaluation Plan Elements • Who is responsible for overall evaluation strategy. • Strategy for collecting and analyzing outcomes data and connecting with HMIS system. • Strategy for documenting and assessing specific innovation practices. • Strategy for network evaluation. • Evaluation timeline and budget.

35. Resources • Evaluation and learning resources being provided by the Fireman Foundation • $25,000 grants to each network to support data collection; participation in site visits; and participation in peer learning • One Family support provided free of charge • Separate TA and evaluation contracts • Some ICHH resources to support peer learning process

36. Timeline Elements

37. Summary of Key Points • Every region will need an evaluation plan. The plan will have three components: • Outcome evaluation and common data elements • Evaluation of innovations • Network evaluation • In addition to the development of the plan, participating in the evaluation process will involve: • Participating in interviews with evaluators • Responding to information requests (such as on-line surveys) • Analyzing root causes; participating in peer learning processes • ICHH has organized resources to support this process. • $25,000 grant to each network • Technical assistance resources

38. Additional Questions and Discussion