Rethinking Homelessness

1. Their Future Depends on it! RethinkingHomelessness

2. Program Evaluation Homeless Education State Coordinators Meeting February 3, 2009 Key Bridge Marriott Arlington, VA

3. Program Evaluation • “Program Evaluation” and • “Performance Evaluation” • are comprehensive • terms encompassing two • critical components: • Project Design • Project Evaluation

4. Project Design - Outline • Review performance evaluation purposes and processes • Create sample outcomes and activities with criteria and measures • Assess/create your outcome measures with SMART processes

5. Performance Evaluation Performance measurement: a processthat systematically evaluates whether your efforts are making an impact/change on/to the clients you are serving or the problem you are targeting.

6. Why should programs be interested in performance measurement? We are all in the business of helping people; we need to… • understand whether current activities are working to achieve intended results.

7. Why should programs be interested in performance measurement? We are all in the business of helping people; we need to… • understand whether current activities are working to achieve intended results. • drive program improvement and share information on effective practices with others.

8. Why should programs be interested in performance measurement? We are all in the business of helping people; we need to… • understand whether current activities are working to achieve intended results. • drive program improvement and share information on effective practices with others. • acknowledge that high-performing programs receive credibility/funding especially in competitiveprocesses.

9. Creating SMART Objectives With Measurable Outcomes S pecific M easurable A ctivity-oriented R ealistic/feasible T ime-oriented

10. Building Blocks of Performance Measurement • Inputs • Activities • Outputs • Outcomes

11. Building Blocks of Performance Measurement • Inputsinclude resources dedicated to, or consumed by the program—e.g., money, staff and staff time, volunteers and volunteer time, facilities, equipment and supplies.

12. Building Blocks of Performance Measurement • Activitiesare what the program does with the inputs to fulfill its mission–e.g., providing school supplies or staff training, conducting a tutoring or summer program.

13. Building Blocks of Performance Measurement • Outputsare the direct products of program activities, usually represented in terms of the volume of work accomplished—e.g., number of students served and the number of staff trained—and are often confused for outcomes.

14. Building Blocks of Performance Measurement • Outcomesare benefits or changes among clients during or after participating in program activities. Outcomes relate to measurablechange in student knowledge, behavior, skills, conditions, or other attributes. Outcomes impact the target you are addressing, which for us in generally the kids we serve.

15. Building Blocks of Performance Measurement • Outcomes (continued)How does the activity that is engaged in impact on the target?

16. Performance Measurement Process Activities Training Tutoring Summer program Field trips School supplies Inputs Outputs Programs Infrastructure Staff Partners Supplies # staff trained on MV # of students in tutoring # of students in summer program # of students receiving school supplies Outcomes 30% more identified 40% increase TAKS 25% incrs reading retention 50%increased attendance Should we adjust how we spend our resources? How do we document our efforts? Should we add or change use of resources to expand our impact? What did our efforts achieve?

17. Outputs vs. Outcomes Be mindful to distinguishbetween outputs and outcomes. An outcome is: An output is: • Focused on what the student will gain/how will change • Focused on what the program will do (activity) to achieve the outcome. • A way to measure the student-level impact with clear targets and methods for measuring change. • A way to quantify the frequency and intensity of the activity. • Reasonably attributable (a result of) to an output or outputs • Specific to the activity described • Feasible and attainable. • Meaningful and attainable. If outcomes show the program works, then outputs are needed to understand how to replicate results

18. Outcome, Output or Neither? Answers Examples Output Output • School supplies distributed to 150 campuses • 90% of homeless students pass the state assessments • 75% of all LEAs’ staff will be MV trained • 40% of homeless students improve reading by one grade level • Homeless student attendance improves by 50% Outcome Outcome Outcome

19. Linking Outputs to Outcomes • Outputs Outcomes 100 % of students in homeless situations are enrolled immediately LEA staff statewide staff are MV trained 100 percent of homeless students receive school supplies and uniforms Homeless students’ attendance will increase by 50% statewide 75% of homeless students who attend after school tutoring will increase reading by one grade level 95% of homeless students in LEAs attend after school tutoring

20. Collecting the Necessary Data Can you collect thedata you need in order to proveyour outcome?

21. Collecting the Necessary Data • What data will you need to collect? • How will you collect it? • What system will you put in place to collect the data? • What criteria will tell you that you are on the right track? • When do you need to begin collecting data?

22. Budget Support and Alignment • Do large categories and line items support activities? • How will you know if your money is spent in the most effective way? • How will you stay on top of anticipated expenses? • What will you do to build a relationship with your business office? • Are there business office practices that make it difficult for you to work from current figures, or know what invoices are paid?

23. Project Design: Summary • Need (with baseline reference) • Objective(s) • Anticipated Successful Target Outcome(s) • Timeline(s) • Inputs • Activities • Anticipated Outputs

24. Project Design: Summary • See sample schedule for • converting this project • design into a planning and • implementation document.

25. Project Evaluation: Summary Objective(s) Actual Outcome compared to Anticipated Target Outcome Activity(ies) Criteria used to measure activities

26. Project Evaluation: Summary Actual Outputs compared to Anticipated Outputs Analyze Status(mid-year and end of the year) What worked? What didn’t? Changes What will you change to more closely align anticipated outcomes to actual outcomes – mid year or for next year?

27. Project Design and Evaluation: Developing M-V Outcomes Objective: What do you hope to accomplish? Outcome (measurable): How will you know you’ve accomplished it? What will tell you that you have been successful?

28. Project Design and Evaluation: Developing M-V Outcomes Activities: What activities will you put in place to accomplish your outcome? Criteria: What criteria will you look at to see if the activities are being completed?

29. Project Design and Evaluation: Developing M-V Outcomes Measure/Output: What will be the measure (output) of each criteria/activity to tell you that you that you are on track to meet your outcome?

30. Project Design and Evaluation: Developing M-V Outcomes • See sample Project Design • and Project Evaluation • handout developing a • statewide M-V outcome for • Florida.

31. Project Design and Evaluation: Group Activities • See sample M-V outcome • activity sheets, or develop • outcomes selected by the • group.

32. Special Thanks To: Brooke Spellman & Michelle Abbenante For sharing their knowledge and expertise regarding program evaluation, and for granting their permission for to use/revise much of their presentation delivered at the NAEH Conference on Ending Family Homelessness - February 8, 2008

33. Nothing will work if YOU don’t work!