2010 National Evaluation Institute Williamsburg, Virginia October 8, 2010 Diana Bowman

1. Evaluating McKinney-Vento Programs: How Do We Support Children and Youth Experiencing Homelessness 2010 National Evaluation Institute Williamsburg, Virginia October 8, 2010 Diana Bowman Beth Garriss-Hardy Patricia A. Popp

2. How many children experience homelessness? • 10% of children living in poverty • Urban Institute • 1 in 50 children experience homelessness • National Center on Family Homelessness • Over 40% of children who are homeless are under age 5 • Urban Institute • 956,914 enrolled in school 2008-2009 (41% increase over two years) • US Department of Education

3. Primary Nighttime Residence Virginia 2008-09

4. McKinney-Vento Homeless Assistance Act – EHCY Program • Reauthorized in 2001 as Title X, Part C of ESEA • Provides for educational rights and services for children and youth experiencing homelessness • Main themes • School access • School stability • Support for academic success • Child-centered best interest decision making • Local homeless liaison • (State Coordinator)

5. Educational Barriers • Enrollment requirements (school records, immunizations, proof of residence and guardianship) • Prejudice and misunderstanding • Stress, hunger, fatigue • Lack of: • School stability and educational continuity • Access to programs • Transportation • School supplies, clothing, etc.

6. Research and Evaluation Questions • What is the impact of school stability and school mobility on students? • How effective are academic interventions on student achievement? • What other interventions impact the achievement of homeless students? • What is the lasting impact of homelessness on students?

7. What are the challenges to research and evaluation? • Homeless students are not homogenous • High mobility interferes with longitudinal tracking • Limitations in sample size (accessibility of common data) • Impact of homelessness lasts beyond time when homelessness occurs

8. Challenges (continued) • Random sampling not possible • Impacts on achievement • In school and out of school – hard to determine program impact • No consensus on what should be measured • Lack of standard definitions

9. Challenges • Potential systematic nature of incomplete or missing data (selection and sorting biases) • Results cannot be stated in terms of causality (however positive relationships can be noted) • Results cannot be generalized

10. Case Study: Has keeping homeless students in their school of origin improved their academic outcomes relative to those homeless students that changed schools? • Substantial literature of adverse impacts school mobility has on student outcomes • Literature on homeless students is scant • SPPS requested study of 2006-2007 school district data for homeless students • Reviewed homeless student data for recorded school moves for three fields: last school attended, school while in shelter, and school after shelter

11. Impact of School Moves Analysis of Data

12. Data challenges • Most students had missing values for school after shelter – primarily used last school attended and school attended while in shelter to identify “movers” • Data did not capture students moving back and forth between schools multiple times • Difficult to distinguish between actual moves and duplicate records for students at various schools • Wide variation in student proficiency and quality of schools moved to or from • Reviewed records for 1,343 homeless students; only 306 were complete – sample sizes across grade levels small; most in early elementary grades not tested

13. Overall Finding Given the data limitations, no discernable pattern of impact from homeless student moves can be detected, whether negative, neutral, or positive on SPPS homeless student state achievement test scores.

14. Similar Study in 2004 • A similar (unpublished) study conducted in 2004 addressed the relationship between school stability and school performance. • Significant challenges related to availability, accessibility and utility of data • Lack of common data collection and storage capacity and mechanisms across school districts and states reduced the potential pool of participating school districts to a sample size judged insufficient to support conclusions and generalization.

15. Questions • Do homeless students who remain in one school during a given academic year show more academic growth that do homeless students who attend two or more schools during the academic year? • Do homeless students who remain in one school during a given academic year attend school more regularly than those who attend two or more schools during the school year? • Do homeless students who remain in one school during a given academic year have fewer discipline referrals and/or school suspensions?

16. BUT…….. • Possibility of credible research in homeless education; should be encouraged. • Benefits of strong, well-designed descriptive studies, with well-defined components and procedures. • Strong program evaluation efforts, based on standards and indicators, enhances the body of evidence.

17. Available Tools • McKinney-Vento Data Standards and Indicators of Quality Programs (2006 revisions) • Educating Homeless Children and Youth: Conducting Needs Assessments and Evaluating Services – A Guide for SEAs, LEAs, and Local Schools • Available at the National Center for Homeless Education (NCHE) • www.serve.org/nche

18. Contact Information • Diana Bowman: dbowman@serve.org • Beth Garriss-Hardy: drbethg@aol.com • Patricia A. Popp: pxpopp@wm.edu