Moving to Opportunity TOWN HALL Abt Associates, June 26, 2003 Background and Challenges: Judie Feins Findings: Robin

1. Abt Associates Inc. • Moving to Opportunity • TOWN HALL • Abt Associates, June 26, 2003 • Background and Challenges: Judie Feins • Findings: Robin Jacob • Implications: Larry Orr

2. WHY MTO? • Program Objective: Provide poor families living in high-poverty public or assisted housing with the opportunity to move to low-poverty neighborhoods with a Section 8 rent subsidy and counseling. • Research Objective: Measure the impact of moving to lower poverty neighborhoods on the outcomes of adults and children.

3. Precursors of MTO • Long history of research showing harmful effects of bad neighborhoods • Chicago’s Gautreaux Program: court-ordered mobility of poor, African-American families seemed to show positive effects of suburban life—especially on the children • Trend of increasingly concentrated poverty in American cities 1980-1990 suggested growing harm, unless a remedy could be found to reverse these neighborhood effects.

4. The Demonstration Program • The MTO program ran from 1994 to1998 in five cities: Baltimore, Boston, Chicago, L.A. and N.Y. • Families with children were recruited from public and assisted housing in concentrated-poverty neighborhoods. Many of the housing developments were very distressed and dangerous places. • Any resident who wanted the chance to get a mobile housing subsidy and move out could sign up, as long as the family met Section 8 eligibility rules. • Some 5,300 families volunteered; 4,608 were found eligible.

5. The Research Design • For research purposes, families were randomly assigned into one of three groups: • The experimental group received special Section 8 vouchers that could be used only in census tracts with poverty rates below 10 percent. Nonprofit counseling agencies in each city helped the experimental group families to locate and lease suitable housing in low-poverty areas. • The Section 8 group received regular Section 8 vouchers, which could be used anywhere they found a suitable unit with a willing landlord and rent below the program cap. These families did not receive any mobility counseling. • The control group received no vouchers but continued to be eligible for project-based assistance.

6. CBS News Correspondent John Roberts. From Ghetto To White Picket FenceBALTIMORE 6/5/2000 11/17/2000 The Los Angeles Times Housing, Not School, Vouchers Are Best Remedy for Failing Schools - Jan 31, 1999, Larry Cuban A Social Experiment in Pulling Up Stakes; Aid: Does neighborhood affect economic and school success? Five cities relocate poor families to find out.- Sep 23, 1997; pg. 1, Larry Gordon A Fresh Start Housing: The Moving to Opportunity program will take families out of the projects to see if a new environment helps them succeed. Nov 8, 1994; pg. 1, Larry Gordon Washington Post " In Baltimore, Getting a Lease on Middle-Class Life. " May 10, 2000; pg. 1 Amy Goldstein. New York Times " Better Than a Voucher, a Ticket to Suburbia. " October 18, 2000. Richard Rothstein • Chicago Tribune • "Foes kill housing plan funds" - Dec 15, 1994; Laurie Abraham •  "Where Should Poor Families Live?" - Jul 23, 1994; Lori Montgomery •  "Hostility Toward Relocating the Poor is a Matter of Race" - Apr 27, 1994; Clarence Page

7. A Long and Challenging Effort

8. THE EARLY CHALLENGES Abt Associates began working on the MTO design with HUD in September 1993. Our initial job: • Convincing HUD to shift to a 3-group design • Developing uniform (but flexible) procedures • Managing implementation (started July 1994) • Surviving a political fire-storm • Quietly growing the demonstration • Finishing intake and lease-up (March 1999).

9. AND THEN… • Database construction: We designed and built sophisticated relational database, which continues to evolve; • Sample tracking: We kept up with 4,608 families (19,000+ persons) by active and passive means, despite high mobility rates and the disruption of public housing revitalization and demolition.

10. CHALLENGES OF THE INTERIM EVALUATION • Research team: Abt partnered with a group of academics, the Urban Institute, and two data subcontractors to win the interim evaluation contract in July 2000. • Broad scope: HUD’s interest extended well beyond the usual housing and neighborhood issues to education, health, delinquency, employment and earnings, income and self-sufficiency. • Complex data collection: The study required that several kinds of data be gathered from almost 11,000 sample members in a compressed period.

11. Evaluation Team • ANALYSTS FROM: • Abt Associates (design, implementation, data collection, impact analysis) • Urban Institute (qualitative analysis) • National Bureau of Economic Research (impact analysis) • Georgetown University (collection and analysis of crime data)

12. Interim Evaluation Funders • U.S. Department of Housing & Urban Development • Grants via National Bureau of Economic Research • National Institute for Child and Health Development (NICHD) • National Science Foundation (NSF) • National Institute of Mental Health (NIMH) • Robert Wood Johnson Foundation • Smith Richardson Foundation • Russell Sage Foundation • W.T. Grant Foundation • Spencer Foundation • MacArthur Foundation • Grants via Georgetown University • National Consortium on Violence Research (NSF) • Brookings Institution

13. Interim Evaluation Data Sources • Pre-Survey Qualitative Data • In-depth semi-structured interviews with adult heads of household and children informed survey design. • Quantitative Data • Structured surveys with adults, teens, and children • Achievement tests of teens and children • Measurement of adult blood pressure, child height and weight • Administrative data from state and local agencies (earnings, TANF, food stamps, arrest records)

14. Interim Evaluation Surveys • Full Sample (adults, youth, children) • Fielded from January through June 2002. • Achieved survey response rates of 80% of adults, 77% of children, 76% of youth. • 3-in-10 Sub-sample • Random draw late in June from non-complete cases. • Focused field resources on a sub-set of hard-to-find cases; included travel to remote sites. • Increased response rates considerably and reduced the risk of non-response bias.

15. Interim Evaluation Response Rates • High Effective Response Rates • Attained a weighted adult response rate of 90 percent. • Attained youth/child surveys and achievement test response rates of 86 to 90 percent. • Group Differences • Difference in response rates between random assignment groups was less than 1 percent • Very low risk of non-response bias between groups

16. Interim Evaluation Findings What did we learn?

17. Estimation Methods • All estimates regression-adjusted with standard set of covariates, including (where available) pre-RA value of outcome • ITT = “intention to treat” – impact on entire treatment group, including those who did not lease up • TOT = “treatment on treated” – impact on those who leased up only • Tests of significance at .05 level

18. Mobility Outcomes • Poverty rates of current locations are substantially reduced (entire experimental and Section 8 groups) • Fraction minority population in experimental group locations is reduced, but more than half moved to areas 80%+ minority • Almost half of experimental lease-ups were in tracts with increasing poverty from 1990 to 2000 • More than half of experimental group lease-ups moved again to somewhat higher poverty tracts

19. Impacts on Neighborhood Outcomes • Significant positive impacts for both experimental and Section 8 groups on: • Feeling safe in the neighborhood (day and night) • Police coming when called • All measures of neighborhood quality • Significant reductions for both experimental and Section 8 groups in: • Witnessing drug activity in the neighborhood • See public drinking, groups hanging out • Crime victimization over last six months

20. Impacts on Housing Outcomes • Significant increases for both experimental and Section 8 groups in: • Most measures of housing quality • Utility payment problems • Prevalence of housing assistance receipt • No significant impacts on current total housing cost

21. A Focus on Educational Outcomes The following factors were hypothesized to be mediators of educational outcomes: Community-Level • Quality of Schools • Community Norm and Values • Social and Physical Environment Student and Family-Level • Parent Attitudes and Behaviors • Student Attitudes and Behaviors

22. School Characteristics • Modest improvements for both groups • 70% of Experimental lease-ups remained in the same large urban school district

23. School Climate No significant effects on school climate

24. Other mediators of educational outcomes Community-Level Mediators • Significant impacts on social and physical environment • Some evidence of impact on community norms and values but not on peer role models • Some evidence of positive impact on economic opportunities but not on earnings or employment of sample adults. • Student- and Family-Level Mediators • No significant effects on parental monitoring • No significant effects on parental involvement in school • No significant effects on student school-related behaviors

25. Impacts on Education Outcomes • No significant impact on student achievement • No significant impacts on grades, coursework, special ed placement, graduate rates or college attendance.

26. Other Outcomes: Health • Adult Physical and Mental Health • Physical health & substance use: largely insignificant impacts • Mental Health: significant E-C impacts • Obesity: significant E-C impacts • Youth Physical and Mental Health • Physical health: no significant impacts • Mental health: improvement for girls

27. Other Outcomes: Youth Behavior • Youth Delinquency & Risky Behavior • Behavior Problems Index: significant increase in self-reported behavior problems among boys • Delinquency: no significant impact • Arrests & Risky Behavior: substantial gender differences • Employment and School Attendance • No significant effects for boys, but an increase in full-time school attendance for girls in the experimental group and a reduction in full-time employment for girls in Section 8

28. Other outcomes: Income & Earnings • Receipt of Public Assistance • No significant impacts on current receipt of public assistance, either for full sample or subgroups by ethnicity and barriers to employment • Household Income & Poverty Status • No significant impacts on income, poverty, food security and self-sufficiency

29. Implications for Policy

30. Does this mean there are no impacts on these outcomes? • On many outcomes, only fairly large impacts can be detected with confidence—e.g., to be 80% sure of detecting impacts as significant: • Adult earnings would have to be increased by about 40% in the experimental group, 30% in the Section 8 group • TANF benefits would have to be reduced by 50% • Youth asthma attacks would have to be reduced by 67% The fact that an impact estimate is not statistically significant does not mean there was no impact—it means we don’t know if there was an impact.

31. Policy Implications - Context • The worst concentrations of urban poverty are usually HUD-subsidized. • In US, high-poverty neighborhood almost always means high-crime neighborhood. • There are legitimate concerns about “relocating the ghetto”. These might lead to restricted mobility programs.

32. Will mobility programs work? • Can voluntary restricted mobility programs move extremely low-income public housing tenants to middle-class neighborhoods? • Yes—at least, for 48%. • However, geographic restrictions come at some cost to lease-up (60% of Section 8 comparison group moved) • Those who move are likely to follow the “path of least resistance.” In MTO lease-ups were predominantly in minority neighborhoods, where poverty rates trending up.

33. Who benefits, and for how long? • Those who move enjoy substantially better housing, safer neighborhoods, less stress, better mental health, and lower obesity. There is some indication that girls behavior and motivation to succeed improves. • These benefits are still significant 4-7 years after the initial move. • Benefits to the rest of society are less clear. There is some indication of lower rates of criminal behavior among girls, but higher rates of crime and delinquency among boys. There are no reductions in welfare costs.

34. What do these results imply about choice of policies for improving the lives of low-income families? • Are the problems of low-income families environmental, or due to attributes of the families themselves? • These results imply that a change in environment can improve the family’s sense of well-being directly, and have some salutary effects on the behavior of youth. • But physical health, educational performance and attainment, employment, earnings, and welfare dependence do not appear to be sensitive to residential environment, at least within 4-7 years. • To improve these outcomes within that time frame requires policies designed to deal directly with these specific problems—such as educational improvements, employment and training, or welfare-to-work programs.

35. Analysts: • Dr. Larry Orr, Abt Associates • Dr. Judith Feins, Abt Associates • Dr. Jeffrey Kling, Princeton University • Dr. Jens Ludwig, Georgetown University • Dr. Robin Tepper Jacob, Abt Associates • Dr. Barbara Goodson, Abt Associates • Dr. Lisa Sanbonmatsu, NBER • Dr. Lawrence Katz, Harvard University • Dr. Jeffrey Liebman, Harvard Univsersity • Dr. Erik Beecroft, Abt Associates • Dr. Rhiannon Patterson, Abt Associates • Dr. Alvaro Cortes, Abt Associates • Ms. Carissa Climaco, Abt Associates • Ms. Debi McInnis, Abt Associates • Mr. Robert Teitel, Abt Associates • Dr. Susan Popkin, Urban Institute