Outcomes Based Management

1. Outcomes Based Management Barbara Montero, Vice President of the SARGE Affiliate Network Presented: June 27th, 2013 Renaissance Denver Hotel Denver, CO

2. Introductions & Background

3. Mission of Acelero Learning The mission of Acelero Learning is to bring a relentless focus on positive child and family outcomes to close the achievement gap and build a better future for children, families, and communities served by the Head Start program.

4. What is Acelero Learning? Acelero Learning is a proud provider of Head Start/Early Head Start and other comprehensive early child programs. We were founded in 2001 (by a former Head Start teacher & a former Children’s Defense Fund staffer) and we partner with local communities to manage the highest quality Head Start/Early Head Start programs. Today, we directly administer a network of Head Starts/Early Head Starts and other early childhood programs serving more than 4000 young children and families around the country. Our National Support Center (based in Harlem, NY), supports 3 local delegates in Monmouth/Middlesex, NJ; Camden, NJ/Philadelphia, PA; and Clark County, NV.

5. Acelero Learning Team Aaron Lieberman CEO • Former Head Start teacher, CDA holder • Founder and CEO of Jumpstart • Chairman of SchoolSuccess.net, designed and built first on-line assessment system for HighScope and Teaching Strategies Henry Wilde SVP of Operations • State of Wisconsin Deputy Secretary for Children and Families overseeing childcare under Governor Doyle • Former special assistant to Marian Wright Edelman at the Children’s Defense Fund • Architect of the NJ Abbott Pre-K program and evaluation system • Co-director of National Institute for Early Education Research • Former Head Start teacher, HighScope Training of Trainers co-designer Ellen Frede SVP of Education • Queens Borough Commissioner for Child Protection, NYC Administration for Children’s Services • Deputy Director, Free to Grow, National Program Office, Columbia University SPH, a 20-site national Head Start demonstration program Lori Levine SVP of HS Services Barbara Montero VPof SAN • Manages the T/TA network of Acelero partner programs/customers • Deputy Head Start Director for Clark County • (Las Vegas), NV

6. What is Acelero Learning Training and T/A? • In-person trainings • 1.5 to 2-day in-person trainings, typically sponsored in partnership with NHSA and/or Regional Head Start Associations • We have trained over 500 Head Start leaders over the last three years in using data to monitor and our outcomes-driven approach • SARGE • A comprehensive online library of service area plans, policies and tools to assist Head Start and Early Head Start programs in providing high-quality services and achieving Head Start compliance with huge time savings • Over 90 Head Start programs (serving nearly 80,000 children and families) are SARGE subscribers, and 80% renew year after year • Currently has an optional Re-competition Resources, and a new series of Transition Resources coming soon! • SARGE Affiliate Network • Provides intensive partnerships with a limited number of programs for extensive RFP assistance and ongoing support in PDM, ECE, and FCP • We currently work with 8 programs serving over 20,000 children each year

7. Introduction to Outcomes-Based Management in Head Start • Why now? Head Start is under attack. • 2010 House budget proposed a $1 billion cut • Sequestration caused a 5.7% cut for FY2013 funding beginning in March 2013 • Head Start Impact Study shows gains from Head Start are not as great as we would want them to be Clear, demonstrable outcomes are more important than ever before!

8. The decisions made in this room directly impact the lives of thousandsof children and families!*Concentrate * Focus * * Participate *

9. Agenda • Cornerstone #1 • A Look into Innovative Programs and their Outcomes • Cornerstone #2 • Using Data to Make Decisions – A Head Start Case Study • An Overview of Compliance-Based Monitoring • Breakout #1 • Moving Beyond Compliance into Outcomes-Based Monitoring (School Readiness & Parent, Family & Community Engagement Framework) • Breakout #2 • Strategy & Goal-Setting, Family Engagement

10. A Look at Innovative Programs and their Outcomes

11. ECE Innovations that Achieved Outcomes Older with Proven Long-Term Benefits • High/Scope Perry Preschool Project • Abecedarian Project • Chicago Child Parent Centers

12. High/Scope Perry Preschool What was innovative? • Four half days of center-based education, school year only • One 1.5 hour home visit per week • Extremely qualified staff • 1:6 ratio Cost: $11,300 per child per school year (in 2007 dollars) What were immediate effects on child outcomes? • .8 effect size on the PPVT

13. High/Scope Perry Preschool What were the long-term effects on child outcomes? Educational Effects Berrueta-Clement, J.R., Schweinhart, L.J., Barnett, W.S., Epstein, A.S., & Weikart, D.P. (1984). Changed lives: The effects of the Perry Preschool Program on youths through age 19. Ypsilanti, MI: High/Scope Press.

14. Economic Effects at 40 High/Scope Perry Preschool Schweinhart, L. J., Montie, J., Xiang, Z., Barnett, W. S., Belfield, C. R., & Nores, M. (2005). Lifetime effects: The High/Scope Perry Preschool study through age 40 (Monographs of the High/Scope Educational Research Foundation, 14). Ypsilanti, MI: High/Scope Educational Research Foundation.

15. Crime Effects at 40 High/Scope Perry Preschool Schweinhart, L. J., Montie, J., Xiang, Z., Barnett, W. S., Belfield, C. R., & Nores, M. (2005). Lifetime effects: The High/Scope Perry Preschool study through age 40 (Monographs of the High/Scope Educational Research Foundation, 14). Ypsilanti, MI: High/Scope Educational Research Foundation.

16. Abecedarian Project What was innovative? • Full-day, full-year center-based services for children • Served children between birth and five years old • Home visits • Low teacher ratio, 1:3 for infants and 1:6 for five year olds Cost: $17,099 per child in 2011 dollars What were the immediate effects on child outcomes? • .79 effect size on reading achievement with IQ • Campbell, F. A., Pungello, E. P., Miller-Johnson, S., Burchinal, M., & Ramey, C. T. (2001). The development of cognitive and academic abilities: Growth curves from an early childhood educational experiment. Developmental Psychology, 37(2), 231-242.

17. Abecedarian Project What were the long-term effects on child outcomes? Academic Benefits Barnett, W. S., & Masse, L. N. (2007). Early childhood program design and economic returns: Comparative benefit-cost analysis of the Abecedarian program and policy implications, Economics of Education Review, 26, 113-125; Campbell, F.A., Ramey, C.T., Pungello, E., Sparling, J., & Miller-Johnson, S. ( 2002). Early childhood education: Young adult outcomes from the Abecedarian Project. Applied Developmental Science, 6(1), 42-57.

18. Chicago Child Parent Centers What was innovative? • Half day center-based services school year only • 17:2 child to teacher ratio in preschool • ½ day a week of parent involvement, home visits Cost: $8,700 in 2007 dollars What were the immediate effects on child outcomes? • Age 5 Kindergarten Readiness at 47th Percentile (vs. 28th Percentile for comparison group)

19. Chicago Child Parent Centers What were the long-term effects on child outcomes? Academic and Social Benefits at School Exit Temple, J. A., & Reynolds, A. J. (2007). Benefits and costs of investments in preschool education: Evidence from the Child-Parent Centers and related programs. Economics of Education Review, 26(1), 126-144

20. Meta-Analysis of Pre-K Data What was it? • Meta-Analysis of the Effects of Early Education Interventions on Cognitive and Social Development • Gregory Camilli, Sadako Vargas, Sharon Ryan, & W. Steven Barnett • Looked at 123 different intervention studies over last 50 years to synthesize their outcomes • Goal was to determine the potential benefits and costs of implementing specific preschool programs

21. Meta-Analysis:What Determines Cognitive Gains? n= 123 Studies Barnett, W. S., (Nov. 13, 2012). Investing in Early childhood Education and Care (ECEC)

22. Meta-Analysis: What Produces The Most Significant Gains? Barnett, W. S., (Nov. 13, 2012). Investing in Early childhood Education and Care (ECEC)

23. So what programs have weak or inconclusive results? • Multi-generation and home-visiting programs • Even Start unless combined with high quality preschool • Home Based • Parent Child Development Centers • K-3 interventions • Follow Through • Project Developmental Continuity • Head Start Transition Study • One possible conclusion: education has the biggest impact on education especially when added to no education

24. Current Innovations that are Demonstrating Outcomes • Educare • AppleTree • Acelero Learning • Who Else?

25. Educare What is innovative? • Full-day, full-year birth-to-five education services • Maintain small class size and high staff/child ratios • Infant-toddler - 3:8 per classroom • Preschool rooms - 3:17 per classroom • Maintain high staff qualifications, Master Teachers with advanced degrees in early childhood supervise 3 or 4 classrooms • Provide continuity of care, primary teachers remain with children for 3 years, from birth to 3 years old • Second team remains with children from ages 3 to 5 Cost: $18,000 to $20,000 per year

26. Educare What were the immediate effects on child outcomes? • Average exiting score of 95 points on the PPVT for non-ELL children (82.5 for DLL) • Children had a higher PPVT score the longer the more years they stayed in Educare

27. Educare Educare Implementation Study Findings – August 2012. “Results.” http://www.educareschools.org/about/pdfs/Demonstrating-Results.pdf

28. AppleTree What is innovative? • Lead teachers with a Bachelor’s degree, three teachers in every classroom • Low child-to-teacher ratios between 5:1 and 7:1 • Full-day program, school year only • Direct assessment progress monitoring every 6 weeks • Weekly coaching for teachers What were the immediate effects on child outcomes? • Average of 99 on the PPVT for 3 year olds with 1 year • Average of 93 on the PPVT for 4 year olds with 1 year • .53 effect size on the PPVT

29. AppleTree “The AppleTree Approach to School Readiness: A Case Study Using A Longitudinal Population-Referenced Evaluation Framework. “By Craig T. Ramey & Nancy Cromwell. August 2010. “Results.” http://www.appletreeinstitute.org/files/doc/AppleTreeApproach.pdf

30. Acelero Learning What is innovative? • Full-day, full-year • Highly-credentialed staff, 75% with Bachelor’s degrees • Extensive use of data • Weekly MBI report with 100 different summary indicators • Monthly MBI + report with 54 different indicators • Quarterly MBO report looking at outcomes in education, family services and health • An ECE focus on coaching, data, and curriculum • An FCE focus on giving families more information, and having a common approach to family life practice goals Cost: Head Start funding of $8250 per child

31. Acelero Learning Results Our Results: In our latest evaluation report, returning children enrolled for one full year in our Head Start programs achieve 3.5 times the average gains recorded on the 2009 Head Start FACES study on the Peabody Picture Vocabulary Test (PPVT).

32. So How Does It All Stack Up?

33. Innovations Within Your Programs • Look for research-based, proven approaches • Pilot to gain experience, test ideas on small scale • If successful, evaluate, modify, and expand scope • Do not let history dictate your plan for next year! “It is our attitude at the beginning of a difficult task which, more than anything else, will affect its successful outcome.”- William James

34. But remember…. • It is never about just ONE thing! • Focus on Coaching, Curriculum, Families, Data… • In the end, everything comes down to intensity and focus. • Southwest Airlines vs. Ted Generating great outcomes is not a function of one decision you make, but the sum total of EVERY decision you make.

35. Using Data to Make Decisions: Head Start Case Study

36. Introduction: Using Data DATA da·ta’dā-tə, ‘da-tə n. pl. Factual information (as measurements or statistics) used as a basis for reasoning, discussion, or calculation • Merriam-Webster Online Dictionary, www.merriam-webster.com

37. How Do We Use Data to Make Decisions… … at home? … at work?

38. Case Study: PFT Head Start PROCESS: • Read case study individually. • Review and summarize key points together. • In groups, discuss & identify one solution for program improvement.i.e. What would you address, and how? • Report out to the group.

39. Case Study: Key Points

40. Case Study: Key Points (cont’d)

41. Case Study: Solutions

42. Case Study: Solutions

43. Case Study “Plus” – Exhibit A

44. Case Study “Plus” – Exhibit B

45. Case Study “Plus” – Exhibit C

46. Case Study “Plus” – Exhibit D

47. Case Study “Plus”: Solutions

48. Case Study Reflection Compare discussion/solution #1 (based on case study only) and discussion/solution #2 (based on case study + data). What were the key differences … • In your proposed solutions (results)? • In the discussion itself (process)? • In how you felt about it (relationships)?

49. Summary of Key Points • Managing with data enables us to make more objective evaluations/comparisons and more informed choices. • We can “manage by exception” and allocate resources more strategically. • Using data, we typically identify problems and generate solutions in a more specific, focused and effective way.

50. An Overview of Compliance-Based Monitoring: Manage By Information (MBI) Basics