Young Adult Literacy Program

1. Young Adult Literacy Program Jean-Marie Callan NYATEP Youth Academy | March 2014

2. About the Center The Center for Economic Opportunity • Established in 2006 to implement, monitor, and evaluate the City’s ambitious anti-poverty agenda. Innovation Fund • $100 million public-private fund. • Supports the implementation of CEO’s anti- poverty initiatives and pilot programs. • Funds the monitoring and evaluation of programs. Commitment to Evaluation • All program outcomes tracked. • Program-specific evaluation strategies developed. • Evaluation products include early implementation reviews, analyses by program area, and several long-term evaluations. • Evaluation partners include City agencies and independent evaluation firms.

3. At-Risk and Disconnected Youth in NYC Overview • 187,179 young adults between the ages of 16 and 24 are neither working nor in school.* • Of these, approximately half have high school degrees, while others struggle with basic literacy skills and face other barriers to education and employment. Strategies • CEO offers a wide range of programs for at-risk/disconnected youth. • Interventions focus on positive youth development, increasing literacy/numeracy skills and educational attainment, and providing job-readiness and work experience. • CEO has identified 3 effective strategies for engaging at-risk/disconnected youth: • Employment: Subsidized employment to provide youth with real work experience. • Education: Promoting educational attainment to increase economic self-sufficiency. • Case Management: Connecting youth to positive caring adults and addressing barriers to long term economic opportunity. *Source: 2011 American Community Survey as calculated by CEO.

4. How CEO Works Center for Economic Opportunity • Implement new programs in collaboration with multiple City agencies. • Utilize strong internal and external evaluation teams to measure results. • Base future funding decisions on program results. • Share best practices and lessons learned. Agencies • Program design, management of contracts and providers, and program implementation. Providers • Provide direct services, recruit participants, and offer other services. Independent Evaluators • Pool of 9 independent evaluation firms contracted to measure the impact of CEO programs.

5. Young Adult Literacy (YAL) Program Target Population YAL targets young adults ages 16-24 who are out of school, out of work, and test at pre-HSE levels (specifically at 4th-8th grade levels on the TABE). A particular focus is placed on serving those at the 4th-6th grade levels. Program Model Offers literacy and numeracy instruction, work readiness, support services, and stipends to disconnected young adults at the pre-HSE level. YAL is delivered by 16 program sites operated by 3 Public Library systems and 8 CBOs contracted through DYCD.

6. YAL Outcomes and Prior Evaluations Program Outcomes • Enrolled 777 youth in FY 2013, with 450 literacy gains, over 380 numeracy gains, and 626 placed in internships • Monitoring has been used to assess internship participation and data was used to change internship requirements to promote higher uptake Prior Evaluations • On the impact of adding paid internships (Westat & Metis, 2011) • On educational gains and longer term retention (Westat, 2013) • Findings have been used to change the program model, for example adding paid internships to all sites

7. Next for YAL Young Men’s Initiative • YAL was expanded through the Young Men’s Initiative in 2011. • The Young Men’s Initiative is a comprehensive suite of programs and policies designed to address disparities between young black and Latino men and their peers in the areas of health, employment, education, and criminal justice. Project Rise • Elements of YAL informed the Social Innovation Fund’s Project Rise. • In 2010, CEO, in partnership with the Mayor’s Fund to Advance NYC, received a Social Innovation Fundaward to expand 5 promising anti-poverty programs. • Project Rise is a twelve-month education-conditioned internship program for disconnected youth ages 18-24 in NYC, Newark (NJ), and Kansas City (MO). Forthcoming Evaluation • MDRC study on lessons learned from the implementation of YAL

8. For More Information NYC Center for Economic Opportunity Jean-Marie Callan, Senior Program and Policy Advisor 212-341-0714 JMCallan@cityhall.nyc.gov www.nyc.gov/ceo /NYCOPPORTUNITY @NYCOPPORTUNITY

9. New York City’s Young Adult Literacy Program Findings from an Implementation Study Farhana Hossain, MDRC NYATEP Youth Academy | March 2014

10. Overview • The StudyDescription of study design and activities • The ParticipantsCharacteristics of youth who participate in YAL • Implementation LessonsCommon features and practices of high-performing sites • The ChallengesFactors that challenge the implementation of YAL • ConclusionBroad lessons learned

11. The Study • Goal: Evaluate well-performing sites to explore unique or common factors that appear to lead to their success. • Five Selected Programs • Performed well in some or all outcome measures • 3 housed in CBOs; 2 within library systems • Data Collected (August and early Sept. 2013) • Interviews with 28 staff; focus groups with 27 students • Observe classroom and internship activities • Limitations • Low-performing sites not included • Cannot make causal claims on practices and outcomes

12. The Participants • YAL studentsat the five study sites (n = 98) • 45% age 21 or over • 70% male • 94% Black or Hispanic • 20% parenting • Many with criminal justice history • 47% read below the 6th grade level • 60% were below the 6th grade level in math • 57% attended at least one cycle of YAL before • Most did not have significant work experience Demographic Academic+Work

13. The Participants • Barriers mentioned most frequently by staff: • Lack of stable home lives • Responsibilities of parenthood and family life • Requirements of other public systems • Learning disabilities • In their own words: • Main goal for joining YAL is to get HSE credentials • What they liked most? • Small classes; individual attention; teacher persistence • Relationships with staff and peers

14. Implementation Lessons • Organization and staffing • Characteristics of the provider organizations play a significantrole in shaping the implementation of YAL, which draws financial, administrative, and staff support from the providers. • Common features of high-performing programs: • Investment in building organizational capacity to serve disadvantaged youth • Integration of YAL into broader organizational effort for youth • Creating a collaborative, structured community environment • Leveraging resources inside and outside of the organization, including community partnerships

15. Best Program Practices • Common features observed at high-performing programs: • Structured, multi-step recruitment and intake • Clear articulation and consistent enforcement of rules • Full-time academic instructors with time for lesson-planning • Daily routines of classroom activities • Setting goals inside and outside of the classroom • Making deliberate connections between academic andnon-academic components of the program • Fostering relationships among students and staff • Deliberate and structured transition from program

16. Implementation Challenges • Setting and managing student expectations • Designing curricula for a heterogeneous population • Reinforcing norms while accommodating life challenges • Designing rigorous, engaging work experiences • Finding transition for older youth

17. Concluding Thoughts • The study reinforced the need for pre-literacy services among disconnected youthin New York City. • Staffing stability and consistency in the classroom are important for engagement and retention. • Strong implementation of programs for disconnected youth may require: • Investments in building organizational capacity, includingstaff training and building community relationships. • A holistic approach towards serving the population and building a continuum of services.

18. Thank you! For more research on programs and policies that affect youthand other disadvantaged groups in society, please visit: www.mdrc.org /MDRCNews @MDRC_News

19. Elements of an Effective Youth Program

20. Overview BronxWorks • Setting the Stage • Program Space • Scheduling • Staffing • Recruitment and Orientation • Information Session • Intake and Assessment • Orientation • Program Structure and Retention • Setting Goals and Expectations • Wraparound Services • Transition and Follow Up • Next Steps

21. Setting the Stage-Program Space BronxWorks Are your classrooms and program areas appropriate for young adults? • Welcoming: clutter-free, literacy-rich, etc. • Accessible: public transportation, easy to identify, etc. • Appropriate: furniture, supplies, age-appropriate materials, etc.

22. Setting the Stage- Scheduling BronxWorks Does your class schedule and program hours respond directly to the needs of your target population? • Strict yet flexible • Full-Time vs. Part-Time • Morning vs. Evening • Accounts for Other Responsibilities

23. Setting the Stage-Staffing • Supportive • Engaging • Approachable • Knowledgeable • Resourceful BronxWorks Have your instructors and support staff received training and understand youth development principles?

24. Recruitment and Orientation-Information Sessions BronxWorks How can you transform your first encounter into a valuable experience? • Use it as the first opportunity to set your program standards and expectations! (Dress code, scheduling, etc.) • Have designated staff so that the message is consistent! • Be very honest and transparent with potential participants!

25. Are you getting an accurate assessment? • Identifying additional strengths and potential barriers beyond academics (i.e. childcare, commute, lack of basic resources, etc.) • Overcoming the limitations of standardized assessments like TABE by adding additional components (i.e. San Diego Quick Assessment and a writing cample.) Recruitment and Orientation-Intake and Assessment BronxWorks

26. Recruitment and Orientation- Orientation BronxWorks Elements of an Effective Orientation: • Days and times should mirror actual program hours • Create a standard Orientation Packet that includes: • Information about your agency and other available services • Program structure • Rules and Regulations (signed by participant) • Self-reflection worksheets • An orientation assessment to get participant feedback • All hands on deck! Get staff involved so that they get to know participants prior to the academic component.

27. Program Structure and Retention- Setting Goals and Expectations BronxWorks Work with participants to help them set long and short term goals! • Recognize that some participants may take longer to fully transition through the program, and this is ok! • Be honest, and help them to identify the small victories that lead them to the big ones. Hold participants accountable for their success! • Review and sign program/attendance contract. • Meet with students after set number of absences. • Revisit goal worksheets periodically.

28. Program Structure and Retention- Wraparound Services • Assign a Primary Person to each participant. • Assign themes to each month and host personal development workshops that address each of those topics. • Organize resource fairs! • Make outside referrals as necessary. BronxWorks Ensuring success for disconnected youth means moving beyond literacy and numeracy levels! How do you address non-academic barriers and challenges?

29. Program Structure and Retention- Transition and Follow Up BronxWorks Helping students transition to the next step is critical for keeping them engaged with the program! • Evaluate program structure to ensure that it facilitates the transition process and responds to the needs and abilities of participants. • Schedule a “Transition Meeting” with the former instructor, new instructor and primary person to discuss overall progress and areas for growth • As always, be honest! Have an open conversation with the participant about this step in their development.

30. Next Steps BronxWorks So Now What? • Solicit feedback from youth about what works. • Develop a system for seeking continued feedback (town hall meetings, evaluations, an open door policy, etc.) • Review materials to ensure that they are engaging. • Get staff onboard!