TIMELESS Learning Policy & Practice

1. TIMELESS Learning Policy & Practice

2. JD Hoye President National Academy Foundation

3. Theory of Change Student Outcomes Academy Students Local Partners & Advisory Board Community Benefits Increased Capacity to Implement NAF Model NAF Supports Local Education Partners Academy Outcomes NAF Funders & Partners

4. Research, Data and Evaluation PANEL DISCUSSION

5. Moving Toward a Performance Measurement System

6. Performance Measurement • Where We Are Going: • Clarifying What it Means to Be a NAF Academy • Streamline Data Collection and Reporting • Self-Assessment Tool revised based on feedback from Academies and additional input from experts • Increase Opportunities For Teachers, Administrators, Parents, and Students • Where We Have Been: • Creating Standards for High Fidelity • Essential Data to Collect • Including all Members of the Network in the Discussion • Rich Dialogue Around Performance Measurement

7. Performance Measurement System • Clearly Articulated Standards & Benchmarks • System for Collecting Data For Those Indicators & Benchmarks • System for Analyzing Reporting, • Process for Using Indicator Data to improve student performance

8. Components of the NAF Performance Measurement System Self-Assessment & Planning Tool Student-level Data (ConnectEDU) Stakeholders Surveys Creating a Culture of Inquiry

9. Context for Performance Measurement System

10. Key Findings From Previous Research • MDRC Random Assignment Study Most Definitive • No Measurable Effects • Academic Achievement • Graduation Rates • Post-Secondary Education Outcomes • Positive Effects On • Student Engagement • Career Related/Work-Based Learning Experiences • Long Term Earnings (Substantial Effects)

11. Key Developments in Research and Evaluation • Increased Premium on Rigorous Evaluation and Reliable Estimates of Program Effects • Increased Attention to Career and College Readiness and Overlap Between the Two • Early Indicators of High School Success • Emphasis on Data-Driven Decision-Making

12. Need For Additional Research • Impacts on Academic Outcomes, Post-Secondary Education • Key Drivers of Positive Impacts • Understanding Cross-Site Variation in Impacts • NAF Specific Impacts

13. Key Research & Evaluation Questions for NAF • What does it mean to be a NAF Academy? • What Experiences Should Students Have? • What Outcomes Should Fully Implemented NAF Academies Achieve? • What Data Are Required • To Evaluate NAF? • To Monitor Implementation? • To Maximize program Effectiveness?

14. Components of the NAF Performance Measurement System Self-Assessment & Planning Tool Student-level Data (ConnectEDU) Stakeholders Surveys Creating a Culture of Inquiry

15. Self-Assessment and Planning Tool Overview • Development Process: • Conducted three tier process: • 280 academies reviewed • Variance review, 5 in each region • Visit 15 sites • Site visits conducted by a team of experts including MDRC, National Career Academy Coalition, and members of the Data Tracking Steering Committee to the 15 Academies that scored in the Leader/ Distinguished category or are AOEs • Self-Assessment Tool revised based on feedback from Academies and additional input from experts • Guiding Principles: • Clearly articulate a tangible rubric on what defines a NAF Academy • Demonstrate emerging and best practice along the rubric • Develop a tool that allows Academies to reflect on their performance against the rubric • Encourage and inform an exchange of best practices and lessons learned among Academies and between NAF and the Academies

16. The Self-Assessment Tool has 4 key sections Each Section outlines the core concepts in the rubric… …and drives each concept to a level of tangible detail Academy Development Advisory Board Curriculum & Instruction Work-Based Learning & Internships • Recruitment • Cohort scheduling ensures that academy students experience NAF academy courses plus a minimum of two core subject at each grade level. • Cohort scheduling ensures that students receive instruction in a personalized environment where every child is known by a caring adult. • Cohort scheduling assumes a schedule where there is weekly common planning time for the academy team of at least three staff, so that integrated learning, student supports and individualized student assessment can occur. • Length of Program • Cohort Scheduling • Academy Decision-Making • Data Collection & Revision • Board Membership • Professional Development • Board Function • Academy Leadership • Board Supports • Integration • Instructional Practices • Instructional Supports • Provides a WBL Program • Internship Completion • Students’ Career Goals

17. Components of the NAF Performance Measurement System Self-Assessment & Planning Tool Student-level Data (ConnectEDU) Stakeholders Surveys Creating a Culture of Inquiry

18. Why Student Data?

19. Why ConnectEDU?

20. Why ConnectEDU?

21. Why ConnectEDU?

22. Components of the NAF Performance Measurement System Self-Assessment & Planning Tool Student-level Data (ConnectEDU) Stakeholders Surveys Creating a Culture of Inquiry

23. Why Do Surveys? • Provide NAF periodic snapshots of how academies and students are performing. • Provide individual academies information to help inform their own growth and improvement. • Fill in the gaps in the performance measurement system with data that can not be obtained using other data sources.

24. Whom To Survey? • Students • Teachers • Parents • Employers

25. Surveys: Next Steps Focus on a student survey to measure behavior, attitudes and outcomes for which we don’t have measures in school records. Select/create instrument or instruments. Select a sample of academies that represent the network. Survey all students in the sampled academies. Conduct in-class or on-line student survey every two years to measure changes.

26. Components of the NAF Performance Measurement System Self-Assessment & Planning Tool Student-level Data (ConnectEDU) Stakeholders Surveys Creating a Culture of Inquiry

27. Preparation • Ensure Data Quality • Organize for collaborative work • Build assessment literacy and skill in data interpretation

28. Inquiry • Create data overview • Dig into student data

29. Move Beyond Data Analysis • Examine policy and procedures and instruction • Plan to assess programs • Acting and assessing

30. Components of the NAF Performance Measurement System Self-Assessment & Planning Tool Student-level Data (ConnectEDU) Stakeholders Surveys Creating a Culture of Inquiry