300 likes | 310 Views
TIMELESS Learning Policy & Practice. JD Hoye President National Academy Foundation. Theory of Change. Student Outcomes. Academy Students. Local Partners & Advisory Board. Community Benefits. Increased Capacity to Implement NAF Model. NAF Supports. Local Education Partners.
E N D
TIMELESS Learning Policy & Practice
JD Hoye President National Academy Foundation
Theory of Change Student Outcomes Academy Students Local Partners & Advisory Board Community Benefits Increased Capacity to Implement NAF Model NAF Supports Local Education Partners Academy Outcomes NAF Funders & Partners
Research, Data and Evaluation PANEL DISCUSSION
Performance Measurement • Where We Are Going: • Clarifying What it Means to Be a NAF Academy • Streamline Data Collection and Reporting • Self-Assessment Tool revised based on feedback from Academies and additional input from experts • Increase Opportunities For Teachers, Administrators, Parents, and Students • Where We Have Been: • Creating Standards for High Fidelity • Essential Data to Collect • Including all Members of the Network in the Discussion • Rich Dialogue Around Performance Measurement
Performance Measurement System • Clearly Articulated Standards & Benchmarks • System for Collecting Data For Those Indicators & Benchmarks • System for Analyzing Reporting, • Process for Using Indicator Data to improve student performance
Components of the NAF Performance Measurement System Self-Assessment & Planning Tool Student-level Data (ConnectEDU) Stakeholders Surveys Creating a Culture of Inquiry
Key Findings From Previous Research • MDRC Random Assignment Study Most Definitive • No Measurable Effects • Academic Achievement • Graduation Rates • Post-Secondary Education Outcomes • Positive Effects On • Student Engagement • Career Related/Work-Based Learning Experiences • Long Term Earnings (Substantial Effects)
Key Developments in Research and Evaluation • Increased Premium on Rigorous Evaluation and Reliable Estimates of Program Effects • Increased Attention to Career and College Readiness and Overlap Between the Two • Early Indicators of High School Success • Emphasis on Data-Driven Decision-Making
Need For Additional Research • Impacts on Academic Outcomes, Post-Secondary Education • Key Drivers of Positive Impacts • Understanding Cross-Site Variation in Impacts • NAF Specific Impacts
Key Research & Evaluation Questions for NAF • What does it mean to be a NAF Academy? • What Experiences Should Students Have? • What Outcomes Should Fully Implemented NAF Academies Achieve? • What Data Are Required • To Evaluate NAF? • To Monitor Implementation? • To Maximize program Effectiveness?
Components of the NAF Performance Measurement System Self-Assessment & Planning Tool Student-level Data (ConnectEDU) Stakeholders Surveys Creating a Culture of Inquiry
Self-Assessment and Planning Tool Overview • Development Process: • Conducted three tier process: • 280 academies reviewed • Variance review, 5 in each region • Visit 15 sites • Site visits conducted by a team of experts including MDRC, National Career Academy Coalition, and members of the Data Tracking Steering Committee to the 15 Academies that scored in the Leader/ Distinguished category or are AOEs • Self-Assessment Tool revised based on feedback from Academies and additional input from experts • Guiding Principles: • Clearly articulate a tangible rubric on what defines a NAF Academy • Demonstrate emerging and best practice along the rubric • Develop a tool that allows Academies to reflect on their performance against the rubric • Encourage and inform an exchange of best practices and lessons learned among Academies and between NAF and the Academies
The Self-Assessment Tool has 4 key sections Each Section outlines the core concepts in the rubric… …and drives each concept to a level of tangible detail Academy Development Advisory Board Curriculum & Instruction Work-Based Learning & Internships • Recruitment • Cohort scheduling ensures that academy students experience NAF academy courses plus a minimum of two core subject at each grade level. • Cohort scheduling ensures that students receive instruction in a personalized environment where every child is known by a caring adult. • Cohort scheduling assumes a schedule where there is weekly common planning time for the academy team of at least three staff, so that integrated learning, student supports and individualized student assessment can occur. • Length of Program • Cohort Scheduling • Academy Decision-Making • Data Collection & Revision • Board Membership • Professional Development • Board Function • Academy Leadership • Board Supports • Integration • Instructional Practices • Instructional Supports • Provides a WBL Program • Internship Completion • Students’ Career Goals
Components of the NAF Performance Measurement System Self-Assessment & Planning Tool Student-level Data (ConnectEDU) Stakeholders Surveys Creating a Culture of Inquiry
Components of the NAF Performance Measurement System Self-Assessment & Planning Tool Student-level Data (ConnectEDU) Stakeholders Surveys Creating a Culture of Inquiry
Why Do Surveys? • Provide NAF periodic snapshots of how academies and students are performing. • Provide individual academies information to help inform their own growth and improvement. • Fill in the gaps in the performance measurement system with data that can not be obtained using other data sources.
Whom To Survey? • Students • Teachers • Parents • Employers
Surveys: Next Steps • Focus on a student survey to measure behavior, attitudes and outcomes for which we don’t have measures in school records. • Select/create instrument or instruments. • Select a sample of academies that represent the network. • Survey all students in the sampled academies. • Conduct in-class or on-line student survey every two years to measure changes.
Components of the NAF Performance Measurement System Self-Assessment & Planning Tool Student-level Data (ConnectEDU) Stakeholders Surveys Creating a Culture of Inquiry
Preparation • Ensure Data Quality • Organize for collaborative work • Build assessment literacy and skill in data interpretation
Inquiry • Create data overview • Dig into student data
Move Beyond Data Analysis • Examine policy and procedures and instruction • Plan to assess programs • Acting and assessing
Components of the NAF Performance Measurement System Self-Assessment & Planning Tool Student-level Data (ConnectEDU) Stakeholders Surveys Creating a Culture of Inquiry