Going Beyond the Indicators to Assess Progress in the Advanced Placement Incentive Program

1. Going Beyond the Indicators to Assess Progress in the Advanced Placement Incentive Program Barbara D. Acosta Lori A. McGee

2. AIMS AND GOALS Eliminate the disparity between poverty and non-poverty students in their academic achievement Goal 1: Increase the participation of low-income students in rigorous courses Goal 2: Increase the number of AP courses offered Goal 3: Increase the number of low-income students completing AP courses, taking AP exams, and successfully passing AP exams Goal 4: Increase the capacity of districts, schools, and communities to support the implementation of rigorous course work GW-CEEE 2009

3. APIP COMPONENTS • VERTICAL LEARNING TEAMS • Vertical alignment • Data-based decision-making • Instructional strategies • AP MENTORING CADRE • Instructional strategies • Data-based decision-making PARENT OUTREACH TRAINING GW-CEEE 2009

4. METHOD: GUSKEY’S 5 LEVELS 5. STUDENT LEARNING OUTCOMES GW-CEEE 2009

5. ADVANCED PLACEMENT INCENTIVE PROGRAM GW-CEEE 2009

6. PROGRAM LOCATIONS • 11 districts • 15 High Schools • 1,683 AP students • 18 Middle Schools • 6,374 Grade 8 students • 42% low-income • Most schools “Under Improvement” GW-CEEE 2009

7. EVALUATION QUESTIONS GW-CEEE 2009

8. DATA COLLECTION • DOE • Ongoing communication with project staff • Document review • Student outcome data • VERTICAL LEARNING TEAMS • District leader observations • Focus groups • Teacher survey* • PARENTS • Focus groups • Parent survey • AP MENTORING CADRE • Teacher survey* GW-CEEE 2009

9. OUTCOMES

10. OUTCOMES – State Assessment GW-CEEE 2009

11. OUTCOMES – AP Enrollment GW-CEEE 2009

12. Outcomes – AP Tests GW-CEEE 2009

13. FORMATIVE FINDINGS

14. PROGRESS IN IMPLEMENTATION GW-CEEE 2009

15. Guskey Level 3 • Middle school students had 2 options: Honors and College Prep • Some teachers said they only recruited students for Honors who were “motivated” and had a “strong work ethic.” • Everyone else was placed in College Prep “The honor students…see the value in their education so they’re here to get what they can. A lot of our College Prep students don’t see any immediate returns for the work that we’re asking them to do and therefore, they can’t see the future and they don’t care.” – MS teacher GW-CEEE 2009

16. New indicators added in Year 2 GW-CEEE 2009

17. Selection Criteria for Honors/AP • Selection should be based on professional judgment and a variety of indicators • Students’ interests and aptitudes should be considered • Admission to the program should be open and ongoing • In eighth, ninth, and tenth grade, students should exceed the standards in relevant content areas in the [state test] because they will have to attend pre-AP courses and succeed in AP classes in a relative shorter period of time. [In Year 2 of the APIP] the high schools participating in PSAT testing of whole classes will be trained to use College Board AP Potential software to identify students with the potential to succeed in AP courses and tests. •  SOURCE: AP Incentive Program Goals GW-CEEE 2009

18. TEACHERS’ EXPECTATIONS% of teachers who agreed that all groups of students are equally likely to succeed in AP and honors courses. GW-CEEE 2009

19. What prevents more students from enrolling and succeeding in honors courses? (% of teachers) GW-CEEE 2009

20. CONCLUSIONS • After two years of implementation, progress was attained largely on the first two Guskey levels • Rudimentary structural changes were in place • Vertical learning teams appeared to be a useful and flexible structure for bringing about change • Participants in most districts were still struggling to apply their knowledge • Some teachers’ expectations for who could succeed in rigorous coursework had not changed • Structural barriers remained: • Poor preparation in earlier grades • Tracking • Professional judgment for selecting students + low expectations = few changes to business as usual GW-CEEE 2009

21. RECOMMENDATIONS • Address the decision-making process to ensure that more groups of students can participate and succeed in rigorous coursework • Conduct vertical alignment and increase instructional rigor starting in earlier grades • Provide more – and earlier – supports • Reconsider tracking • Reconsider decision-making process/criteria • Professional development should address teachers’ assumptions about who is capable of succeeding in challenging coursework GW-CEEE 2009

22. Lessons for evaluating programs with equity goals • Examine assumptions underlying the theory of change, particularly in relation to deficit orientations • Consider: • Is the professional development addressing the right issues? • What structural barriers might prevent attainment of goals? • Guskey Level 3 is key GW-CEEE 2009

23. References • Ford, D. Y., Grantham, T. C., and Whiting, G. (2008). Culturally and linguistically diverse students in gifted education: recruitment and retention issues. Exceptional Children, 74(3), 289-306. • Guskey, T. R. (2000). Evaluating professional development. Thousand Oaks, CA: Sage. • Ruggiero, R.A., and T.W. Jarrell. (July, 2005). Measuring Advanced Placement Participation and Access among Minority and Low Income Students in Delaware. National Center for Education Statistics, Washington D.C. GW-CEEE 2009

24. Contact information • Barbara Acosta bacosta@ceee.gwu.edu • Lori McGee lmcgee@ceee.gwu.edu • The George Washington University Center for Equity and Excellence in Education • Phone: 703.528.3588 GW-CEEE 2009