220 likes | 369 Views
Promoting Academic Success Evaluation Progress Washington Education Research Association December 7, 2006. Robert Barnoski (Barney) Washington State Institute for Public Policy Phone: (360) 586-2744 E-mail: barney@wsipp.wa.gov Institute Publications: www.wsipp.wa.gov.
E N D
Promoting Academic Success Evaluation Progress Washington Education Research Association December 7, 2006 Robert Barnoski (Barney) Washington State Institute for Public Policy Phone: (360) 586-2744 E-mail: barney@wsipp.wa.gov Institute Publications: www.wsipp.wa.gov
Washington State Institute for Public Policy • Created in 1983 by the state Legislature • Mission: Carry out non-partisan research on projects assigned either by the legislature or the Institute’s Board of Directors • 8 legislators • 4 higher education provosts or presidents • 4 state agency directors
2006 Legislature directed Institute to Evaluate Promoting Academic Success The evaluation should focus on determining: (a) overall effectiveness in helping students pass the WASL; (b) relative effectiveness of different remedial strategies offered; and (c) relative effectiveness by student characteristics, including economic status, limited English proficiency, and ethnicity. The office of the superintendent of public instruction shall provide all data necessary to conduct such analyses, and shall help coordinate data collection directly from districts administering the remedial programs as necessary. ESSB 6386, §607 (11), Chapter 372, Laws of 2006, supplemental operating budget.
An interim report is due to the Legislature by December 15, 2006. The final report is due December 15, 2007.
Analyses of 10th Grade Spring 2006 WASL DataReports on Website www.wsipp.wa.gov
(a) Evaluating Overall Effectiveness of Summer 2006 PAS Compare WASL Retake Results for: • Students who participated in Summer 2006 PAS and retook WASL in August, • With similar group of students not in PAS but who retook WASL in August. • Key is finding a similar comparison group. • Statistical control variables (covariates) are still used in multivariate analysis to estimate impact of PAS participation on WASL retake. • Two dependent variables: • Met standard and • Gain on WASL scale scores.
(a) Evaluating Overall Effectiveness of Summer 2006 PAS PAS Student Data Collection: • OSPI built an application for recording students participating in PAS. • Concerns about privacy kept OSPI from requesting the names of the teachers who provided the PAS instruction to the PAS students. • As a result, Institute must rely upon the using district, school, content area, and PAS program name to link teacher surveys to student WASL results. • Institute must reply upon school districts recording which students took summer PAS. • OSPI called districts to encourage them to enter Summer 2006 PAS student data. • School districts have until June 2007 to enter PAS students.
(b) Evaluating Relative Effectiveness of Different Remedial Strategies for Summer 2006 PAS Measuring Different Remedial Strategies: • Institute developed three surveys for PAS teachers after discussions with OSPI staff (Reading, Writing and Math). • Two versions were created: paper and pencil, and online. • The teachers were to be surveyed at the end of the summer 2006 effort. • We will analyze the associations between different strategies and gains on WASL retakes for students who experienced the strategies. • Statistical control variables (covariates) are still used in multivariate analysis when estimating impact of PAS strategies on the WASL retake.
(b) Evaluating Relative Effectiveness of Different Remedial Strategies for Summer 2006 PAS Remedial Strategy Data Collection: • OSPI built an application to record PAS program and teacher contact information. • Institute must reply upon district PAS coordinator recording the summer 2006 PAS program/teacher contact information. • Not all districts entered these data before the end of summer school. • School districts continued to enter Summer PAS data in August and September. • Institute sent out teacher surveys as soon as we received the PAS teacher’s contact information. • Institute hired WSU Social & Economic Sciences Research Center to do follow-up phone calling.
Examples of Strategies on Math Survey for Summer 2006 PAS What assessments were used to determine student math needs at the beginning of this instruction? (Check all that apply) Diagnostic test(s) Classroom assignments WASL math strand results Quizzes/tests Homework assignments Teacher observation Standardized achievement tests WASL math scores None Other (specify):
PAS teacher survey response rate is approximately 65 percent.
Examples of Strategies on Math Survey for Summer 2006 PAS In your section(s), approximately what percentage of time did you spend in each of the following areas? (Must add to 100%) __ Academic/vocational/career counseling __ General instruction for math __ Motivation __ Specific instruction on the EALRs and GLEs __ Study habits __ WASL test taking skills: guessing and scoring rules __ Other (specify):
Examples of Strategies on Math Survey for Summer 2006 PAS What instructional materials did you use in your section(s)? (Check all that apply) OSPI Instructional Support Modules WASL practice tests/released items other than from OSPI Instructional Support Modules WASL teaching materials other than OSPI Instructional Support Modules Materials aligned with EALRs/GLEs other than OSPI Instructional Support Modules Materials specifically obtained for this remedial instruction other than OSPI Instructional Support Modules District-adopted instructional materials Technology-based materials. Name(s): Other materials (describe):
Examples of Strategies on Math Survey for Summer 2006 PAS What special training have you received to teach students who have not met the 10th-grade math standards? (Check all that apply) OSPI Instructional Support Module training from OSPI staff OSPI Instructional Support Module training from district or ESD staff None • Other training (describe): In which of the following WASL math activities have you participated? (Check all that apply) Item writing Scoring Range finding Data review OSPI/ESD deep alignment Curriculum alignment None Other (specify):
(c) Estimating the Influence of Student Characteristics for Summer 2006 PAS • Analyze association between different strategies and gains on WASL retakes by student characteristics.
Math Teacher Survey In your section(s), approximately what percentage of time did you spend in each of the following areas?