1 / 22

Measuring Implementation: School-Wide Instructional Staff Perspective

Measuring Implementation: School-Wide Instructional Staff Perspective. Amy Gaumer Erickson, Ph.D. University of Kansas Evaluator: Kansas & Missouri SPDGs SIG Evaluators Webinar March 17, 2011. What types of measures are implemented?. Self-Report Interviews/Focus Groups Observations

rafer
Download Presentation

Measuring Implementation: School-Wide Instructional Staff Perspective

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Measuring Implementation:School-Wide Instructional Staff Perspective Amy Gaumer Erickson, Ph.D. University of Kansas Evaluator: Kansas & Missouri SPDGs SIG Evaluators Webinar March 17, 2011 Gaumer Erickson (2011)

  2. What types of measures are implemented? • Self-Report • Interviews/Focus Groups • Observations • Retrospective/Reflective Learning • Goal Attainment • Ratings of Fidelity • Outcome Data Gaumer Erickson (2011)

  3. What perspectives do we need to get a complete picture of implementation in schools? • Measures/observations completed by: • Implementation Coaches • School Leadership Teams • All School Instructional Staff • Students • Families/Communities Gaumer Erickson (2011)

  4. Measures completed by Implementation Coaches • SW-PBS Benchmarks of Quality (BoQ) • MiBLSI: http://miblsi.cenmi.org/MiBLSiModel/Evaluation/Measures/BenchmarksofQuality.aspx • SW-PBS School-wide Evaluation Tool (SET) • OSEP Technical Assistance Center on PBIS: http://www.pbis.org/evaluation/evaluation_tools.aspx Gaumer Erickson (2011)

  5. Measures completed by School Leadership Teams • SW-PBS Team Implementation Checklist (TIC) • PBIS Surveys: http://www.pbssurveys.org/pages/Home.aspx • Effective Behavior Support Team Implementation Checklist (EBS-SAS) • PBIS Surveys: http://www.pbssurveys.org/pages/Home.aspx • Planning and Evaluation Tool for Effective Reading Supports – Revised (PET-R) • DIBELS Data System: http://dibels.uoregon.edu/resources/ • Implementation Rubrics • Illinois PBIS Network: http://www.pbisillinois.org/ Gaumer Erickson (2011)

  6. Student Measures • DIBELS • DIBELS Data System: https://dibels.uoregon.edu/resources.php • AIMSweb • AIMSweb: http://www.aimsweb.com/ • School-Wide Information System (SWIS) • SWIS: http://www.swis.org/ • State Performance Plan APR Data (Indicators 1, 2, 3, 4, 5, 13, 14) • Student Perceptions Gaumer Erickson (2011)

  7. Family/Community Measures • Parent Satisfaction • Missouri Improvement Program Advanced Questionnaire http://dese.mo.gov/divimprove/sia/dar/advance_questionnaire_surveys.html • Family as a Teacher Inventory • Parent-Teacher Involvement Questionnaire • Fast Track Project http://www.fasttrackproject.org/data-instruments.php • Interviews/Focus Groups Gaumer Erickson (2011)

  8. Measures completed by All School Instructional Staff • Self-Assessment Survey (SAS) • PBIS Surveys: http://www.pbssurveys.org/pages/Home.aspx • Standards Assessment Inventory • National Staff Development Council http://www.learningforward.org/standards/sai.cfm • Classroom/Teacher Observations Gaumer Erickson (2011)

  9. Which perspective do you think your grants measure most effectively? • A: Implementation Coaches • B: School Leadership Teams • C: All School Instructional Staff • D: Stakeholders: Student/Parents/Communities Gaumer Erickson (2011)

  10. School Staff Survey Design • Identified essential features of school reform initiatives (e.g., RTI, SW-PBS, PLCs, High Schools that Work, Reading First) • Analyzed other measures completed by school instructional staff (e.g. PBIS Self-Assessment Survey, RTI Implementation Tool, National Staff Development Council Standards Assessment Inventory, Missouri School Improvement Process Faculty Advanced Questionnaire) • Wrote over 100 items and then narrowed it down to 33 through pilot testing with schools and analysis by a state evaluation team • Implemented survey in 14 schools Gaumer Erickson (2011)

  11. School Staff Survey Reliability • Overall Alpha: 0.98 • Three Factors • School Implementation (17 items): 0.95 • Classroom Implementation (10 items): 0.85 • Individual Student Implementation (5 items): 0.90 • Four Groups • Teachers (N=294): 0.96 • Administrators (N=8): 0.90 • Other Certified Staff (N=20): 0.97 • Noncertified Staff (N=6): 0.85 Gaumer Erickson (2011)

  12. School Implementation • I can summarize the school's shared vision/mission. • I have a clear understanding of the phrase: “tiered levels of academic & behavior support.” • I feel that my administrators are committed to implementing tiered levels of academic & behavior supports. • I receive coaching/mentoring to implement evidence-based instructional practices. • I think my school does a good job of addressing the academic & behavior needs of students at tier 1 (universal). • I think my school does a good job of addressing the academic & behavior needs of students at tier 2 (small group). • I think my school does a good job of addressing the academic & behavior needs of students at tier 3 (intensive). • I regularly see students move between tiers of support as their academic needs change. • I regularly see students move between tiers of support as their behavior needs change. • I am involved in meetings where data results are discussed & problem solving occurs. • I receive school-wide academic & behavior data in usable & understandable formats. • I participate in professional development where I learn how to monitor students' progress & use progress monitoring data. • I have the time necessary to analyze student data & problem solve with my colleagues. • I think my school is a good place to work. • I think my school has an effective process in place to identify available resources (e.g., materials, technology, people). • I evaluate the effectiveness of core instruction based on progress monitoring data. • I think that the current behavior/academic programs in my school are improving education for students in my school. Gaumer Erickson (2011)

  13. Classroom Implementation • I use assessment data at least three times a year to monitor students' progress. • I adapt the environment, curriculum, & instruction based on each student’s academic & behavior data. • I participate in professional development where I learn ways to improve my instructional practices. • I receive coaching/mentoring to help me implement tiered levels of academic & behavior support. • I have the technology & resources that I need to provide effective instruction. • I am able to meet the students' diverse needs. • I consider my students' background when I teach and/or interact with students. • I regularly communicate with families regarding student academic & behavior goals/progress. • I make informed decisions based on feedback from families. • I collaborate with my colleagues on a regular basis. Gaumer Erickson (2011)

  14. Individual Student Implementation • I am involved in action planning tiered supports with the other staff & administrators at my school. • When I am concerned about a student’s academic success, I collaborate with a team to identify intervention. • I feel that the team that addresses academic needs provides valuable feedback & makes informed decisions. • When I am concerned about a student’s behavior success, I collaborate with a team to identify intervention. • I feel that the team that addresses behavioral needs provides valuable feedback & makes informed decisions. • I think my school does a good job in including parents as team members in data-based decision-making. Gaumer Erickson (2011)

  15. Reporting Results • School Summaries (to support data-based decision-making and action planning) • School Comparisons Across Years • Dissemination & Evaluation of Program Effectiveness • SPDG APR Gaumer Erickson (2011)

  16. Gaumer Erickson (2011)

  17. Gaumer Erickson (2011)

  18. Gaumer Erickson (2011)

  19. Gaumer Erickson (2011)

  20. SPDG APR Reporting Percent of school personnel involved in the MIM who report the use of data-driven decision making; [correlated with SPP Indicator 3; Program Performance Measure 1.2]. To identify the level of data-driven decision making, school staff were asked to rank the statement, I adapt the curriculum, instruction, & environment based on each student’s academic & behavior data, on the MIM School Staff Survey. The results show that 79% of MIM school staff report using data to make adaptations for students. In addition, school staff was asked to rank the following statements related to data-driven decision making: • I use assessment data at least three times a year to monitor students' progress. Out of 429 school staff who responded, 351 (82%) ranked this statement as a 4 or 5. • I am involved in meetings where data results are discussed & problem solving occurs. Out of 444 school staff who responded, 302 (68%) ranked this statement as a 4 or 5. • My students move between tiers of support as their academic needs change. Out of 424 school staff who responded, 247 (58%) ranked this statement as a 4 or 5. • My students move between tiers of support as their behavior needs change. Out of 410 school staff who responded, 204 (50%) ranked this statement as a 4 or 5. Gaumer Erickson (2011)

  21. Discussion • What other methods/tools are you using to obtain data from all school instructional staff? • Is a measure like the School Staff Survey something that would be helpful on your projects? Gaumer Erickson (2011)

  22. For More Information: Amy Gaumer Erickson, Ph.D. University of Kansas aerickson@ku.edu 785-864-0517 Gaumer Erickson (2011)

More Related