1 / 25

Determining Validity For Oklahoma’s Educational Accountability System

Determining Validity For Oklahoma’s Educational Accountability System. Prepared for the American Educational Research Association (AERA) Oklahoma State Department of Education April 15, 2004. Education Perspective for Oklahoma, 2002-2003.

madge
Download Presentation

Determining Validity For Oklahoma’s Educational Accountability System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Determining Validity For Oklahoma’s Educational Accountability System Prepared for the American Educational Research Association (AERA) Oklahoma State Department of Education April 15, 2004

  2. Education Perspectivefor Oklahoma, 2002-2003 • All Local Education Agencies (LEAs) in the State of Oklahoma participate in Title I programs. • There are 541 total LEAs • 44 LEAs with a student enrollment of 2,500 or greater • 201 LEAs with a student enrollment between 499 and 2,500 • 296 LEAs with a student enrollment of less than 500

  3. Oklahoma Demographics, 2003 Total school districts: 541 Independent (K-12): 430 Elementary (K-8): 111 Total School Sites: 1791 Teachers: 47,259

  4. Oklahoma Demographics, 2003 Average Daily Membership: 618,358 Special Education (K-12): 13.6% 83,812 Alternative Education (K-12): 2.5% 15,312

  5. Oklahoma Demographics, 2003 American Indian/Alaskan: 17.9% Asian/Pacific Islander: 1.5% Black/Non-Hispanic: 10.9% Hispanic: 7.0% White Non-Hispanic/Other: 62.6%

  6. State Standards, Assessments & Accountability Timeline 1990 - Oklahoma Education Reform and Funding Act of 1990 1991 - Initial development of core curriculum with statutory requirement for review and revision every 3 years

  7. State Standards, Assessments & Accountability Timeline 1992 - Legislation mandated the development of Criterion-Referenced Tests (CRTs) in seven (7) content areas for Grades 5, 8, and 11 1993- First review and revision of core curriculum, Priority Academic Student Skills (PASS) 1995- Implementation of CRTs in Grades 5, 8, and 11

  8. State Standards, Assessments & Accountability Timeline • PASS revisions – 1997, 2000, 2003 • Revisions have provided more specific, detailed and clear standards • PASS includes the Oklahoma School Testing Program (OSTP) “blueprints” for the CRTs

  9. State Standards, Assessments & Accountability Timeline 1989 - Low-performing, high challenge schools identification process adopted in state law; includes sanctions and technical assistance 1999 - Academic Performance Index (API) adopted in state law; includes API Awards

  10. Oklahoma’s Accountability System Proficiency for all Content and Performance Standards Sanctions and Rewards To guide instruction To promote a climate of change Decision Rules Assessments Additional Indicators To measure achievement

  11. Validity Questions for Oklahoma’s Accountability System Do our sanctions and rewards promote change towards intended outcomes? Do our decisions rules accurately identify schools? Do our assessments and performance indicators measure performance standards (i.e. achievement)? Do our standards help guide instruction?

  12. Key Elements in Determining Validity and Reliability • Ensure that the system has provided the intended outcomes. • Conduct research on additional information to corroborate findings. • Analyze design and implementation of each component of the system. • Conduct Analysis on several levels.

  13. Sources of External Evidence • Studies and research using additional indicators to corroborate findings or results of the system. • Surveys on attitudes and opinions. • Outside reviews for components of system.

  14. Sources of Internal Evidence • Ongoing review of policies and procedures. • Data analysis for various levels such as State level, district level, and for particular types of schools. • Evidence of Quality Control measures and Data Audits. • Trend analysis over time.

  15. Oklahoma’s Steps Towards ValidationStudies and Reviews • PASS - external reviews have included: • 2001-02: Achieve, Inc. • 2001-02: Southwest Educational Development Laboratory (SEDL) • 2001-02: Technical Issues in Large Scale Assessment (TILSA) • 1997-2003: “Quality Counts” report cards by Education Week

  16. Studies and Reviews • TILSA Review: Curriculum and Assessment Alignment Study (2001-02) with • Dr. Norman Webb, University of Wisconsin • Results of the study enabled Oklahoma to conduct more in-depth alignment of standards and assessments • Provided common language to facilitate communication in building curriculum and designing aligned assessments

  17. Studies and Reviews • Research Study (2002-03) by • Dr. John Poggio, University of Kansas • Study to determine levels of cognitive complexity of multiple choice test items

  18. Studies and Reviews • Implementation Survey (January 2002) • After initial implementation of the API an attitude and opinion survey was collected. • Additional surveys specifically on Adequate Yearly Progress (AYP) will be collected in October 2004.

  19. Studies and ReviewsOngoing Study • Study on School Improvement Schools • Collecting additional local LEA assessments from additional grades. • Purpose: • To corroborate the results of the decision rules. • To look at trends of schools being identified as school improvement .

  20. Grants and Projects • No Child Left Behind (NCLB) • Enhanced Assessment Grant • Oklahoma selected as lead state in collaborative curriculum/assessment alignment project with Council of Chief State School Officers (2003) • Continues the alignment work done in 2001 • Includes vertical alignment of curriculum and assessments (Grades 3-8) • Includes alignment of special education alternate assessments

  21. Grants and Projects • Project with Council of Chief State School Officers (SCASS) - 2003 • Develop survey tools to determine the level of implementation of a state’s standards-based language arts curriculum • To provide support for teachers in meeting NCLB requirement to increase student achievement

  22. Policies A few examples of changes or development as a result of reviewing the system. • Developed policy for appeals of Adequate yearly progress. • Developed a Technical Advisory Committee to review and offer expertise in regards to technical issues of the State testing program. • Implemented new rules and regulations regarding data audits and security.

  23. AYP Statistics • 367 (20.5%) of Oklahoma Schools did not make AYP. • 198 (36.5%) of Oklahoma School Districts did not make AYP. • 1.7 is the average number of subgroups that did not make AYP by school. • 1.5 is the average number of subgroups that did not make AYP by district.

  24. AYP Statistics • Total number of subgroups not meeting specific performance targets, ordered from greatest to least. • Reading target • Math target • Additional Indicator (attendance or graduation) • Participation target

  25. AYP Statistics • Total number of subgroups not meeting specific performance targets, ordered from greatest to least. • Reading target • Math target • Additional Indicator (attendance or graduation) • Participation target

More Related