State Assessment Systems: Issues and Options

1. State Assessment Systems: Issues and Options David T. Conley, Ph.D. Professor University of Oregon Director Center for Educational Policy Research

2. Purposes of State Assessments • A state assessment can have multiple purposes • School accountability • Student accountability (exit requirement) • System monitoring and policy decisions • Performance feedback to improve teaching • College placement or admission • Most state assessments end up addressing multiple purposes • Some end up with confused purposes or no one clear purpose

3. The Power Equation of State Assessments • State assessments represent an exercise by the state of its power over local school systems • State constitutions grant states control over subsidiary governmental units, such as school districts • However, long traditions of local control come into conflict with this exercise of state authority in the educational policy arena • State assessments ultimately become a compromise between achieving state goals and incurring the political costs of exercising state power to intervene into the functioning of local schools

4. What Is Feasible? • State assessments should meet the following criteria: • Be manageable by school districts • Be consistent with state goals • Be taken seriously by educators and students • Lead educators and students toward desired behaviors • Be reasonable in cost, technical requirements and time • Meet technical adequacy standards for assessments

5. The Reliability-Validity Tradeoff • Generally, assessment is a tradeoff between reliability and validity • The more standardized the assessment, the better the reliability and the more limited the validity • The more directly a reflection of actual classroom work, the better the validity and the more challenging the reliability • Americans tend to value reliability above all else • Other nations favor validity

6. Where Are We in Oregon? • Oregon assessment, as originally designed, is admirable in many respects • Contained mix of methods • Multiple choice, performance tasks, work samples • Has long track record • Has online component (TESA) • Appears to be reasonably well accepted by schools • Meets NCLB requirements

7. Where Are We in Oregon? • However, significant differences exist from elementary to secondary levels • Lower scores, less concern with the assessments • Some schools take them seriously, others appear less concerned with flat scores • While scores can conceivably be used by higher ed (via PASS), they do not appear to be used in this fashion by many students • Legislature seems to need a primer on the system each session

8. Considering Options • Note that there are tradeoffs with each option- there is no one best option • Development and implementation costs are high for any new system or substantial modification • Oregon educational system does not seem particularly ripe for innovation or significant change at the moment

9. Defend Current System • This is sensible to do for many reasons • The current system aligns well (best?) with state standards • It is institutionalized and runs smoothly • More educators appear to be more capable of using test data for instructional decisions • However, it does leave the state exactly where it is currently, with an assessment that is largely in search of a purpose (beyond meeting NCLB requirements)

10. Revise Current System • Augment the test to include items that enable the test to be used for college readiness feedback and perhaps limited college placement purposes • Attach stakes to current tests so students and schools take them more seriously • E.g., compensatory scale composed of GPA and test scores, with minimum score required to avoid mandatory remediation or some other mild sanction

11. Adopt a National Test Tied to College Admission or Placement • ACT system (Explore, PLAN, ACT) or PSAT/SAT • Will be taken more seriously by students, teachers • Allows national comparisons of Oregon students • Is relatively efficient to administer • Does not necessarily cover state standards as well as state assessment (this needs to be documented, however) • May lead to more students considering college, based on IL and CO experiences

12. Adopt National Tests • Advanced Placement incentives or subsidy • IB subsidy • Work with postsecondary institutions to use data from these tests to recruit more students, award scholarships and credit, and place students

13. Replace or Augment State Test with End-of-Course Exams • Key benchmark courses such as Algebra II, Biology, 11th grade English • Creates a better balance between reliability and validity • Development costs, implementation complexity can be high • Can align highly with state standards • Can provide diagnostic feedback to students, teachers • Eventually requires specifying curriculum for tested courses, something that will be difficult for and alien to the state

14. End-of-Course Exams • Can address content knowledge and essential skills • Help limit curriculum inflation and drift • Can be made a component of the course grade • Can be combined in a compensatory fashion with other measures • Some efforts are underway nationally to develop consistent end-of-course exams

15. Develop School-based Assessment System • Can be collection of evidence or extended application-type project • Data collected over time or on a one-time basis, or both • Best system for providing diagnostic feedback to students • Can be well aligned with state standards • Requires local buy-in and a specific mechanism to guarantee rater consistency across schools

16. School-based Assessments • Senior year review • Student must demonstrate competency in key metacognitive skills such as logical and analytical thinking, problem solving, interpretation, teamwork • This can be done through a variety of means to accommodate college-bound and non college-bound students • No student is denied a diploma, but the information is used to judge schools and provide students with a better assessment of their current ability levels in these key capabilities for future success in college and career

17. School-based Assessments • Collection of evidence • Builds upon current work sample requirement • Links with PASS • Can be incorporated into a senior review • Can be used to provide ongoing gauge of college readiness for students and parents • Postsecondary ed can align entry-level courses with collection performance levels

18. Overall Criteria to Consider • Is system’s goal to gauge basic skills or highest achievement? • Is system to inform educators or policy makers? • Will system have any effects, consequences, or incentives for test-takers and for educators? • Does system achieve or promote key state goals for education?

19. Thinking about a System • What are the state’s high school exit standards? • What is it students should know and be able to do upon graduation? • How close are these to college readiness standards? • What does performance look like at 8th and 10 grade for students who will be ready to meet 12th grade exit standards? • How can 8th grade assessments also be “exit” measures • What assessment system generates this information?

20. Oregon Challenges • Balance of state influence-local control • What purposes should local control serve? • Lack of articulation mechanisms • Joint boards notwithstanding, state has few ways in which the three sectors work in a coordinated fashion • Funding • As always, do more with nothing • No 12th grade exit standards • CAM had career-related standards, but no content standards • Weak connections between assessment system and classroom practices or (state) accountability system