1 / 11

One State’s Implementation of Computer-based High-stakes Assessments in Public Education

One State’s Implementation of Computer-based High-stakes Assessments in Public Education. Sarah Susbury, Director Office of Test Administration, Scoring & Reporting Virginia Department of Education. Conceptual Framework and Methods.

etenia
Download Presentation

One State’s Implementation of Computer-based High-stakes Assessments in Public Education

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. One State’s Implementation of Computer-based High-stakes Assessments in Public Education Sarah Susbury, Director Office of Test Administration, Scoring & Reporting Virginia Department of Education

  2. Conceptual Framework and Methods • Participant-observer case study of state perspective (implementation, assessment-related, technology-related, organizational, financial, political) • Interviews with state-level individuals associated with the Web-based Standards of Learning Technology Initiative • Review of all existing documentation (legislative documents, procurement documents, project plans, status reports, risk mitigation reports, meeting minutes, memos, emails, etc.) • Constant comparative methodology with procedural additions (Glaser, B. G., & Strauss, A. L. (1967), Lincoln, Y. S., & Guba, E. G. (1985)Maykut, P., & Morehouse, R. (1994))

  3. Findings • Technology Initiative with an assessment focus • Proof of Concept or Demonstration Phase • Existing high-stakes assessment program transitioned to computer-based program • Leadership and multi-level organizational change • Phased Implementation with change management • Implementation models for paper/pencil versus computer-based high-stakes testing

  4. Conclusions • Pairing technology and assessment • Supporting technology readiness and assessment readiness • Phased implementation overall and at different school levels • Change management • Leadership • Training and communication

  5. One successful state’s perspective on factors that affect the extent of adoption of technology-enabled assessments for K12 statewide accountability • There is a global need for assessments to measure skills that cannot be measured with paper and a pencil but the practical challenges of school capacity make the ideal model of simultaneous testing for all students in a statewide computer-based accountability test almost impossible. • There are factors associated with these challenges that could be addressed by experts once they are known and understood; this study aims to define these factors. Diana W. Cano Rutgers, The State University of New JerseyGraduate School of New BrunswickNew Brunswick, New Jersey

  6. Conceptual Framework and Methods • What does adoption mean in the context of testing and technology? • General adoption. The innovation adoption cycle is most often used by technologists to understand how organizations go about making changes that are innovative. • Adoption by middle level of a federal system. Three categories of factors interact with each other in the management of our policies on education assessments: district and school capacity; state policy; national policy. • What are the state factors that influence adoption? • There are four broad categories: standards and accountability; school capacity; teacher quality; network strength. • Research has shown that the setting of standards leads to better student performance on accountability assessments. • There are negative effects of high accountability on a school’s capacity to improve its academic performance. • Teachers play a critical role in the successful adoption of innovation. • A school’s adoption of technology may also indicate a willingness to adopt innovative educational solutions. • Using social network analysis, studies have been done to describe the characteristics of a policy network that enable or inhibit innovation adoption. • Methods • Sampling. One state was purposely chosen because of their extensive experience with computer-based testing. • Recruitment. Interviewees were the state’s director of testing and two team members. • Interviews. These interviews lasted approximately 40-60 minutes and were digitally audio-recorded.

  7. Findings • How do communications and autonomy factors constrain or facilitate adoption? • communications • “If they don't do well, then we're not going to do well” • “we'd set them up, and we'd be right there.” • “If there were seats and they wanted to come, we let them come. And we figured the more information we could get out there, the better.” • “what I was hearing from someone in the field and I could share that with someone else. So that, that flow of information was important.” • school migration path • “beef up your network” and “grown into the testing” and “phased it in” and “building a capacity to do it” • “they could choose the subject. It didn't matter to us.” • How do the factors of national policies, state policies and school capacity constrain or facilitate adoption? • state policy, school capacity and state organizational structure and dynamics factors influence the adoption of TEAs • state policy and school capacity are dependent upon each other • communication between these stakeholders affects adoption

  8. Conclusions • Key factors affecting adoption of TEAs for statewide accountability • State and district communications • Schools defining their own migration schedule • State policy and school capacity dependencies • State policy, organizational structure and school capacity • States use the Internet to gather technical information • The data illustrate that the factors most affecting innovation adoption are rooted in state-level bureaucrats respecting the districts through substantive communications and autonomy over how they can accomplish the migration from paper-based tests to TEAs. • This conclusion is especially important as states rely on two national consortia to set timelines for implementation, regardless of district readiness and school capacity. • PARCC states will be expected to be online in the 2014-2015 school year • The state has spent years successfully migrating all schools to TEAs through state leadership and the acknowledgement that this capability needs to be built locally at a reasonable rate and funded accordingly. • They are 100% online this year (exceptions only for accommodations) • They have a strong network of district leaders who manage year-round testing • Recommendations for Future Research • Further research on factors influencing adoption • Further study within the state from other perspectives in the districts • Additional research between states as a cross-state comparison I am recruiting states… please let me know if you are interested!

  9. Showcase: From Research to Reality: What States Need to Do to be Successful with K12 Adoption of Computer-Based Accountability Assessments Jennifer Dugan Minnesota Department of Education “Leading for educational excellence and equity. Every day for every one.”

  10. Online Participation There are ~63K students/grade. education.state.mn.us

  11. Next Steps education.state.mn.us • Scheduling is main concern impacting current participation in online testing • Pursue delivering tests on mobile devices with smaller screens; districts changed their approach from their original advice • What is best path for items that currently utilize all of the 11.6 inch screen real estate in moving to 9.7 inch screen? • Planning for increased need for on-site technical assistance

More Related