South Carolina Arts Assessment Program (SCAAP)

1. South Carolina Arts Assessment Program (SCAAP) South Carolina State Department of Education In Collaboration With Office of Program Evaluation College of Education University of South Carolina

2. SCAAP Objectives • Adhere to accountability act • Align curriculum, instruction, and assessment • Construct standards-based assessments • Encourage implementation of standards • Provide feedback about students’ achievement in dance, music, theatre, & visual arts

3. Dance & Theatre Assessments • Dance & Theatre Assessment • 1st pilot-test conducted in 2005 • Only administered 2 multiple-choice forms due to lack of funding • 2nd Pilot test in 2007 • 2 multiple-choice test forms • 45 items with interpretive materials • 2 Performance tasks • Dance: 2 video recording of student performances • Theatre: 1 voice recording and one video recording of student performances

4. Testing with Entry-Level Students • 2 theatre and 2 dance web-based multiple-choice forms • 2 theatre and 2 dance performance tasks • Score performance tasks using rubrics

5. Theatre Assessment • Students demonstrate their knowledge of and skills in • theatre vocabulary • evaluation of performance problems • performance of theatre skills • Test format • Two 45-item multiple-choice test forms • Two performance tasks

6. Table of Specifications: Theatre

7. Dance Assessment • Students demonstrate their knowledge of and skills in • dance vocabulary • types of dance • evaluation of performance problems • performance of dance skills • Test Format • Two 45-item multiple-choice test forms • Two performance tasks

8. Table of Specifications: Dance

9. Performance Tasks • Theatre • Improvise a monologue • Express a situation in pantomime • Dance • Demonstrate a range of dance movements • Choreograph and perform a dance study

10. Test Administrator Training • Two test administrators assigned for each school • Multiple training sessions held across the state • Test Administration manual • Mass register students • Record performances • Timeline and checklists • Hands-on demonstrations • Regular e-mail updates

11. Procedure • Administer web-based multiple-choice test forms in small groups • Theatre performance tasks • Administer to individual students • Digitally recording students’ voices • Video-record students’ performances • Dance performance tasks • Administer to individual students • Video tape student performances

12. Required Material • PC Computers • Internet Explorer 5.0 and higher • Sound card • Headphones - provided • Headset – provided (Theatre) • CD players • Camera & Tripod - provided • Timers – provided • Flash Drive - provided

13. Testing Time • Flexible Scheduling March 15th-April 15th Estimated Time • Multiple-Choice Test Form: • 60 minutes • Theatre Performance Tasks: • Task 1: 5 minutes/student • Task 2: 10 minutes/student • Dance Performance Tasks: • 10 minutes/students

14. Participants • Schools • Pilot testing in schools that have both theatre and dance programs • Introductory Students • Students should have at least one semester of theatre/dance, but no more than one year of theatre/dance

15. Data Analysis • Multiple choice • Item analysis • Descriptive statistics • Test Reliability • Performance tasks • Inter-rater reliabilities • Reported school level results • Descriptive statistics by test forms

16. Web-Based Rating • Benchmark student responses • Validation committee • Rubrics • Theatre: Holistic rubrics/Analytic rubrics • Dance: Holistic rubrics • Raters training session • Practice test • Qualifying test

17. Reporting Results • Web-based reports on individual school results • for each test form • for each rubric criteria individual performance tasks • percent correct for each item identified by standards • Access IDs for principals

18. Overview of MC Results

19. Multiple-Choice Results

20. Multiple-Choice Results

21. Performance Task Results

22. Research & Presentations • Assessing the reading of music notation (SCEPUR, 2004) • Comparing arts achievement to English Language Arts and Mathematics Achievement in Arts Education Reform Schools (SCEPUR, 2005) • Rating design and score dependability of music performance tasks (SCEPUR, 2005) • Evaluating the program characteristics of arts schools with disparate achievement levels (SCEPUR, 2006) • Comparing the dimensionality structures of music and visual arts multiple-choice assessments (SCEPUR, 2006) • An exploratory study of the dimensionality structure of a music multiple-choice assessment (AERA, 2006) • Determining the effectiveness of the rater monitoring procedure (SCEPUR, 2007 & AERA, 2007) • Comparing student ability estimates using IRT and MIRT models (SCEPUR, 2007) • Investigating Arts Programs & Implementation Strategies for Infusing Arts into Curriculum (AERA, 2007)

23. Contact Info Dr. Ching Ching Yap South Carolina Arts Assessment Program, Project Director Office of Program Evaluation College of Education, University of South Carolina Columbia, SC 29208 E-Mail: CCYap@gwm.sc.edu Phone: 803-777-3459 Fax: 803-777-8838