Michigan’s Experience Incorporating the ACT into a High School NCLB Assessment

1. Michigan’s Experience Incorporating the ACT into a High School NCLB Assessment Joseph Martineau, Director Office of Educational Assessment & Accountability Michigan Department of Education State Superintendent’s Next Generation Assessment Task Force December 2, 2008 Madison, Wisconsin

2. Why is Michigan Using the ACT? • Cherry Commission report • 25% of adults in MI have a bachelor’s degree or higher • Goal is to double that rate • College Entrance examination for all high school students is one component of the plan to achieve the goal

3. Why is Michigan Using the ACT? • Legislation enacted establishing • The “Michigan Merit Examination,” or MME • A $4,000 scholarship for students who take the MME • Three categories of eligibility • Not eligible • NON-valid score in reading, writing, math, OR science • Eligible after 2 years of successful post-secondary education • Maintaining at least at 2.5 GPA • Valid scores in reading, writing, math, AND science • Eligible for early disbursement (to pay for tuition in advance) • Proficient scores in reading, writing, math, AND science

4. Why is Michigan Using the ACT? • Legislation required the following components of the MME • College entrance examination • Bid process resulted in selection of ACT • Work skills examination • Bid process resulted in selection of WorkKeys • Social studies assessment • Compliance with NCLB • Means augmentation to round out alignment to Michigan’s high school content standards

5. What is the MME? • 3-day assessment • Day 1: ACT + Writing • Reading • English • Mathematics • Science • Writing prompt • Day 2: WorkKeys • Reading for Information • Applied Mathematics • Locating Information (starting Spring 2009) • Day 3: Michigan Augmentation • Mathematics • Science • Social Studies

6. Other States’ Statewide Use of ACT • Illinois is the only other state using ACT as a part of NCLB assessment • Maine is using the only state using the SAT as a part of its NCLB assessment • Other states use ACT statewide, but not for NCLB assessment • Colorado • Kentucky • West Virginia • Wyoming • Others…

7. What Counts Toward What?

8. Benefits • Rigor • Free college entrance examination for all students • Free career readiness certificate for all students • Double the submissions of ACT scores to Michigan universities • High quality program • High quality contractor • Significantly improving contractor relationships • Significantly improving state-needs focus of contractor

9. Challenges and Cautions • Accommodations • Limited number of vendors • Major changes in administration • Student motivation • Peer Review • Cost

10. Challenges and Cautions • Accommodations • Three types • ACT-allowed • College-reportable ACT scores • Career readiness reportable WorkKeys scores • State-allowed • No official ACT/WorkKeys score reports • Counted toward State/NCLB subject scores • Non-standards • No official ACT/WorkKeys score reports • Does not count toward State/NCLB subject scores

11. Challenges and Cautions • Accommodations • No ACT-allowed accommodations for ELL/LEP students at this point • No guarantee that IEP- or 504-designated accommodations will be approved by ACT • Several state-allowed accommodations for SWD not allowed by ACT • ACT indicates that studies will be carried out to evaluate whether some state-allowed accommodations will become ACT allowed

12. Challenges and Cautions • Limited number of vendors • College Entrance • ACT • College Board/ETS • Career Readiness • WorkKeys (ACT) • ASVAB (Military)

13. Challenges and Cautions • Limited number of vendors • Bargaining leverage is limited • Expect to be frustrated at times • Getting better, however • Expect high costs • Very little to no volume discount • Contract costs will inherently contribute to subsidizing substantial overhead of research-campus style operations of ACT or ETS • As competition is already severely limited… • Don’t write program (e.g. ACT, SAT, WorkKeys, ASVAB) into legislation • Make sure that all options are viable

14. Challenges and Cautions • Major changes in administration • Old test • Three-week administration window • Flexibility in environment • Flexibility in timing • State flexibility in determining validity • New test • Specific three days for initial • Specific three days for makeup • ACT-defined procedures, environment, timing • ACT-defined validity determination

15. Challenges and Cautions • Student motivation • ACT (not college bound?) • WorkKeys (college bound?) • Augmentation (is the incentive enough?) • Attemptedness criteria • Make sure students at least try on all three • Valid score criterion

16. Challenges and Cautions • Peer Review • Alignment, alignment, alignment • Cost implications • Time implications • Complexity of what counts toward what • Psychometric implications • 3-PL versus Rasch models • Perception of value • College entrance, versus • Work Skills, versus • Augmentation

17. Challenges and Cautions • Old test • $19/student for all five subjects • New test • $185/student, spring 2006 • Pilot test with standard setting • $77/student, spring 2007 • First operational, statewide administration • $102/student, fall 2007 • Retest for eligible 12th graders • $71/student, spring 2008 • Second operational, statewide administration • $121/student, spring 2009 projection • New contract

18. Contact Information Joseph Martineau, PhD Director Office of Educational Assessment & Accountability Michigan Department of Education 517-241-4710 martineauj@michigan.gov