Critical Thinking Test Selection - Process & Evaluation

1. Critical Thinking Test Selection - Process & Evaluation Frances Dearing, PhD Assistant Dean of Assessment, Academic Support and Placement, Westchester County Community College

2. Test Selection & Issues Test Selection • Critical Thinking • Standardized Test Option Issues • Lead time • Community College student population • Sample size / course(s) selection • Administrative/Faculty Senate approval • Test administration • Interpreting results

3. Lead Time • Request (SUNY) to fulfill Critical Thinkingassessment requirement AY 2004-2005 • No feasible Critical Thinking assessment plans available Fall 2004 (Plan due to SUNY 12/17/04) • No feasible Critical Thinking assessment plans available Fall 2004

4. Lead Time • Formed Critical Thinking Faculty Committee • English/Reading • Mathematics • Computer Programming • Recommendation to review a standardized Critical Thinking test

5. Reviewed: • California Critical Thinking test • Test of Everyday Reasoning • ACT CAAP Critical Thinking test

6. Recommendations of Committee • CCTT and TER inappropriate for a community college student population • CCTT deemed too difficult for Westchester Community College students • TER less difficult, but focused on logic versus Critical Thinking • Both CCTT and TER were normed against four-year college and graduate students

7. CAAP Critical Thinking Test Selection Rationale: • Only test normed against two-year public college students • Format – four passages with a series of multiple choice questions • numbered references for reading passages

8. CAAP Critical Thinking Test Selection Rationale: • Measurement available for each SUNY Critical Thinking learning outcome • Test used by several other SUNY institutions • Test does not have to be completed to obtain an accurate assessment

9. Student Sample • Target sample – 400 – 500 students • Actual sample – 372 students

10. Sample Size/Course Selection Course(s) • ENG 102 Composition & Literature II Rationale: 1. Using Composition and Literature II avoided duplicate sampling. The course itself was not part of a current assessment plan. 2. Since most students eventually must take Composition & Literature II, a broad test sample was provided.

11. Sample Size/Course Selection Rationale: 3. Critical Thinking is infused throughout the General Education curriculum. Composition and Literature II is just being used for testing purposes. 4. Westchester Community College will need an unduplicated sample to norm for Critical Thinking by AY 2005-2006 for SUNY’s strengthened Campus-Based assessment.

12. Student Motivation • No incentive was provided to ensure student motivation.

13. Administration & Faculty Senate Approval Administration Concerns: • Cost • Logistics of test administration • Faculty and student motivation Faculty Senate Concerns: • Use of standardized test would preclude future use of a locally-developed instrument

14. Test Administration • Seventeen sections of Comp Lit II • one week period • Four proctors • Required professors to be present • No extra credit allowed by English Dept. • Some slight student confusion with test form • Most students required 45-50 minutes for exam completion • ESL and some students required a longer time period

15. Interpreting Results • Work in Progress • Scoring Categories • Based on a mean of 19.03 and a standard deviation of 6.30