1 / 14

Pilot Study of the CAAP Critical Thinking Test April 27, 2005

Pilot Study of the CAAP Critical Thinking Test April 27, 2005. Lanette Raymond Research Associate, Suffolk County Community College. CAAP Test Description. 32 item multiple choice test Designed for use with college students Administered within a single class meeting,

charlottemiller
Download Presentation

Pilot Study of the CAAP Critical Thinking Test April 27, 2005

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pilot Study of the CAAP Critical Thinking Test April 27, 2005 Lanette Raymond Research Associate, Suffolk County Community College

  2. CAAP Test Description • 32 item multiple choice test • Designed for use with college students • Administered within a single class meeting, • Appeared relatively credible in an in-class administration protocol • Provided documentation of reliability and validity across community college populations

  3. CAAP Subscore Customization • CAAP Critical Thinking test contents closely match the SUNY CT learning objectives • Variance in the way the results are reported • ACT developed a customized report for these sub-scores showing normative comparisons against ACT national community college data. • ACT provided the student data files to SCCC, for further analysis of this data

  4. Administration and Sample • Fall, 2004 • Administered in-class to 154 SCCC students in 7 general education courses • Predominately white (77%) • Traditional age (60% 20 years old or younger, 25% between 21 and 25 years old) • 50% male, 50% female • Mostly sophomore status (46%) • Fulltime enrollment (85%)

  5. Student Motivation • No motivational tactics were employed • CAAP-CT instrument included an item that addressed students' self-reported motivation levels • One-third of students (n = 52) did not respond to the motivation item • 5 students indicated that they “gave no effort” (n = 1) or “gave little effort” (n = 4) to the assessment test.

  6. Student Motivation • Lower motivation results in less optimal performance • Less motivated students’ scoresare less reliable and less valid • Reporting sample is based on data from the 97 students who reported moderate to best effort • The reliability coefficient (calculated with the data from the original 154 tests) for objective 1 (26 items) is within acceptable range (alpha = .80) • Due to the small number of items (6 items) contributing to objective 2, its reliability coefficient is much lower (alpha = .49).

  7. Results • Confirmatory factor analysis substantiates the utility of the CAAP-CT test as a measure of 2 separate but related sets of critical thinking skills based on the 2 GEAR learning objectives

  8. 2-factor model of Critical Thinking based on the GEAR Objectives

  9. Results • All of the items loaded well onto their respective factors, with item 1 being only slightly below 1.96 (at 1.81). • The model shows an excellent fit to the data (χ2(463) = 466, p = .46, CFI = .94), providing additional context validity to the assessment.

  10. ** Data for item 20 should be listed under objective 2

  11. Standards Does not meet standard 59% or less Approaches standard 60% - 69% Meets standard 70% - 79% Exceeds standard 80% or more

  12. Standards

More Related