Assessing SUNY General Education Learning Outcomes with CAAP

2. AssessingSUNY General Education Learning Outcomes with April 27, 2005

3. Presenters • David Chadima Consultant, Postsecondary Assessment Services, ACT • Robert Ziomek Director of Education and Workforce Research Services, ACT • Frances Dearing Assistant Dean of Assessment, Academic Support and Placement, Westchester County Community College • Lanette Raymond Research Associate, Suffolk County Community College

4. Presentation Outline • CAAP Overview • SUNY Outcomes / CAAP Test Module Alignment & Report Customization • Critical Thinking Pilot Studies • Suffolk County Community College • Westchester County Community College • Q&A

5. What is CAAP? Collegiate Assessment of Academic Proficiency

6. The standardized assessment program from ACT that enables postsecondary institutions to measure, evaluate, and enhance the outcomes of their general education programs.

7. CAAP Features • Six independent, 40-minute test modules • Flexible, in-class administration • Comprehensive reporting at the cohort and individual student levels • Extensive user norms • Linkage to ACT, COMPASS or ASSET to measure “value added”

8. CAAP User Norms to

9. CAAP User Norms • National norms for each test module (and subscore) • Broken out by: • Institution type • Year of students tested • Institution ownership • SUNY system norms (2-year & 4-year if sufficient volume.

10. CAAP Test Modules

11. CAAP Test Modules • Five objective tests • Writing Skills • Reading • Mathematics • Science • Critical Thinking • One writing essay test

12. SUNY Learning Outcomes Knowledge and Skills Areas • Critical Thinking • Basic Communications (Written) • Mathematics

13. Outcomes/Test Alignment

14. Outcomes/Test Alignment

15. Outcomes/Test Alignment KEY ISSUE: • For each test module, provide CAAP subscores that map to SUNY learning objectives

16. Critical Thinking

17. Critical Thinking

18. Critical Thinking

19. Critical Thinking • 32 items / 4 passages • Subject matter: Social Sciences, Natural Sciences, Humanities • Formats: • Statistical Arguments • Experimental Results • Overlapping Position • Case Studies • Dialogues • Debates • Editorials

20. Critical Thinking Sample Question following passage It is obvious from Favor's speech that she believes which of the following? A. Most prescriptions are unnecessary. B. Senator Counter will oppose the bill. C. If the bill is passed into law it will greatly reduce the cost of all medical treatment. D. If the bill is passed the average costs for treatment of minor ailments would be reduced significantly

21. Basic Communication (Written)

22. Basic Communication (Written)

23. Writing Essay • Two independent prompts • 20 minutes each • Essays scored in a holistic manner • ACT can score • Local faculty can score

24. Writing Essay Sample Your college administration is considering whether or not there should be a physical education requirement for undergraduates. The administration has asked students for their views on the issue and has announced that its final decision will be based on how such a requirement would affect the overall educational mission of the college. Write a letter to the administration arguing whether or not there should be a physical education requirement for undergraduates at your college.

25. Writing Skills • 72 items / 6 passages • Subscores: • Usage / mechanics • Rhetorical skills • Spelling, vocabulary, and rules of grammar not tested

26. Writing Skills Specs Content Category% of Test # of Items Usage/Mechanics .44 32 Punctuation .08 6 Grammar .11 8 Sentence Structure .25 18 Rhetorical Skills .56 40 Strategy .21 15 Organization .14 10 Style .21 15 Total 1.00 72

27. Writing Skills Sample By the fourth day I had to face the truth: my body was slowly changing to becoming dough. So I tied on my running shoes and loped out to the main road in search of a five-mile route. A. NO CHANGE B. become C. being D. OMIT the underlined portion

28. Mathematics

29. Mathematics

30. CAAP Administration • Select test modules that match the needs of your gen ed program • 2-week test window • Convenient in-class administration • 40 minutes per test • Pencil & paper • Proctored

31. CAAP Administration • Return materials to ACT for scoring & reporting • Objective tests - ACT proprietary scanning • Essay tests - two ACT-trained scorers • Three week turn-around time

32. CAAP Reports • Standard Reporting Package • SUNY Subscore Reports

33. Standard Reporting Package Includes: • Institutional Summary Report • Student Score Reports (2) • Student Roster Report • Student Certificates of Achievement • Subgroup Reports (first 3 included free) • Summary of Student Motivation • Data files available

34. SUNY Subscore Report • Subscores map to SUNY learning objectives (CT) • Groups students by • top 25%  middle 50%  bottom 25% • Compares local student performance with state or national norms

35. SUNY Criteria Compliance YES • Will it directly measure student learning? • Will it measure the objective it is intended to measure? YES • Is the measure is reliable? YES • Does the plan include externally referenced norms? YES • Will the data that are reported be representative? YES • Is it capable of measuring value-added? YES

36. CAAP Resources • www.act.org/caap

37. CAAP Resources • CAAP manuals • Planning & Forms • Tech Handbook • Reporting & Research Services • Motivating Students

38. Contact: David Chadima ACT Postsecondary Assessment Services 500 ACT Drive P.O. Box 168 Iowa City, Iowa 52243-0168 Tel: (800) 294-7027 Fax: (319) 337-1790 E-mail: david.chadima@act.org