Analyzing the content of caap

1. Learning College Conference February 26-27, 2009 Analyzing the content of caap

2. ACT product Use four modules college wide Each region gives only 2 of the 4 Rotation schedule ensures that regions administer a different pair each academic year Required of all associate degree graduates Administered in capstone courses Collegiate Assessment of Academic proficiency (cAAP)

3. To measure 4 of the College’s 8 general education outcomes • Demonstrate critical and creative thinking. • Apply basic scientific concepts in a variety of settings. • Communicate effectively in written, oral and symbolic forms. • Exhibit quantitative literacy. Purpose of CAAP

4. To identify areas of strength and weakness as compared to graduates of other public community colleges Results reviewed by curriculum committees, regional academic officers, central academic affairs staff To inform changes designed to improve student learning Purpose of CAAP

5. Descriptive statistics Mean scores by module Mean scores by subcategory Percentile ranks Comparison of mean scores to national norms Level of effort student reported giving the module Mean scores and level of effort by region Results currently reported

6. Enable analysis of specific content areas: • Writing Skills • Punctuation • Basic Grammar and Usage • Sentence Structure • Strategy • Organization • Style CAAP Content analysis reports

7. Mathematics • Prealgebra • Elementary Algebra • Intermediate Algebra • Coordinate Geometry • College Algebra • Trigonometry CAAP Content analysis reports

8. Science • Analysis • Generalization • Understanding • Critical Thinking • Analysis of Arguments • Evaluation of Arguments • Extension of Arguments CAAP Content analysis reports

9. Students (ITCC and nationally) are classified into proficiency groups • Top 25% • Middle 50% • Bottom 25% • For each student, percentage of items answered correctly is calculated • For each proficiency group, students’ percent-correct scores are averaged and displayed using bar graphs act’s Process

10. Writing skills: Punctuation http://www.act.org/caap/tests/writing.html

11. Reports differences in percentages between Ivy Tech and national groups • <5% difference considered negligible • 5%-10% difference considered moderate • >10% difference considered substantial • Negative differences indicate areas where Ivy Tech students “had more difficulty with content category items than did the normative group” • Positive differences indicate that Ivy Tech students found items easier than did the norm Comparisons and differences

12. Writing skill comparison N=2627

13. Divide into teams and identify a “reporter” • Take results of one module • Discuss and determine the following: • In relationship to students at other community colleges, where are our students’ strengths? ….where are their weaknesses? • What should the College do to improve students’ ability in the content area being assessed? • What can your program do to improve students’ ability in the content area being assessed? Group tasks – 15 minutes

14. In relationship to students at other community colleges, where are our students’ strengths? ….where are their weaknesses? • What should the College do to improve students’ ability in the content area being assessed? • What can your program do to improve students’ ability in the content area being assessed? Reports

15. Cherry Kay Smith Executive Director of Academic Policy and Assessment 317.916.7810 csmith@ivytech.edu Analyzing the content of caap