Tracking College Students Over Time: Using CIRP & CSS Data for Longitudinal Assessment

1. Tracking College Students Over Time: Using CIRP & CSS Data for Longitudinal Assessment Victor Sáenz (UCLA) John Pryor (UCLA) & Gavin Henning (Univ. of New Hampshire) Wednesday, May 17th, 2006 AIR Forum Chicago, IL

2. Presentation Goals • Overview of CIRP Freshman Survey • Introduce the CSS survey as a longitudinal follow-up to the CIRP Freshman Survey • Discuss how institutions can utilize HERI longitudinal data to assess various aspects of the first-year experience • Share descriptive findings from the 2005 CSS national aggregate data • Institutional perspective on using HERI longitudinal data

3. Higher Education Research Institute CIRP Cooperative Institutional Research Program Funded Research Freshman Survey YFCY CSS • Atlantic Philanthropies • Templeton Foundation • National Institutes • of Health Faculty Survey

4. CIRP, CSS, & the I-E-O Model Environments CSS (e.g., place of residence during college, interactions with peers and faculty, curricular and co-curricular experiences) Inputs CIRP Freshman Survey (e.g., academic performance in high school, financial concerns prior to college entry, expectations for college, degree aspirations, self-concept in high school) Outcomes CSS (e.g., satisfaction with college, retention, post-college plans)

5. Using CIRP & CSS Data to Enhance Campus Assessment Efforts: Methodologies • Descriptive analyses with your campus data • Comparative analyses • Measures of association • Longitudinal analyses • Factor analyses • Multivariate analyses

6. Using CIRP & CSS Data to Enhance Campus Assessment Efforts: Purposes • Self-study reports • Retention studies • Recruitment issues • Examining group differences among students • Strategic planning • Presentations to various constituents • Creation of a student information system • Measuring student development & institutional impact

7. CIRP Freshman Survey • Largest and longest-running national study of American college students…2006 is 40th Anniversary • Initiated in 1966 at the American Council on Education; Housed at HERI (UCLA) since 1973 • Administered annually to over 400,000 incoming freshmen at more than 600 colleges and universities nationwide • Since 1966: 12 million students; 1,800 institutions

8. CIRPFreshman Survey Items • Demographic Characteristics • Expectations for College • High School Experiences • Degree Aspirations & Career Plans • College Finances • Attitudes, Values, & Life Goals • Reasons for Attending College • FRESHMAN TRENDS (1966 – present)

9. CIRP FS Trends: Entering College Freshmen

10. CIRP FS Trends: Entering College Freshmen

11. CIRP FS Trends: Entering College Freshmen

12. CIRP FS Trends: Entering College Freshmen Men Women

13. CIRP Freshman Trends • Socio-historical context • Consistent trends over time • Changing student profile over the last 40 years (e.g, values, reasons for going to college, etc.) • What about students’ college experiences?

14. CSS • HERI follow ups since 1967 • Sponsored research • College Student Survey • Launched in 1993 • Allowed institutions to participate on their own timetable • Most use CSS as an exit survey for seniors

15. CSS Major Themes • Academic and social adjustment • Sources of satisfaction and dissatisfaction • Academic, residential, and employment experiences • Plans for the next academic year • Patterns of behavior • Life goals • Self-concept and feelings of personal success

16. 2005 CSS* • 116 Colleges and Universities • 17,929 students • 85% took the CIRP Freshman Survey in 2001 • 62% Female • 38% Male *Data are unweighted.

17. Results: 2005 CSS • Higher Satisfaction • 4 out of 5 would make the same choice over again to attend their college • Class size (89.3%) • Quality of Instruction (88.0%) • Major courses (86.5%) • Sense of community (76.9%) • Ability to find faculty/staff mentor (67.5%)

18. Results: 2005 CSS • Higher Satisfaction • Internet access (81.2%) • Computer facilities (73.3%) • Opportunities for Community Service (59.2%) • Leadership opportunities (59.6%)

19. Results: 2005 CSS • Lower Satisfaction • Student Housing (54.0%) • Campus Health Services (43.7%) • Job Placement (36.4%)

20. CSS (by racial group):Overall Satisfaction w/College Experience

21. Students felt successful in…

22. CIRP vs. CSS(longitudinal results) • How do students change over their years of college?

23. Longitudinal Expectations/Reality: CIRP Freshman Survey and CSS

24. Longitudinal Expectations/Reality: CIRP Freshman Survey and CSS

25. Longitudinal Results: CIRP Freshman Survey & CSS

26. Longitudinal Results: CIRP Freshman Survey & CSS

27. Using CIRP – CSS at University of New Hampshire

28. Tradition of Using CIRP Freshman Survey at UNH • UNH was pilot school in 1966 • Data for 16 years going back to 1966 • Have administered biennially since 1999 • Have used data to better understand incoming students • Haven’t been using data for any type of evaluation or outcomes assessment • UNH is lacking specific outcomes

29. College Student Survey at UNH • In 2001 UNH began to look more at student outcomes • Focus on learning, not just satisfaction and general programmatic outcomes such as attendance • Viewed CSS as a tool for that assessment

30. College Student Survey at UNH • Compared CSS to NSSE and chose CIRP/CSS • High response rate for CIRP • Control of pre-college factors with CIRP • Great deal of question overlap between CIRP/CSS and NSSE • CIRP/CSS was cheaper for UNH • Ability to pre-/post-test with same panel of students, not just sample from same cohorts • Need SSNs • 30 extra questions on CIRP/CSS to customize for our campus

31. College Student Survey at UNH • CSS administration challenges • Inadequate mail/email addresses in 2001 • Tried to have departments administer in capstone experiences and other activities where seniors were together • Only three departments participated • Lack of intentional purpose for using data once collected • UNH academic plan is general not specific enough to evaluate against • We are in the process of identifying institutional student outcomes

32. College Student Survey at UNH • CSS administration spring 2005 • Pilot test with 789 students who took CIRP in 2001 and that provided SSNs for linkage • Able to reach 763 of 789 students from this sampling frame • 97% coverage rate • 278 of 763 students responded • 36% response rate

33. Using CIRP/CSS Data at UNH • Understanding student characteristics • Needs assessment • Satisfaction survey of various campus services • Outcomes assessment • Benchmarking • Program evaluation • Strategic planning

34. Using CIRP/CSS Data at UNH • Understanding student characteristics • How they spent their time, behaviors, values, etc. and how this has changed over their college career • CIRP 2006: 27, 32, 33, 36, 38 • CSS 2006: 7, 8, 13, 15, 21 • Educational activities engaged in at college • CSS 2006: 7, 9

35. Using CIRP/CSS Data at UNH • Needs assessment • Areas of self-identified weakness • CIRP item 28, 39 • Satisfaction survey of various campus services • CSS 2006: 10, 18

36. Using CIRP/CSS Data at UNH • Outcomes assessment • Achievement of UNH student outcomes • Can use extra 30 questions • Can investigate relationships between variables to identify possible predictors for outcomes while controlling for pre-college inputs • Examining how students’ perceived skill level/traits has changed during college career • CIRP 2006: 31/CSS 2006: 23 • CSS 2006: 19

37. Using CIRP/CSS Data at UNH • Outcomes assessment (continued) • Understanding the perceived impact of faculty • CSS 2006: 24

38. Using CIRP/CSS Data at UNH • Benchmarking against other institutions on any variety of variables • Program evaluation • Can compare students in certain programs based on CSS variables, or change between CIRP and CSS on particular variables • Academic program • Co-curricular program

39. Using CIRP/CSS Data at UNH • Strategic planning • Helping to assess selected goals and strategies of the strategic plan • Increase opportunities for student engagement • CSS 2006: 7, 9, 13, 19 • Strengthen and facilitate sense of community • CSS 2006: 10, 18, 22, 24, 26

40. Using CIRP/CSS Data at UNH • Strategic planning (continued) • Helping to assess selected goals and strategies of the strategic plan • Actively develop multicultural competence and support diversity • Establish and maintain a healthy/safe environment for students and staff • CIRP 2006: 26/CSS 2006: 15 • CSS 2006: 26

41. Fostering use of CIRP/CSS at UNH • Commitment to assessment and evidence-based planning and decision-making • Concrete goals, strategies, and outcomes that can be evaluated • Marketing how the data can be used • Putting the data into practice for assessment, planning, and decision-making

42. For More Information: http://www.gseis.ucla.edu/heri/heri.html (310) 825-1925 http://www.gseis.ucla.edu/heri/cirp.htm http://www.gseis.ucla.edu/heri/css.html Victor Saenz vsaenz@ucla.edu John Pryor john.pryor@ucla.edu Gavin Henning gavin.henning@unh.edu