Using CIRP Data for Accreditation, Assessment, and Accountability

Using CIRP Data for Accreditation, Assessment, and Accountability AIR Forum Kansas City, MO  June 4, 2007 John H. Pryor Victor B. Saenz Director CIRP Research Manger Cooperative Institutional Research Program Cooperative Institutional Research Program Higher Education Research Institute University of California at Los Angeles

Introduction • CIRP surveys have been used for decades to understand and illustrate the impact of college. • Can use the surveys as cross-sectional surveys, but the key advantage of CIRP surveys is that they are longitudinal.

Higher Education Research Institute CIRP Cooperative Institutional Research Program Funded Research Freshman Survey YFCY CSS • Spencer Foundation • Templeton Foundation • National Institutes • of Health Faculty Survey

Astin’s I – E – O Model Environments YFCY/CSS (e.g., place of residence during college, interactions with peers and faculty, curricular and co-curricular experiences) Inputs CIRP Freshman Survey (e.g., academic performance in high school, financial concerns prior to college entry, expectations for college, degree aspirations, self-concept in high school) Outcomes YFCY/CSS (e.g., satisfaction with college, retention, gains in college, post-college plans)

Reporting • Each institution receives • Institutional report with comparison groups • Raw data for analysis • Can order aggregate customized peer reports • New option of PowerPoint presentation

Accreditation • Provide evidence of assessment; in specific, issues of student achievement • Longitudinal design is the best way to assess change • Why? • You need to account for what the student brings to the institution in order to determine what the institution adds

Importance of Input • Degree Attainment Rates at American Colleges and Universities (DARCU) • CIRP Freshman Survey data (input) paired with registrar data on completion 4, 5, and 6+ years from entry (262 institutions). • “…two-thirds of the variation among institutions in their degree completion rates is attributable to differences in their entering classes..” • Astin and Oseguera

Retention • Your retention rate alone is not a good measure of institutional effectiveness • Comparing your actual retention rate with the expected retention rate is a good measure of institutional effectiveness • DARCU allows schools to calculate an expected retention rate using CIRP Freshman Survey data • 72 percent more of the variance is explained using CIRP Freshman Survey data in the calculation in place of institutional data (sex, race, HS grade, and SAT/ACT) • If actual retention rate is higher than expected, good • If not, not so good

Student Learning • YFCY and CSS can be used, with TFS as control or as stand alone • CSS measures of student learning • Satisfaction • Behaviors • Self-reported outcomes

Academic Programs • Since entering college, have you: • Enrolled in honors or advanced courses • Taken an ethnic studies course • Taken a women’s studies course • Failed one or more courses • Taken a remedial course • Participated in a study abroad program

Academic Programs • Participated in a study abroad program • Importance of • Improving my understanding of other countries and cultures • Helping to promote racial understanding • Keeping up to date with political affairs

Academic Programs • Participated in a study abroad program • Gains in • Knowledge of people from different races/cultures • Ability to get along with people of different races/cultures • Understanding of social problems facing our nation • Understanding of global issues • Foreign language ability

Student Satisfaction:Academic Life 2005 College Student Survey (CSS)

Academic Engagement • Since entering college, how often: • Worked on independent study programs • Performed community service as part of a class • Discussed course content with students outside of class • Studied with other students • Tutored anothercollege student • Had difficulty getting the courses you needed • Used/purchased notes from a professional service • Contested a grade • Asked professor for advice outside of class • Challenged a professor’s ideas in class

Academic Disengagement • Since entering college, how often: • Failed to complete homework on time • Felt bored in class • Came late to class • Missed class due to employment • Missed class due to other reasons • Fell asleep in class

Engagement with Faculty • Since entering college, how often: • Worked on independent study projects • Asked a professor for advice after class • Challenged a professor’s ideas in class • Been a guest in a professor’s home • Hours per week: • Talking with faculty during office hours • Talking with faculty outside of class or office hours

Engagement with Faculty • How often have professors at your college provided you with: • An opportunity to work on a research project • Encouragement to pursue graduate/professional study • Advice and guidance about your educational program • Help to improve your study skills • Feedback on your academic work (outside of class) • Emotional support and encouragement • A letter of recommendation • Intellectual challenge and stimulation • An opportunity to discuss coursework outside of class • Help in achieving your professional goals • An opportunity to apply classroom learning to “real-life” issues

Technology Behaviors • Since entering college, how often: • Used the Internet for research or homework • Received course assignments electronically • Turned in course assignments electronically

Gains • Compared with when you first entered college, how would you now describe your: • General knowledge • Analytical and problem-solving skills • Knowledge of a particular field or discipline • Ability to think critically • Knowledge of people from different races/cultures • Leadership abilities • Interpersonal skills • Ability to get along with people of different races/cultures • Understanding of the problems facing your community • Understanding of social problems facing our nation • Writing skills • Public speaking ability • Mathematical skills • Computer skills • Preparedness for employment after college • Preparedness for graduate or advanced education • Ability to manage your time effectively • Understanding of global issues • Foreign language ability

Diversity • Trustees see this as biggest student issue • Chronicle of Higher Education Survey of Trustees (May, 2007) • Basic enrollment data • CIRP versus institutional data • 32 total items directly concerning diversity

Diversity • Since entering college have you: • Taken an ethnic studies course • Taken a women’s studies course • Attended a racial/cultural awareness workshop • Participated in an ethnic/racial student organization • Had a roommate of different race/ethnicity

Diversity • Personal importance of: • Helping to promote racial understanding • Improving my understanding of other countries and cultures • Agreement with: • It is important to have laws prohibiting homosexual relations • Racial discrimination is no longer a major problem in America • Colleges should prohibit racist/sexist speech on campus • Same-sex couples should have the right to legal marital status

Diversity • Experiences with students of a different racial/ethnic background from your own: • Dined or shared a meal • Had meaningful and honest discussions about racial/ethnic relations outside of class • Shared personal feelings and problems • Attended events sponsored by other race/ethnic groups • Had intellectual discussions outside of class • Studied or prepared for class • Socialized or partied • Felt insulted or threatened because of your race/ethnicity • Had guarded interactions • Had tense, somewhat hostile interactions

Diversity • Interaction with students: • White/Caucasian • African American/Black • American Indian/Alaska Native • Asian American/Asian/Pacific Islander • Hispanic/Latino • Students from outside the United States • Student from a different religion than yours • Students from a different economic background

Diversity • Satisfaction with: • Respect for the expression of diverse beliefs • Gains in: • Knowledge of people from different races/cultures • Ability to get along with people of different races/cultures

CIRP Research Reports

Faculty • The biggest issue for Trustees concerning Faculty? • Chronicle of Higher Education Survey of Trustees (May, 2007) • Inadequate Salaries • HERI Faculty Survey (every three years) • Salary amount • Satisfaction with: • Salary • Health Benefits • Retirement Benefits

For More Information http://www.gseis.ucla.edu/heri John H. Pryor john.pryor@ucla.edu Victor Saenz vsaenz@ucla.edu Go to the HERI booth or use our new website:

Using CIRP Data for Accreditation, Assessment, and Accountability