Institutional Research to Improve Student Participation and Retention in North America

1. Institutional Research to Improve Student Participation and Retention in North America Victor Borden Associate Vice Chancellor Information Management and Institutional Research Scott Evenbeck Dean University College Indiana University-Purdue University Indianapolis

2. Overview • Access and success: Contradiction and/or imperative? • Accepting the challenge: A case study • Developing a culture of evidence

3. 50 45 40 35 30 25 20 15 10 1993 1994 1995 1996 1997 1998 1999 2000 1992 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 Access and Success: U.S. Trends Percent of 18 to 24 Year Old H.S. Grads Enrolled in College Percent of All 18 to 24 Year Olds Enrolled in College Percent of 24 to 29 Year Olds with Four + Years of College Percent of 25 + Year Olds with Four + Years of College

4. 60 55 50 Percent of Freshman Graduating within 5 Years 45 40 35 30 Percent of 18 to 24 Year Old H.S. Grads Enrolled in College 25 20 15 10 1992 1993 1994 1995 1996 1997 1998 1999 2000 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 National Graduation Rates in Context

5. International Comparisons

6. Accepting the Challenge • Increased access results in greater diversity of learners • No longer just self-directed learners • H.E. institutions and faculty must take responsibility for student learning, and not just teaching • Requires… • Shift from “instructional” to “learning” paradigm • Approach informed by national and local research

7. Teaching Knowledge is “out there” Comes in chunks and bits and delivered by instructor Learning is cumulative and linear Teacher-centered and controlled Requires live instructor and live students Learning is competitive and individualistic Talent and ability are rare Learning Knowledge exists in each person’s mind Knowledge is constructed, created, and “gotten” Learning is a nesting and interacting of frameworks Student-centered and controlled Active learner required but not necessarily live instructor Learning is cooperative, collaborative and supportive Talent and ability are abundant From Teaching to Learning

8. Teaching Instructor is primarily a lecturer Instructor and student act independently and in isolation Instructors classify and sort students Staff serve/support teachers and instructional process Any expert can teach Learning Instructors as designers of learning environments Instructor and students work in teams with each other and other staff Instructor helps develop every student’s competencies and talents All staff are educators with responsibility for student learning Promoting learning is an expertise in itself Role of Instructor

9. National Models • National Survey of Student Engagement (NSSE) • AAC&U Greater Expectations Project • Policy Center on the First Year of College • Hallmarks Project

10. IUPUI as a Microcosm of U.S. Public Higher Education • Over 29,000 students • 22 Schools (Faculty), 82 departments, 180 degree programs • 73 % Undergraduate 27% Graduate/ Professional (incl, Medicine, Dentistry Law) • $200 million annual research and rising • Arts and Sciences offer majority of undergraduate credit hours but minority of student majors • Undergraduates 98% commuters • 80% work average of 30 hours/week • Freshmen include many ‘at-risk’ students • Degree recipients mostly transfers

11. Improving Access & Retention • Focusing the campus’s attention through the creation of University College (UC) • Inculcating IR/Assessment into the UC Culture • Spreading the mentality to all academic and administrative units • The Culture of Evidence • Universities as (vs.) learning organizations

12. University College • Academic unit formed in 1997 to provide gateway to academic programs for entering students • Houses numerous first-year programs and services including advising, orientation, math assistance center, writing center. • Develops and implements academic support courses including first-year seminars, critical inquiry, structured learning assistance • Serves over 8000 students annually • Faculty governed

13. UC Three-Phase Assessment Framework

14. Quantitative Assessment at UC • Conduct analyses to determine program impact on academic performance, retention rates, and DFW rates. • Describe retention rates and GPAs in defined populations over semesters and years • Examine participants verses non-participants with regard to Fall GPA and retention while controlling for background differences • Examine predicted vs. actual retention, course grades, and DFW rates • Administer student surveys to assess student needs, satisfaction, engagement, program impacts, reasons for leaving, etc.

15. Qualitative Assessment at UC • Brings Awareness Of Program Implementation Differences • Provides In-Depth Understanding of Student Responses and Interactions • Part of a Long Term Strategy of Formative Evaluation • Several Qualitative Assessments Conducted: First-Year Seminars, Critical Inquiry, Summer Bridge, Orientation

16. Faculty Associates • Awarded annually to faculty ($5000 stipend) • Contracted to do a particular quantitative and/or qualitative study • Use disciplinary methods of inquiry • Receive support from IR staff

17. Putting It All Together Satisfaction Surveys Portfolios Survey Self-Reports Focus Groups Standardized Tests Interviews GPAs CATs Academic Performance (GPAs; DFWs) National Survey of Student Engagement Retention Graduation Rates Campus Climate for Diversity Survey Degree Attainment Focus Groups Interviews

18. Example: Assessing First-Year Seminar • Processes: • Instructors’ Interpretation and Prioritization of Outcomes • Instructors’ Experience with Pedagogical Strategies • Students’ Perception of Valuable Components • Students’ Criticism of Seminar • Outcomes: • Instructors’ Ratings of Outcome Attainment • Students’ Reports of Improvements in Ability • Students’ Reports of Changes in Behavior

19. Findings on Outcomes: Students’ Report of Improvement in Abilities • Find resources at IUPUI 62% • Use the library 53% • Seek help when needed 52% • Use online resources 51% • Understand course expectations 47% • Participation in class discussion 47% • Manage own time 39% • Cope with stress 28% • Write for course assignments 24% • Think critically 23% (n = 221)

20. Students’ Report of Changes in Behavior • About half of the 221 students reported changes in one or both of two clusters of attitudes and behavior: • Becoming a better student • Taking course demands more seriously • Developing better study habits • Organizing time better • Becoming more outgoing • Trying to get to know students and instructors in other courses • Expressing self more, having more self-confidence

21. Impact of Participation in a First-Year Seminar: One Year Retention

22. Program Implications • Simplify, clarify template learning outcomes. • Front-load seminar in semester. • Differentiate, clarify, and integrate team member responsibilities. • Improve preparation and ongoing support for faculty. • Clarify relationship to linked academic course. • Give students more feeling of having accomplished something. • Make amount of work appropriate for one credit course. • Treat students like college students, not children.

23. Developing a Culture of Evidence • Faculty have no problem with evaluating ‘newfangled’ programs catering to first-year students • Accredited professional schools have evaluation as part of their culture • Traditional arts and science faculty find the concept troubling • Teaching viewed as a private activity • Unprepared learners (on their terms) is an input problem

24. An Assessment Matrix

25. Doing Assessment Right (Banta, 2002) • Participative planning • Assessment must be explicitly tied to institutional functions and be associated with goals that faculty and administrators value • Implementation • Assessment is most powerful at the unit level where changes will most immediately affect the faculty and students • Improving and sustaining • while the results of assessment may be used to respond to various external accrediting and stakeholder constituents, it will first and foremost provide the means by which the organization can continuously assess and improve itself

26. Improvement 1. Reporting to internal constituents 2. Demonstrating accountability to external stakeholders 3. Applying findings in campus improvement initiatives 4. Proposing improvement initiatives 5. Improving assessment methods · Web-based data · Electronic portfolios Assessable Outcomes Culture Of Evidence Application of Findings Instrumentation Data Collection Analysis

27. Parting Thoughts • How can we improve if we don’t know how we’re doing? • How can we know how we’re doing if we don’t agree on what we’re doing? • How can we agree on what we’re doing if we don’t work together? • How can we expect students to learn if we don’t demonstrate learning in our own teaching and organizational behaviors?

28. Conclusion • Conducting effective institutional research or assessment is necessary, but not sufficient for self-directed quality enhancement • First and foremost, we must develop shared goals and objectives that focus on our core mission: student learning