Keeping an Open Mind: Self-Examination and the College Experience

1. Keeping an Open Mind: Self-Examination and the College Experience Thomas Geaghan Office of Institutional Research Case Western Reserve University Presentation at the 2009 Research ShowCASE

2. Abstract The purpose of this study is to examine how keeping an open mind contributes to the college experience and whether gender acts as a moderator in the relationship between open-mindedness and learning outcomes. Each spring, from 2005 to 2007, first-year students were invited to participate in the National Survey of Student Engagement. As part of this survey, students were asked three questions assessing their openness to new ideas and the ideas of others (reflective learning). A Two-Step Cluster Analysis conducted on these three items revealed two distinct groups of students: high-open-minded students (N = 522) and low-open-minded students (N=650). These two groups did not differ in terms of gender, ethnicity, or SAT scores. High-open-minded students reported significantly greater gains in four learning outcomes (personal/social development, practical competence, general education, and diversity of experiences). Similarly, the high-open-minded group was significantly more satisfied with the educational experience at Case and were more likely to say they would still choose to attend Case if given the opportunity again. Results of the moderation analysis revealed that gender acted as a moderator for the relationship between open-mindedness and social development, practical competence, and choosing to attend Case again. Specifically, there was a significantly stronger relationship between open-mindedness and learning outcomes/satisfaction for men than for women.

3. Introduction • Researchers have begun to examine the effects of “deep learning” on the college experience. • Past work established that curricula including aspects of “deep” or “reflective” learning improve student outcomes. • This type of learning increases students’ feelings that the coursework is relevant and has been shown to improve learning in physics and in medical school. • Research established a link between studying abroad and deep learning. Students who studied abroad engaged in more reflective learning.

4. Introduction • One area of “deep learning” is reflective learning, or an openness to new ideas and the ideas of others. • A recent study examined the interaction between reflective learning and academic disciplines on student’s satisfaction with the college experience. • Regardless of academic discipline, reflective learning was positively related to student satisfaction. • Students in the so-called “hard” disciplines (i.e., life sciences, engineering) reported a lower frequency of “deep learning” than did their peers in the “soft” disciplines (i.e., arts, humanities).

5. The Current Study • The purpose of the current study is twofold: • We first intend to examine whether reflective learning, a critical component of “deep learning,” is related to first year students’ satisfaction and learning outcomes. • Next, we examine whether gender plays a moderating role in the relationship between reflective learning and first year students’ satisfaction and self-reported gains in learning outcomes.

6. Method Survey • In the spring semester, from 2005 through 2007, all first-year students enrolled at Case were sent an e-mail message inviting them to participate in the National Survey of Student Engagement (NSSE). • E-mail messages contained a unique link to the survey instrument.

7. Measures Predictor • Reflective Learning: Students indicated the frequency with which they a) examined the strengths and weaknesses of their own views; b) tried to understand someone else’s views by imagining how an issue looks from their perspective; and c) learned something that changed the way they understood an issue or concept. • A Two-Step Cluster Analysis conducted on these three items revealed two distinct groups of students: those who often engaged in reflective learning (high reflective learners; N = 522) and those who did not (low reflective learners; N=650).

8. Measures Outcome Measures – Learning Outcomes • Gains in Personal and Social Development: Seven-item scale assesses the extent to which students feel that Case has contributed to their personal and social development (e.g., understanding of self; learning effectively on your own). • Gains in Practical Competence: Five-item scale assesses the extent to which students feel their experience at Case has contributed to practical skills (e.g., working well with others; analyzing quantitative problems).

9. Measures Outcome Measures – Learning Outcomes, Continued • Gains in General Education: Four-item scale assesses the extent to which students feel their experience at Case has contributed to academic abilities (e.g., writing/speaking clearly and effectively). • Diversity of Experiences: Three-item scale assesses the extent to which students interact with peers from a different racial/ethnic group or with those of different religious or political beliefs. • These four scales are based on scales tested for reliability in a previously published NSSE study. Scales show acceptable to high reliability in the present sample (Cronbach’s α-values range from .66 to .79).

10. Measures Outcome Measures – Student Satisfaction • Along with the four scales, we examine differences between high and low reflective learners on two NSSE items: • a) how would you evaluate your entire educational experience at this institution? • b) If you could start over again, would you go to the same institution you are now attending?

11. Characteristics of the Sample • Men made up the majority of the sample (53%); the majority had permanent residences in Ohio (51%) and most (83%) identified as Caucasian or Asian. • That said, there were no significant differences between the high-and low reflective learners in terms of these three demographic variables. • Similarly, the two types of students did not differ in terms of SAT Math, SAT Verbal, or ACT Composite scores, nor did they differ in terms of high school class rank.

12. Characteristics of the Sample • Breakdown of high and low reflective learners within each intended major. Differences that exist are not significant.

13. Question 1 Study question • Does reflective learning affect student learning outcomes and satisfaction? Analysis plan • A series of One-Way Analyses of Variance (ANOVAs) were conducted to determine whether high and low reflective learners differed on student self-reported learning outcomes and student satisfaction.

14. Question 1: Results • Students engaging in more reflective learning reported significantly higher gains on all four of the measured learning outcomes (all ps<.0001).

15. Question 1: Results • High reflective learners were significantly more satisfied with the two satisfaction measures (ps <.01).

16. Question 2: Results Study Question • Does gender moderate the relationship between reflective learning and learning outcomes/satisfaction Analysis plan • Tests of moderation were conducted using the regression procedure outlined by Baron and Kenney (1986). • Six regression equations were calculated, one for each of the outcome variables. • In each regression equation, gender (F = 0, M = 1) and open-minded group membership (low = 0, high = 1) were entered in the first block. The interaction term was entered in the second block.

17. Question 2: Results • Gender is not a moderator for the relationship between reflective learning and gains in general education, diversity of experiences, or satisfaction with the entire educational experience. • Gender moderates the relationship between reflective learning and gains in practical competence, gains in personal/social development, and students’ willingness to say they would attend Case if given the choice again.

18. Question 2: Results • For the three items in which there was a significant moderation effect of gender, the moderating variable works such that, in the low reflective learning group, women report significantly greater gains/satisfaction than do men; however, men and women in the high reflective learning group do not differ from one another. • The next three graphs illustrate this relationship.

22. Summary • First-year students who are more open to new ideas and the ideas of others report greater gains in all measured learning outcomes. • Highly reflective learners are more likely than their peers to be satisfied with their experiences at Case. • Gender moderates the relationship between reflective learning and three of the six outcome variables. • For these three items, women in the low reflective learning group significantly outperformed their male peers; however, gender differences disappeared among students in the high reflective learning group

23. Conclusion • Findings highlight the importance of reflection in learning and satisfaction among first year students. • Results highlight the importance of reflective learning among first-year males. • It is unclear whether this type of learning is driven by the student’s personality or if it develops over time. • Future work should examine how students’ openness to new ideas changes throughout their college careers.

24. Questions? Comments? • Visit: http://www.cwru.edu/president/cir/cirhome.htm • Contact: Tom Geaghan Senior IR Analyst trg9@case.edu 368-1500