Maureen Pettitt, Ph.D. Skagit Valley College Beth Hartsoch, M.A. Western Washington University

1. A Collaborative Study on the Preparation of Community College Students for the Four-Year Experience,With a Focus on Integrated Learning Maureen Pettitt, Ph.D. Skagit Valley College Beth Hartsoch, M.A. Western Washington University 2008 AIR Forum Seattle, WA

2. Session Overview • Measuring Integrated Learning – Learning Community Outcomes • Background: Learning Communities at SVC • Measurement Strategies • Established Assessment Approaches • Student Writing, Course Satisfaction Survey & CCSSE • New Assessment Approaches • Academic performance indicators • Western Educational Longitudinal Study (WELS) • Survey Focus Groups • Findings

3. Measuring Learning Community Outcomes Questions: • How can we measure the impact of learning communities at the 2-year and 4-year level? • What do we gain by using multiple assessment approaches?

5. Background: Integrated Learning at Skagit Valley College • Interdisciplinary courses are a means of delivering instruction and fostering student learning • A response to curricular issues: Faculty felt that students • did not see connections between and among disciplines, and • needed to engage subjects more fully, to see education as a dynamic and interconnected process of exploration and discovery

6. Typical Learning Communities • Fully Collaborative • Feast or Famine (Nutrition and Sociology) • Sex.comm (Human Sexuality and Mass Communication) • Stating the Matter (Chemistry and English Composition) • Developmental (federated, co-enrollment) • Reading Between the Numbers (developmental Math and Reading)

7. Typical Learning Communities • Federated • Celluloid Science - science majors enroll in an introductory film course and one of the courses required for their major , with the explicit purpose of exploring how films portray scientists, scientific practices, and concepts • This, That, and the Other - students co-enroll in a research paper course and one of several social science courses, with the goal of researching topics specific to their field of study

8. Past Assessment Approaches CCSSE Student Satisfaction Survey Student Writing

9. 1. Student Writing Study • Faculty/IR team developed two-year research project to assess student attainment of overarching General Education learning outcomes using student writing

10. Student Writing Study: Method • Identified courses being taught in Learning Communities, stand-alone, and DE that could be “matched” over a two-year period • Faculty agreed to participate and give students course credit • Developed a set of questions based on Gen Ed goals and faculty input • Students surveyed beginning-, mid-, and end-course

11. Student Writing Study: Findings • Students’ responses regarding valuing learning, connecting classroom learning with the world, and connecting with other were fairly similar in both stand-alone and LC courses. • Students in LC courses were far more likely to cite applying or appreciating a variety of perspectives to problem-solving (55% versus 15%)

12. Student Writing Study: Findings • Comparing student responses in the stand-alone courses with students in collaborative courses—same courses with the same instructor(s): • Students in stand-alone courses were less likely to write about these connections or about learning, and • tended to focus on personal growth, liking faculty, etc.

13. Student Writing Study: Student Comments • “After taking this course I feel that I can make connections to various things, such as history, influences, people, and culture. This course taught me the value of making connections and things from my own perspective.”

14. 2. Quarterly Surveys • Short survey administered by staff member in all Learning Communities Scale from “strongly disagree” to “strongly agree” • Faculty are provided their course results and comments, plus the cumulative for the quarter for all courses

15. Learning CommunitySurvey Questions • "This learning community has been a valuable educational experience." • "I believe it was probably more valuable to have taken these classes together than it would have been to take them separately."

16. Mean Responses to Value Questions (N = 5156)

17. Student Comments • “This class has been fun and not sucky at all. I think I have learned a lot.” • “I did not learn too much other than how to explore more in depth, how to write more.” • “If they had been separate, I would have known the what and where, but not the why, and the why is always the most important question.”

18. 3. Community College Survey of Student Engagement • Administered the CCSSE in 2003, 2005, and 2007 • Went to this survey because it most closely matched the institution’s focus on student learning and engagement • The LC question was a bonus extra!

19. CCSSE LC Question • Stem: “Which of the following have you done, are you doing, or do you plan to do while attending this college?” • Category: “Organized learning communities (linked courses/study groups led by faculty or counselors)” • Response Categories: • I have done • I plan to do • I have not done nor plan to do

20. Method • Used T-test for independent samples to examine differences in effort and engagement between students who had taken Learning Communities and those who had not done nor planned to do. • Results are only for the latest survey administration (Spring 2007)

21. Results Prompt: “In your experiences at this college during the current school year, about how often have you done each of the following?”Responses range from 1 (“Never”) to 4 (“Very Often”)

24. Convergence of SVC Data • Findings from LC surveys, student writing and the CCSSE strongly suggest that taking Learning Communities results in higher levels of • effort and engagement • collaborate with peers • interaction with faculty

25. What we didn’t have…. …Perceptions or performance of students after they transferred from Skagit Collaboration with Western Washington University’s Office of Survey Research

26. Outcomes/Data Sources

27. Notes • Quality and type of experience is unknown, and we know from quarterly surveys that this varies widely • linked course vs. fully integrated • Instructor investment and experience • Students who took no LCs could be a special case that is not accounted for with available controls • Those with schedule constraints such as employment or family may be less likely to take LCs

28. Notes • When looking at significance, we used a very liberal p value of 0.1 to select outcomes. • Since this is an ongoing study used to improve programs, we want not only evidence of effects, but also suggestions for further investigation. • In the case of WELS outcomes, our sample is small when broken down by number of LC experiences. • There are no substantial consequences for being wrong. • WWU will conduct another WELS transfer baseline in 2008, and we may use that opportunity to gather additional information about some of these items.

29. WWU Academic Performance Indicators • Outcomes • Fall 1: credits enrolled, credits earned, GPA • Winter 1: retention • Year 1: GPA, mean credits earned/quarter • Fall 2: retention • Controls • Community college, sex, race/ethnic, age, transfer credits, transfer GPA, associates degree, WWU full-time status • Variables in outcomes list used as controls when appropriate • Independent variables • 1 LC, 2 LCs, 3 LCs. No LCs is excluded category.

30. WWU Academic Performance Indicators

31. WWU Academic Performance Indicators • No significant differences for: • Fall - Winter 1 retention • Year 1 major declared • Year 1 credits earned • Year 1 GPA

32. Western Educational Longitudinal Study (WELS) • Fall 2005 Transfer Cohort • 1051 students – 44 SVC respondents & 38 Seattle District respondents • Survey items related to students’ expectations, pre-WWU experiences, college preparedness, etc. • Three separate surveys of incoming transfer students administered on-line, with phone follow-up • Prior to starting classes - baseline • After the first quarter - transitions • After finishing three quarters & yearly follow-ups (not included in this analyses)

33. WELS BaselineAcademic Preparation

34. WELS BaselineExpected Difficulty Negative beta indicates less difficulty expected

35. WELS TransitionsPerceived Difficulty – Fall 1 Negative beta indicates less difficulty perceived

36. Results – WELS No significant differences for: • Baseline: • Academic, community and campus engagement • Expected integration into college • Using computers for academic work • Using library resources for research • Preparing for & presenting work in front of a class • Transitions: • Time and task management index

37. Student Engagement – Previous CollegePercent responding that they did this 4 or more times

38. Focus Groups Making connections among disciplines…

39. Focus Groups: Design • WWU generated list of SVC transfers • SVC did transcript analysis to identify student course taking, particularly Learning Communities • Identified two groups for interviews: 1) a mix of none, one, or two or more LCs taken at SVC 2) two or more LCs taken at SVC

40. Questions for Both Focus Groups • What aspects of your learning experiences at Skagit Valley College have helped you in your courses at Western Washington? • If you could change anything about your learning experiences at Skagit Valley College so that you would be better prepared to succeed at Western, what would that be? • Can you talk a little bit about how Learning Community courses have helped or hindered your academic progress at Western? • If you took an English course that was linked with another course, can you describe how that learning experience has been useful or not in your studies at Western?

41. Additional Group-Specific Questions • For the first group: • If you have taken more than one Learning Community or English Link, how did taking a second or third learning experience of this kind impact the development of your skills or knowledge? • For the second group: • How did taking the second or third learning experience of this kind impact the development of your skills or knowledge?

42. Focus Group Responses We summarized the most commonly cited ways in which students reported benefiting from their LC experiences. In most cases there were students who disagreed with these ideas, and did not feel they benefited in this way. Our purpose is not to quantify the responses, but to report the types of benefits reported by students.

43. Helpful in Transition to WWU • Interdisciplinary learning • Analytical/critical thinking • Research and writing • Group work • One on one interaction with instructor • Course content in specific courses • Additional helpful aspects noted: completion of GURs, knowing people who have attended Western, and the financial benefit of attending SVC

44. Interdisciplinary Learning • The interdisciplinary aspect of learning communities helps to make classes more interesting and fun at SVC. • Exposure to interdisciplinary learning at SVC helps students to feel more confident in their classes at Western. • This is supported by WELS outcomes on expected and perceived difficulty.

45. Interdisciplinary Learning Students say they have learned to make connections and to apply what they’ve learned to another topic or to the real world.

46. Writing • Most students say English LCs were useful in their studies at SVC and in transferring to Western—challenged students to think and write analytically, outside of the box. • This is supported by WELS outcomes for expected and perceived difficulty with writing. • English was more interesting because it was linked to a content area which provided something to write about.

47. Research and Writing • Research and writing experiences in English links and learning communities at SVC helped to prepare students for research and writing assignments at Western. • This is supported by WELS outcomes for planning, implementing and organizing research.

48. Analytical/Critical Thinking • The emphasis on critical thinking at SVC helps students in their classes at Western because they learned how to analyze information, make connections, and synthesize major ideas.

49. Group Work • Group work in links and learning communities helped to prepare students for group work assignments at Western. • This is supported by WELS outcomes for expected and perceived difficulty with group work.

50. Faculty Coordination • Coordination between instructors is critical. Absent this coordination, linked course experiences are no different from stand-alone courses. • Co-teaching enhances the learning community experience further.