Assessing Integrated Learning

Assessing Integrated Learning Maureen Pettitt, Ph.D. Skagit Valley College Beth Hartsoch, M.A. Western Washington University PNAIRP – Oct ‘07

Session Overview • Learning Communities at SVC • Established Approaches • Student Writing, Course Satisfaction Survey & CCSSE • SVC/WWU Project • WELS Survey & Focus Groups • Themes from the Data • New Studies in Progress

Interdisciplinary Learning at SVC • Interdisciplinary courses are a means of delivering instruction and fostering student learning • A response to curricular issues: Faculty felt that students • did not see connections between and among disciplines, and • needed to engage subjects more fully, to see education as a dynamic and interconnected process of exploration and discovery

Past Approaches:Multiple Assessment Methods CCSSE Student Satisfaction Survey Student Writing

1. Student Writing Study • Faculty/IR team developed two-year research project to assess student attainment of overarching General Education learning outcomes

Student Writing Study: Goals Assess students' ability to: • Apply a variety of concepts/texts/contexts and perspectives to solving problems, thinking about issues • Connect one’s own life experience, ideas and abilities with those that others bring • Understand and value the learning process for oneself and for others • Connect to external, lifelong social issues

Student Writing Study: Method • Identified courses being taught in Learning Communities, stand-alone, and DE that could be “matched” over a two-year period • Faculty agreed to participate and give students course credit • Developed a set of questions based on Gen Ed goals and faculty input • Students surveyed beginning-, mid-, and end-course

End of Course Questions • Have your learning expectations been met? • Have you learned things that you hadn't anticipated? If so, please describe. • What do you think are the most important aspects of your experiences in this course that account for your learning? • What have you learned in this course that will matter to you five years from now?

Student Writing Study: Findings • Students’ responses regarding valuing learning, connecting classroom learning with the world, and connecting with other were fairly similar in both stand-alone and LC courses. • Students in LC courses were far more likely to cite applying or appreciating a variety of perspectives to problem-solving (55% versus 15%)

Student Writing Study: Student Comments • “After taking this course I feel that I can make connections to various things, such as history, influences, people, and culture. This course taught me the value of making connections and things from my own perspective.”

Student Writing Study: Student Comments • “By combining course topics you get the ‘bigger picture’ and are able to sort of apply what we are learning better. By applying a subject or topic to another subject or topic you have to comprehend what you are learning and apply it to other things.”

Student Writing Study: Comparisons • Comparing student responses in the stand-alone courses with students in collaborative courses—same courses with the same instructor(s): • Students in stand-alone courses were less likely to write about these connections or about learning, and • tended to focus on personal growth, liking faculty, etc.

2. Quarterly Surveys • Faculty-developed pen-and-paper survey • Scale from “strongly disagree” to “strongly agree” • Faculty are provided their course results and comments, plus the cumulative for the quarter for all courses • Several years ago, we created a version that could be scanned

Learning CommunitySurvey Questions • "This learning community has been a valuable educational experience." • "I believe it was probably more valuable to have taken these classes together than it would have been to take them separately."

Mean Responses to Value Questions (N = 5156)

Linked & Federated Composition LCs(N = 2780)

Student Comments • LC: “This class has been fun and not sucky at all. I think I have learned a lot.” • Link: “I did not learn too much other than how to explore more in depth, how to write more.” • LC: “If they had been separate, I would have known the what and where, but not the why, and the why is always the most important question.”

3. Community College Survey of Student Engagement • Administered the CCSSE in 2003, 2005, and 2007 • Went to this survey because it most closely matched the institution’s focus on student learning and engagement • The LC question was a bonus extra!

CCSSE LC Question • Stem: “Which of the following have you done, are you doing, or do you plan to do while attending this college?” • Category: “Organized learning communities (linked courses/study groups led by faculty or counselors)” • Response Categories: • I have done • I plan to do • I have not done nor plan to do

Method • Used T-test for independent samples to examine differences in effort and engagement between students who had taken Learning Communities and those who had not done nor planned to do. • Results are only for the latest survey administration (Spring 2007)

Findings Prompt: “In your experiences at this college during the current school year, about how often have you done each of the following?”Responses range from 1 (“Never”) to 4 (“Very Often”)

The advantage of using multiple methods…. • …you get to do a “Reality Check” on a periodic basis

What we didn’t have…. Perceptions of students after they transferred from Skagit Collaboration with Western Washington University’s Office of Survey Research • WELS (Western Educational Longitudinal Study) Data • Focus Groups

WELS Transfer Study • Process began in 2001; baseline data Fall 2003 • Survey items related to students’ expectations, pre-WWU experiences, college preparedness, etc. • Three separate surveys of incoming transfer students administered on-line, with phone follow-up • Prior to starting classes • After the first quarter • After finishing three quarters

Results for SVC/Non-SVC Transfers

Focus Groups • WWU generated list; SVC did transcript analysis to identify student course taking, esp. Learning Communities and English Links • Identified two groups for interviews: 1) a mix of none, one, or two or more LCs taken at SVC 2) two or more LCs taken at SVC

Questions for all SVC transfers • What aspects of your learning experiences at Skagit Valley College have helped you in your courses at Western Washington? • If you could change anything about your learning experiences at Skagit Valley College so that you would be better prepared to succeed at Western, what would that be? • Can you talk a little bit about how Learning Community courses have helped or hindered your academic progress at Western? • If you took an English course that was linked with another course, can you describe how that learning experience has been useful or not in your studies at Western?

Additional Group Questions • For the first group: • Of you who have taken more than one Learning Community or English Link, how did taking a second or third learning experience of this kind impact the development of your skills or knowledge? • For the second group: • How did taking the second or third learning experience of this kind impact the development of your skills or knowledge?

Helpful in Transition to WWU • Students say the following aspects have helped: interdisciplinary learning, analytical/critical thinking, research and writing, group work, one on one interaction with instructors, and course content in specific courses • Additional helpful aspects noted: completion of GURs, knowing people who have attended Western, and the financial benefit of attending SVC

Interdisciplinary Learning • Students say that the interdisciplinary aspect of learning communities helps to make classes more interesting and fun at SVC. • Exposure to interdisciplinary learning at SVC helps students to feel more confident in their classes at Western. Students say they have learned to make connections and to apply what they’ve learned to another topic or to the real world.

English Links • Most students say English links were useful in their studies at SVC and in transferring to Western—because English links challenged students to think and write analytically, outside of the box. • Students say English was more interesting because it was linked to a content area which provided something to write about. They felt better prepared for researching and writing analytical papers at Western.

English Links • A few students say their English link was not useful in their studies at SVC. They say the focus was more on grammar and punctuation, as it was in high school.

Research and Writing • While some students say that Skagit prepared them to think critically and write analytical papers (discussed earlier), other students say they were not prepared for writing analytical papers at Western. • Research and writing experiences in English links and learning communities at SVC helped to prepare students for research and writing assignments at Western.

Other Student Outcomes • Analytical/critical thinking • The emphasis on critical thinking at SVC helps students in their classes at Western because they learned how to analyze information, make connections, and synthesize major ideas. • Group work • Group work in links and learning communities helped to prepare students for group work assignments at Western.

Challenge & Workload • An interesting side note is students’ perceptions about the challenge and workload involved in learning communities and links: • Some students say that learning communities and links are more challenging because of the level of analyzing involved. For some students, there is also a greater workload than expected. • Other students say there is not a greater workload.

Faculty Coordination • When instructors do not coordinate well together, the classes seem separate—not linked. Students find that the lack of coordination between some instructors means lack of integration between courses that are supposed to be linked. • Learning communities work best for students when the instructors coordinate well together, and particularly when instructors are in the classroom teaching together.

Changes Students Would Make • Students say the biggest issue for them is academic advising—because advising directly impacts students’ learning experiences at SVC as well as their preparation for Western. • Students also say they would finish a DTA to avoid difficulties in transferring, make sure they had learning experiences in a large lecture class, and do something about disruptive students.

Future WELS Analyses • Comparisons • SVC degree vs. no degree (or ranked by number of LCs taken) • SVC vs. other CCs • LC comparison between CCs • SVC vs. other CC vs. non-transfer WWU third-year students

Future WELS Analyses • Surveys • Baseline – administered prior to start of Fall qtr • Focus = experiences at previous college & expectations at WWU • Transitions – administered at the end of Fall qtr • Focus = first quarter WWU experiences • Spring Follow-Up – administered at the end of third quarter • Focus = first year experiences at WWU • Analyses • Descriptives and regression

GPA Analysis – LCs Taken There were no significant differences between groups on either variable.

GPA Analysis - Degree There were no significant difference between groups on entering GPA but there was for GPA after two quarters (p=.000)

Convergence of the Data • Findings from surveys, focus groups, and student writing strongly suggest that taking Learning Communities results in higher levels of • effort and engagement • collaborate with peers • interaction with faculty

Convergence of the Data • The WWU data about transfer students also suggest that students who complete Learning Communities at Skagit or the degree perform better than those who do not • However, we may need to look at the kinds and level of writing students are required to do in their courses

It’s all about perceptions…

Latest SVC Efforts…. • Learning Communities Assessment Project (Washington Center for the Improvement of Undergraduate Education) • Protocol developed at Harvard (Project Zero) • Rubric being tested • Developmental Learning Communities Project • SVC project to research student success based on developmental course delivery methods, including counseling-enhanced

Thanks for coming!

Assessing Integrated Learning