A New Web-Based Tool for Assessing the Student Experience in Learning Communities

1. A New Web-Based Tool forAssessing the Student Experience in Learning Communities PNAIRP 2009 Portland, OR

2. Presenters:Maureen PettittSkagit Valley CollegeGillies MalnarichWashington Center for Improving the Quality of Undergraduate Education

3. Session Outline • Why this survey? • Development • Survey items • Feedback/revision • Fall 2009 Survey Priorities • Future Priorities • An Invitation

4. Thinking about learning

5. Thinking about learning • What behaviors and/or activities create more than mere rote learning?

6. Survey Development • The survey was created as a result of input from colleges and universities involved in the Assessing Learning in Learning Communities project who expressed the need for a survey that examined students’ experiencesin learning communities, especially with regard to integrative learning. • Washington Center approached Skagit Valley to help design and administer the online survey.

7. Goals • Collect national data about the “what” and “how” of substantive learning in learning communities • Use that data to guide the creation of online resources and tools that help faculty to enhance students’ -- and their own -- experiences in the learning communities

8. Definitions • Learning community (LC) • A common cohort of students who attend two or more classes together in which they experience an explicitly designed opportunity for integrative learning. • Integrative learning • the intentional integration of ideas and/or practices drawn from two or more disciplines (or areas of expertise) which invite new questions, understanding, explanations, and applications.

9. Development Process • Review of literature and surveys about student learning and integrative learning • Draft multiple versions (lots and lots of revisions!) based on: • Input from colleagues across the country • Recommendations made by students in a Skagit learning community – instructors used a “think aloud” protocol to identify and revise questions which lent themselves to different interpretations

10. Survey Elements • Students’ engagement in classroom activities (You just responded to some of those questions!!) • Instructors’ activities that support learning • Students’ perceptions of gains made in their own understanding and abilities • Students’ perceptions of the mental activities they engage in in the LC versus other courses

11. Students’ Experiences of Learning in Learning Communities Fall 2009 - National Pilot • Establish liaison with learning community leads and institutional research offices • Administer ~ 250 surveys per participating college • Trouble shoot technical aspects and get feedback on logistics and survey questions (“revise again!?”)

12. A few quotes about revising... • “There are always too many words at first.”Jacques Barzun • “Forced to weigh your words, you find out which are the styrofoam and which are the heavy gold.”Ursula K. Le Guin • “All first drafts are shit.”Ernest Hemingway

13. Issues Thus Far…. • Individualizing survey – for example, additional questions • Institutional Review Board – colleges will deal with this at the local level • Limiting the pilot to 250 surveys per college – ok we bent a little on that, but don’t tell  • Limitations of Remark software – especially the requests for college-specific questions

14. Students’ Experiences of Learning in Learning Communities Winter 2010 • Compile and send data results to fall 2009 pilot schools • Summarize and respond to feedback • Revise online survey based on pilot feedback • Post new survey for use at minimal cost (e.g. $50 for every 250 surveys) with goal of making survey available for mid-term use

15. Students’ Experiences of Learning in Learning Communities Spring 2010 • Analyze data from all participating campuses (i.e. broad sweep of classroom practice) • Adapt “small group instructional diagnostic” to supplement mid-term survey use • Draft a prototype resource list and faculty development module based on survey results • Report early findings to national learning communities community via listserv

16. Students’ Experiences of Learning in Learning Communities Summer 2010 • Begin work on faculty development section of Washington Center website • Build network of IR specialists and LC leads interested in deepening research on students’ experiences of learning in learning communities • Seek funding for further development of this work

17. Faculty Development Modules (in design) • Sample Question • “Work on an assignments that require integrating ideas, strategies, or skills from classes (or disciplines) included in this learning community” • Faculty development module could include: • References about designing integrated learning from Washington Center’s National Project on Assessing Learning in Learning Communities • Teaching practices drawn from Classroom Assessment Techniques (CATS) and collaborative learning techniques

18. CCSSE Annotated Bibliographies • The CCSSE annotated bibliography can be accessed from the “About the Survey” webpage: http://www.ccsse.org/aboutsurvey/biblio/page1.cfm

19. An Invitation • If your college is engaged in learning communities and you are looking for ways to assess the learning that takes place in the LC, feel free to email Gillies to discuss the possibility of conducting the survey at your college: malnarig@evergreen.edu • If you have questions about survey development, data collection or analysis, feel free to email Maureen: mpettitt@skagit.edu

20. Thank you for your interest in this work!