Using Data from the CCSSE to Encourage Change

1. Using Data from the CCSSE to Encourage Change Maureen Pettitt, Ph.D. Leslie Croot, M.S.

2. Session Topics • CCSSE Overview • CCSSE-Generated Reports • SVC-Generated Presentation & Reports • Use of Reports

3. CCSSE Overview • Emphasis on student engagement and student learning (e.g., Astin, Tinto, Pace) • Adapted from the National Survey of Student Engagement (NSSE) • Funding from Lumina Foundation for Education and Pew Charitable Trusts • Administered by Community College Leadership Program at UTA • Pilot test in 2001; field test in 2002

4. Student Engagement • Students who get involved with the people and activities of the college (beyond the traditional classroom) have • higher retention rates; • greater personal growth, achievement and satisfaction; and • increased participation in further learning opportunities (Astin, 1985)

5. CCSSE Contents • Active & Collaborative Learning • Student Effort • Student-Faculty Interaction • College Contribution to Knowledge, Skills & Personal Development • Mental Activities • Student Demographics

6. 2003 CCSSE Participants • National • 93 community colleges; 65,300 usable surveys • Northwest Consortium • Six WA community colleges & one in BC; 3,480 usable surveys • 19 additional questions • SVC • Surveyed 850 students in 60 classes; 765 usable surveys

7. CCSSE-Generated Reports • College Results • Summary: Successes and Areas for Improvement • Aggregate percents for all colleges and for SVC • Mean Comparisons • SVC, consortium, all colleges • Frequency Distributions • SVC, consortium, all colleges

8. CCSSE-Generated Reports • Benchmarks -- five benchmarks from 38 engagement items on the CCSSE • Active & Collaborative Learning • Student Effort • Academic Challenge • Student-Faculty Interaction • Support for Learners

9. Other CCSSE “Tools” • On the website • Interpreting and working with results • Developing storylines • Powerpoint template & talking points • National reports • Highlights

10. Internal Reports: The Challenge • Creating reports that are both meaningful and can inform change at the institution • Knowing what the issues are and how the data can shed light on those issues (also: what issues can be “fixed” with information?) • Having a network of faculty & staff at the college who know the issues and will ask for specific reports

12. Internally-Generated Reports & Presentations • Gen Ed Task Force Summer Retreat • Two Fall In-Service Presentations • Article for Fall Teaching & Learning Newsletter • Library & Media Services • Diversity Steering Committee • Information Technology • Student Services Programs • Student Government

13. General Education Task Force • Gen Ed Task Force Summer Retreat 2003 • Data presented according to existing gen ed principles and outcomes • Generated discussions about what wasn’t in our current outcomes (i.e., Information Literacy) • Provided insight into the efficacy of our pedagogical approaches (i.e., learning communities, infusion of critical thinking and cultural pluralism, etc.)

14. General Education Outcomes • Communications • Critical Thinking • Quantitative Reasoning • Development & Features of Culture • Putting Knowledge into Action • Overarching Learning & Development Goals

15. To what extent has your experience at this college contributed to your knowledge, skills, and personal development in the following area: Acquiring a broad general education

16. In your experience at this college during the current school year, about how often have you done each of the following: Prepared two or more drafts of a paper or assignment before turning it in

17. In-Service Presentation Fall 03 • Purpose – provide an overview and generate discussions (information was also the basis for T & L Newsletter article) • “Most Frequent” (at least 20% of the respondents reported they did this “very often”) and “Least Frequent” (at least 20% of the students reported they had “never” done this) • General Education-related outcomes • Student satisfaction with services

18. Most Frequent • Worked on a paper or project that required integrating ideas or information from various sources – 31.1% • Prepared two or more drafts of a paper before turning it in – 29.5% • Asked questions in class or contributed to class discussions – 27.1% • Used an electronic medium (list-serv, chat group, Internet, etc.) to discuss or complete an assignment – 24.0%

19. Most Frequent • Discussed ideas from your readings or classes with others outside of class (students, family members, co-workers, etc.) – 23.5% • Worked with other students on projects during class – 22.7% • Had serious conversations with students who differ from you in terms of their religious beliefs, political opinions, or personal values – 20.5%

20. Least Frequent • Participated in a community-based project as part of a regular course – 67.5% • Worked with instructors on activities other than coursework – 62.0% • Tutored or taught other students (paid or voluntary) – 59.4% • Discussed ideas from your readings or classes with instructors outside of class – 35.5%

21. Least Frequent • Used an electronic medium (list-serv, chat group, Internet, etc.) to discuss or complete an assignment – 23.3% • Come to class without completing readings or assignments – 22.3% • Talked about career plans with an instructor or advisor – 20.9% • Used e-mail to communicate with an instructor – 20.4%

22. Learning Communities • Always questions on campus about the benefit of collaborative courses • CCSSE question asking whether the student had taken a learning community, had not, but planned to, or had not and did not intend to take.

23. Mental Activities • Similar in structure to Bloom’s Taxonomy • Prompt: “During the current school year, to what extent has your coursework emphasized the following mental activities…” • Response Options: Range from 1 “Very Little” to 4 “Very Much”

24. Groupings • SVC 1 = SVC students who had taken a learning community or linked course • SVC 2 = SVC students who had not, but planned to take a learning community or linked course • SVC 3 = SVC students who had not taken and did not plan to take a learning community or linked course • Consortium = all students in the Northwest consortium of colleges, excluding those from Skagit Valley College and Douglas College • All = all students who participated in the CCSSE Survey, based on 93 colleges

25. ”memorizing facts, ideas, or methods from your courses and readings so you can repeat them in pretty much the same form.”

26. “analyzing the basic elements of an idea, experience, or theory.”

27. “synthesizing and organizing ideas, information, or experiences in new ways.”

28. Library/Media Services • Additional questions on library use, library resources, use of Internet, and knowledge/skill acquired to conduct information searches • Only comparison with consortium colleges • SVC used the library more, acquired more skills/knowledge, but were less satisfied with the library resources.

30. Diversity Steering Committee • Burning question: how did students of color compare with their white counterparts • There were few significant differences, except SOC had better relationships with administrative personnel • Limitations—N for SOC = 78; these were students of color who had transitioned to the “mainstream” (versus ESL students)

31. Information Technology • IT Dean was then-chair of the State Information Technology Council • Prepared a report he presented at an ITC meeting showing consortium results for nine IT-related CCSSE items • Also prepared an SVC report showing college-consortium comparisons

32. Student Services • Created a 71-page report just for Student Services: • Summary of Significant Findings • Career Counseling • Admissions • Placement • Advising/Financial Aid Advising • Registration • Financial Aid • Student Life

33. Counselors/Advisors • Where do you most often get information about the requirements for your educational program? • Academic advising/planning: How often do you use? How satisfied? How important? • How often do you use career counseling? • How often do you use transfer credit assistance?

34. Students with Risk Factors • Analysis of CCSSE student responses based on risk factors: • Academically Under-prepared • Financially Independent • Single Parent • English Not Native Language • Parent’s Education • Disabled

35. Breakdown by Risk Category (N=765) • Three categories in analysis: • No Risk -- risk factors = 0 (19%) • Low Risk -- risk factors =1 (34%) • At Risk -- 2 or more risk factors (47%)

36. Students at Risk Study • Used this data to follow-up with SVC Student Support Services (TRIO Grant) program participants • SSS program goal: provide educational opportunities for disadvantaged students

37. TRIO Program Evaluation • Created an instrument with CCSSE items; very targeted; no demographic data • Given at end of quarter in College Success Skills course for SSS students • Part of the SSS program’s Outcomes Assessment Plan • Track results of SSS students over time/against CCSSE baseline using students designated as “at risk” (two or more risk factors) for comparison

38. College Support • SSS participants feel the college helped them cope with non-academic responsibilities more than all other groups (p < .001). • SSS participants feel the college helped them thrive socially more than all other groups (p < .001). • SSS participants feel the college provided the support they needed to succeed more than all other groups (p = .006).

39. Use of Services • SSS participants sought financial aid advising, and career counseling more than the "Low Risk" and "At Risk" groups. • SSS participants used services for people with disabilities more than all other groups. • SSS participants used academic advising/ planning more than all of the other groups • SSS participants used peer or other tutoring more than the "No Risk" and "Low Risk" groups, but not the “At Risk.”

40. The "At Risk" group felt they had a poorer relationship with instructors compared to the "No Risk" group (p = .007). However, the "SSS Users" felt they had a better relationship with instructors compared to the "At Risk" group (p < .001). There were no other differences between the groups.

41. The "At Risk" group felt they had a poorer relationship with administrative personnel compared to the "No Risk" group (p = .028). However, the "SSS participants" felt they had a better relationship with administrative personnel than the "Low Risk" group (p = .001) or the "At Risk" group (p < .001). There were no other differences between the groups.

42. Student Government • Short report and presentation at ASSVC meeting • Student government reps were interested in general demographics, how students spent their time, and student participation in extra- and co-curricular activities

43. Using the results

44. Using the results • Included data in Board of Trustee Monitoring Reports on Ends Policies • Student Satisfaction & Success • Institutional Diversity • Provided reports and presentations for various groups

45. Using the Results • Folks at the college who moved to an action phase spent time identifying specific things they wanted to improve, prioritizing them, and designing low-cost, uncomplicated interventions (“quick wins”)

46. Fifth-Year Accreditation Report • Spring 2004 (1 yr later) • All reports compiled in a big notebook • Included documentation from CCSSE recognizing SVC as a “Best Practice College” and one of four colleges in the nation to get the MetLife Foundation award

47. Future Plans • Administer CCSSE in Spring 2005 and 2007 • Cross-time comparisons available for accreditation self-study and 10-year visit in 2009

48. Thanks for your attention!