Taking Evidence Seriously: Assessment as a Tool to Tell a Story

1. Taking Evidence Seriously: Assessment as a Tool to Tell a Story Taé Nosaka, Assistant Director Lory Ann Varela, Research Coordinator Key Learning Communities Center for Advising & Student Achievement

3. Key Communities

4. Social Integration EARLY ORIENTATION RESIDENTIAL COMMUNITY (Braiden Hall) Academic Integration CLUSTER COURSES Intrusive Advising FEEDBACK/ CONNECTION Peer Mentors Key Communities Structure

5. Objectives of Key • Develop a strong community with an academic focus • Provide opportunity for shared learning • Provide access to feedback, information, and opportunities • Foster a healthy, diverse communities in which learning and academic achievement are encouraged and respected • Offer enriched opportunities for involvement and leadership development

6. Objectives (cont’) • Promote high academic performance (GPA) • Promote high retention and graduation rates • Promote academic engagement and campus involvement • Promote student satisfaction with their university experience

7. Outcomes-Based Assessment • Outcome • What do we want Key to accomplish? • Strategy • What is in place to achieve that outcome? • Assessment Method • How are we going to measure that outcome? • Criteria • How will you know if the “numbers” are “good”?

8. Key Outcome Example #1 • Key Communities Outcome #1 • Students in the Key Communities will have higher retention rates then similar students not participating in Key • Strategy • Students meeting 1:1 with Key Mentors three times per semester, Mid-Semester Academic Progress Conferences, Taking Stock at Mid-Semester Early Warning, Academic Success Workshops (2 mandatory), Referrals to campus resources, Co-enrollment in cluster classes. • Assessment Method • First-Year Retention Data • Criteria • Key students will have higher first-year retention rates

9. Key Outcome Example #2 • Outcome • Students in the Key Communities will have an increased awareness of diversity in terms of: 1) Opportunities to learn about other cultures by participating in Key 2) Interacting with students from backgrounds different from their own, 3) Having a greater understanding of people different from self because of experience in Key 4) Attending cultural events outside of their own culture. • Strategy • Living in a diverse community, discussing topics around diversity in KEY 192 classes, requiring students to attend at least 1 cultural activity per semester • Assessment Method • Student reports on Fall and Spring Program Evaluations • Criteria • At least 80% of students on all four goals

10. Key Outcome Example #3 • Outcome • Determine the components of the Key Academic Community that students find most beneficial in the fall semester in order to develop a structure for the spring semester that will help alleviate the “spring semester slump” that has been experienced • Strategy • 1) Co-enrolling in Cluster Classes with Key students • 2) Having a Key Mentor • 3) Living in Braiden Hall with Key students • 4) The Key Seminar Class • 5) The Plan for Excellence filled out during Key Orientation • 6) Key Academic Success Workshops (Time Management, Study Skills, etc) • 7) Moving in early • 8) Mid semester Academic Progress Conferences • 9) Resident Assistant • 10) Key Orientation • Assessment Method • Fall Program Evaluations for 2 years • Criteria • Determine top three most beneficial components

11. Telling the Key Story

12. Program Evaluations: the Students’ Story • Every Fall and Spring Semester • Paper Copy given in Seminar Classes (fall) • Student Voice (spring) • Specific questions addressing outcomes • Sample • What do we learn from these?

13. Highlights from Fall 2007 Program Evaluations • 92% response rate • 82% of Key students were satisfied with their decision to participate in Key • 30% said that Key was a large factor in their decision to stay at CSU (retention)

14. Highlights from Fall 2007 Program Evaluations • 90% indicated that Key was welcoming of students of their background and experience (sense of belonging) • 89% noted that they had the opportunity to interact with students from backgrounds different from their own (diversity)

15. Online Survey: The Story of our Faculty • Fall 2007 using Student Voice • Highlights • 100% satisfied with their experience teaching • Motivated by students, being asked, extra salary • Most positiveexperience: class discussions, students in smaller groups, students’ growth • Challenges: motivating students • Incentives: higher compensation

16. Key Mentor Focus Groups: The Mentors’ Story • Fall 2007 using 2 Focus Group with 17 staff (20 total) • Highlights • 94% indicated they would return to position • Valuable experiences include meaningful connections with people, skills in relationship and advising, and personal growth and development • Skill development reported with counseling, organizational, communication, balance, boundaries and expectations • Challenges included a disconnect with T.A. position • Recommendations on training, advice for new staff on developing relationship, and the meaningfulness of Peer Mentor role

17. Retention and Academic Performance Data Retention Tool developed by CASA Semester GPA information

18. Key Academic Community First Year Retention FA03-FA06 Cohorts

19. Key Academic Community First Year Retention, Students of Color

20. Key Service Community First Year Retention, Non-Residents

21. Key Plus Community

22. Key CommunityAcademic Performance

23. Key Academic and Service GPA Compared to CSU

24. Closing the Loop Using the evidence to change or improve our programs

25. Examples of Program Improvement or Change • Key Academic Community = Spring Seminar • Key Plus Community = Options • “Office Hour” to “Administrative Hour” • Stool in the Key Mentor Office • TA Training for Mentors • Consolidating Key Communities in Braiden Hall

26. Other sources for assessing Key • Taking Stock at Mid-Semester • National Study of Living Learning Programs (NSLLP) • Key Alumni Survey

27. What is the first thing you would do to start assessing your program? Worksheet

28. Questions?