Student Learning Outcomes for Student Services

1. Student Learning Outcomes for Student Services ACCCA Santa Clara, CA February 2005 Terrence Willett Director of Research Gavilan Community College

2. Acknowledgements • Gavilan Student Services Division especially: • John Baker, Margery Regalado, Jane Maringer • Gavilan Instructional Division especially: • Marty Johnson • Marlene Dwyer & her Curriculum Committee • Department Chairs • Academic Senate, Enrique Luna • Research and Planning Group especially: • Harriett Robles, Brad Philips, Fred Trapp, Bob Gabriner, Jerry Rudman • Other colleagues: • Rachel Ruiz, Mesa College • Janet Fulks, Kate Pluta, Sue Dixon, Bakersfield College • WASC • Darlene Pacheco, Barbara Beno

3. Learning Outcomes for Today • Summarize history of accountability and how new standards came to be • Define learning outcomes and explain the underlying philosophy • Describe the assessment cycle and its role in the college • Write appropriate student learning outcomes for your program • Explain at least two methods to assess at least one student learning outcome including needed resources and potential barriers • Identify at least 3 resources to help you succeed in the learning outcomes challenge

4. Your expectations and needs

5. History • 1960’s • In the past, year-end reports demonstrated proper management but not whether the programs directly aided student learning • Colleges measured enrollments, participation rates, budget expenditures • 1990’s • More recently have expanded to measure success, retention, persistence, graduation, and transfer • Focus has also been on instructional delivery versus student learning • 2000’s • Student Learning Outcomes

6. Standard IIB • The institution recruits and admits diverse students who are able to benefit from its programs, consistent with its mission. Student support services address the identified needs of students and create a supportive learning environment. The entire student pathway through the institutional experience is characterized by a concern for student access, progress, learning, and success. The institution systematically assesses student support services using student learning outcomes, faculty and staff input, and other appropriate measures in order to improve the effectiveness of these services.

7. What are Learning Outcomes? • “Learning outcomes” not a new concept • Used in California in K-12, special ed, voc programs • Used in other college systems (i.e. Maryland, England) • Barr and Tagg called for learning outcomes in 1995 • Focuses more on students’ learningrather than just how we instruct or support them; outcome based • Emphasis on higher more complex level of learning • Clearly states educational intentions for students

8. Sounds like behavioral objectives! • Learning outcomes differ from behavioral objectives by • Being broadly stated • Having instructors assess gains in skills rather than creating detailed list of specific topics and abilities to be mastered (Harden 2002) • Some authors do not perceive that the difference between learning outcomes and behavioral objectives is important (Prideaux 2000) • Important to distinguish the two to gain acceptance

9. 6 Principles – O’BanionThe Learning College… • …creates substantive change in individual learners • …engages learners in the learning process as full partners who must assume primary responsibility for their own choices • …creates and offers as many options for learning as possible • …assists learners to form and participate in collaborative learning activities • …defines the roles of learning facilitators in response to the needs of the learners • …succeeds only when improved and expanded learning can be documented for learners

10. Levels of Analysis • Institutional • Department or Program • Class or Service • Assignment or Student Interaction

11. New Perspective • “We must treat the college as a learning laboratory” – John Baker, VP SS Gavilan • Dealing with whole student-their uncertainty, response to that uncertainty, and their wisdom and courage-all matters of being (Harden 2002) • Shift from instruction as “core of the college” towards learning as central focus • True marriage between academic and life skills learning

12. Institutional Concerns • Students have a fundamental right to services and if these services are not or cannot be related to learning outcomes, their existence could be unfairly jeopardized (Collins 2002) • Uncritical application of business models and concepts to education • Fear all this measuring will be a waste of effort and not enhance student achievement – Luna, Gavilan College Academic Senate • Do students really care about learning? - Borden

13. Gavilan’s Approach • Discussion began in Student Services • Had SLO presentations at convocation and meetings of department and program leads as well as attending external workshops • Institutional learning outcomes selected • Course level learning outcomes written • Currently developing program level outcomes • Next steps include publishing outcomes in catalog and implementing assessment of outcomes

14. Gavilan Institutional Learning Outcomes (from Palomar College) • Communication • Cognition • Information Competency • Social Interactions • Aesthetic Responsiveness • Personal Development and Responsibility

15. Gavilan Institutional Learning Outcomes • Communication • Listening • Reading • Writing

16. Gavilan Instutional Learning Outcomes • Cognition • Analysis & Synthesis • Problem Solving • Creative Thinking • Quantitative Reasoning • Transfer of Knowledge & Skills to a New Context

17. Gavilan Instutional Learning Outcomes • Information Competency • Research • Technological

18. Gavilan Instutional Learning Outcomes • Social Interaction • Teamwork • Effective Citizenship

19. Gavilan Instutional Learning Outcomes • Aesthetic Responsiveness • Differentiate between people who can sing and people who can’t • e.g. Brittney Spears vs. Norah Jones • Distinguish between art that offends you, art that is cheesy, art you can’t understand, and art that is too expensive

20. Gavilan Instutional Learning Outcomes • Personal Development and Responsibility • Students will develop individual responsibility, personal integrity, and respect for diverse people and cultures • Self-management: Students will demonstrate habits of intellectual exploration, personal responsibility and physical well being • Ethics and Values: Students will demonstrate an understanding of ethical issues that will enhance their capacity for making sound judgments and decisions • Respect for Diverse People and Cultures: Students will respect and work with diverse people including those with different cultural and linguistic backgrounds and different abilities

23. Your Turn! • Think of an instance you directly observed a student learning while interacting with your program or service outside of a classroom experience • Write a mission statement for your program or service

24. Writing Learning Outcomes • Should be based on our mission and values • In an environment that nurtures creativity and intellectual curiosity, Gavilan College serves the community by providing a high quality learning experience which prepares students for transfer, technical and public service careers, life-long learning, and participation in a diverse global society

25. Writing Learning Outcomes • Outcomes must be observable so you can measure or record them • Course level learning outcomes sound like behavioral objectives (or really are the same) but LO tie into institutional learning outcomes • Keep in mind that outcomes are tools to guide evidence collection for assessment

26. Create or Modify Goals and Outcomes Implement Programs or Strategies Evaluate Assessment Cycle

27. Dimensions of Evidence • Quantitative or qualitative • Not everything that can be counted counts and not everything that counts can be counted -Einstein • Direct or indirect • Norm- or criterion-referenced • Should be representative and relevant • Need several pieces of evidence to point to a conclusion • e.g. Student complains of fever and aches, their temperature is 102º F, tonsils are not inflamed, eyes are red and irritated, posture appears weak. Notice mix of types of evidence that all point to same conclusion…flu!

28. Common Assessment Methods • Tests • Locally developed or Standardized • Performances • Recital, Presentation, or Demonstration • Cumulative • Portfolios, Capstone Projects • Surveys • Attitudes and perceptions of students, staff, employers • Database Tracked Academic Behavior • Grades, Graduation, Lab Usage, Persistence • Embedded Assessment • Using grading process to measure ILO • Narrative • Staff and student journals, interviews, focus groups

29. Follow Up CounselingProgram Assessment Example • Beginning Fall 2000, students on academic dismissal are required to visit a Follow Up counselor before enrolling • The main goals of follow up counseling include: • students raising their GPA above 2.0, • increasing the number of completed units, • reducing withdrawals, • encouraging re-enrollment in subsequent terms

30. Follow Up Assessment Details • 68 students in Fall 2000 • 85 students in Spring 2001 • 25 of those students were seen both terms • 105 visits resulting from outreach efforts consisting of at least 153 letters, 115 cards to those not responding to the letter and 53 calls to those not responding to the card • In Fall 2000, 79% of students visited at least once • In Spring 2001, 66% of students visited at least once

31. Outreach Efficacy Increasing efforts showed diminishing returns suggesting sufficient outreach effort occurred

32. No relation between visits and GPA change

33. Difference in Units Earned Unchanged

34. Significant reduction in drops

35. Significant increase in persistence

36. Significant reduction in units attempted, which was not one of the original objectives. Evidence of learning?

38. Embedded Assessment Example Down for grades, across for outcomes assessment –after Nichols

39. Assessment Tips • Collect data from a representative sample rather than everyone in population • Collect only a few well chosen pieces of data • Have assessment stem from activities you already do if possible • Team up with others to share assessment work • Design activities and outcomes with assessment in mind • Ask for advice from your colleagues - especially vocational faculty who relate to learning outcomes as core competencies

40. Writing Learning Outcomes • Good verbs (Blooms’ Taxonomy): • Compile Arrange Classify • Analyze Identify Operate • Design Solve Write • Apply Differentiate Calculate • Demonstrate Formulate Compose • Explain Predict Assess • Compare Estimate Critique • Bad verbs • Know Understand Appreciate Learn • Good verbs are clear and observable, bad verbs are vague states of mind

41. Example Learning Outcomes at program/service level • Disabled Student Services • The student will be able to explain his/her individual academic strengths and weaknesses • Maryland Health Education • Students will demonstrate an understanding of health promotion and disease prevention concepts to establish a foundation for leading healthy, productive lives. • Admissions and Records • Students can “work the system” • Demonstrate patience while waiting in line

42. More Student Service Examples • From Ventura College SLO Workshop • DSPS = Students will demonstrate self-advocacy skills with instructors and staff • EOPS = Student will identify and describe resources available on the campus. • Counseling = Student will be able to state informed academic goal(s). • Financial Aid = Faculty and staff will be able to describe basic aspects of financial aid available to students and how students can apply for financial aid.

43. Gavilan Counseling SLO’s • Students will identify college resources, procedures, and policies that support their academic success. (Cognition) • Students will complete the college orientation and score at least an 80% on the orientation quiz. • Students will use technology to gather, process, and articulate career options and choices. (Info Comp) • Students referred to the Eureka system (online/center) will complete the assessments and discuss outcomes with counselor. (track completion on SARS) • Students enrolled in Guidance 1 will utilize the internet for research and declare a major at the end of the course.

44. Instructional Program Level SLO Examples • Sinclair College • Math = Apply mathematical models to real world problems • Amer. Gov. = Communicate the significance of facts, concepts, and ideas in spoken and written English, which is clear, precise, and logical • ECE = Demonstrate professionalism in the child care setting

45. More Instructional PSLO • Parkland Community College • English = Students will create college-level written text for multiple purposes and diverse audiences that demonstrate depth of critical thought and that observe the conventions of standard written English. • Psychology = Our students will apply major theoretical perspectives and methodological approaches used in the discipline of psychology to particular social phenomena or autobiographical circumstances.

46. Your Turn! • Write at least 2 learning outcomes for your program and at least 2 assessment strategies for each outcome

47. Reflect on Your SLO’s: • Will the student understand? • Is this truly an overarching outcome? • Can you blend? • Does this take into consideration the other areas your program interfaces with? • If you had only 6 outcomes for your program, is this one of them? • In reality, can this be done?

48. Questions to Contemplate • Why should I use student learning outcomes for evaluating my program? • What are some strategies for writing effective learning outcomes? • What are some differences between qualitative and a quantitative measurements? • How can anecdotal reports or case studies be used in program assessment? • What are the pros and cons of using sampling methods for assessment?

49. Thank You