SLO & SAO Workshop

1. SLO & SAO Workshop Dr. Deborah DiThomas Interim Vice Chancellor Student Services & Operations Cuesta College Student Services Managers Dr. Edward C. Bush Vice President, Student Services

2. SLO 10 Commandments • Acknowledge the SLO/SSO and keep it holy. • Thou shall not do a SLO/SSO in isolation. • Thou shalt make SLOs/SSOs relevant. • Thou shall not use the word data in vain. • Thou shall not bite off more than one can chew. • Thou shall not covet your neighbor’s SLO/SSO. • Thou shall not keep your SLO/SSO on the shelf. • Thou shall not ever become complacent. • Thou shall not kill, hurt, or injure anyone during the SLO/SSO process, • but cursing is okay.

3. History of RCCD Program Review 05-06 : Spring 05: New Standards/SLO Training Draft Program Reviews Spring 05: Student Services Retreat Review Accreditation Standards Revise SLO’s Revise SAO’s Write Evaluation Plans Summer 05/Fall 05: Finalize Program Reviews including SAO’s, SLO’s and Assessment Plans Spring 06: Implement “Assessment Plans” for SAO’s & SLO’s Summer 06: Begin ‘06 Program Reviews

4. What We Learned: Better understanding of campus programs Make our SLO’s and SAO’s more singular—don’t bundle Pick data that can be measured We were too broad Chose one or two specific things to work on Challenged in serving students at the same time we were assessing students Write more measurable objectives See things through the eyes of students Get staff involved Follow the template it gives direction Focus and prioritize Focus of how we state SLO’s and SAO’s Need a way to capture data

5. What We Did Well: Keeping good records so at end it was easy—use timelines Organization of how we did things—using timelines Worked well together—shared responsibilities To truly serve students it takes one on one Education and dissemination of safety information to college community Documented needs Completing the entire process makes us better for next year Comfort level is higher Ahead of academics and using us as example Teamwork Need to communicate changes to students Get staff involved Instruments to measure Increased communication

6. We will do differently: • Get more input from hourly staff • Don’t wait until end to do evaluation • Better timelines and stick to the timelines (monthly calendars…) • Get all members of department on board • Improve communications with students • Identify specific people to do certain things • Include communication with academic side • More campus specific SLO’s SAO’s • Change format of last year

7. Defining SLOs & SSOs Student Learning Outcome - Defines whatstudents should be able to do, think, or know. Service Area Outcome - Defines how the service area performs.

8. Defining Outcomes Detailed, specific, measurable or identifiable, and personally meaningful statements that are derived from the goals and articulate what the end result of a unit, program, course, activity, or process is. Outcomes should be: • Measurable (not necessarily, countable), or observable, • Manageable, and • Meaningful.

9. Standard IIB – WASC The institution recruits and admits diverse students who are able to benefit from its programs, consistent with its mission. Student support services addresses the identified needs of students and create a supportive learning environment. The entire student pathway through the institutional experience is characterized by a concern for student access, progress, learning, and success. The institution systematically assesses student support services using student learning outcomes, faculty and staff input, and other appropriate measures in order to improve the effectiveness of these services.

10. What Does This Mean? • The new standards promote the values of Student Services. • Student Service professionals are leaders in providing responsive services to a diverse and changing student population. • The learning outcomes challenge to Student Services is an opportunity to improve effectiveness and to integrate with instruction.

11. Standard IIB – continued …The institution systematically assesses student support services using student learning outcomes, faculty and staff input, and other appropriate measures in order to improve the effectiveness of these services…

12. Effectiveness is the Ultimate Goal • Learning outcomes focus on what students know and can dorather than what we do to support them. • Shift the paradigm from Student Satisfaction to Student Learning. • Student affairs needs to demonstrate effectiveness thru data. • Campus staff learn how student affairs contributes to student learning.

13. Perhaps the key question to be asked of student affairs personnel might be expressed as follows… “Of those students who use the services, programs and facilities, is there any effect on their learning, development, academic success or other intended outcomes, particularly when compared to non-users?” It is important to note that not all outcomes of interest to student services are learning outcomes; many pertain to the timely, efficient and effective delivery of services. Source: Upcraft, M. Lee and Schuh, John. Assessment in Student Affairs: A Guide for Practitioners. Jossey-Bass, San Francisco, 2000. ISBN 0-7879-0212-8

14. Formulating Service AreaOutcomes (SAOs) Service Area Outcomes Based on Currently Existing Services Name of Unit will provideName of Current Services improve decrease provide Client will be satisfied with Name of Current Service

15. Formulating Student Learning Outcomes (SLOs) Student Service Areas SSA ServiceVerb + Objective Tutoring Improve… Students attending Academic Advising will Increase… Workshops Understand… Counseling Sessions Know…. “Students participating in the development of a student educational plan will understand the educational requirements for their ed goal and major.”

17. Checklist: Student Learning Outcomes • Consistent with Mission Statement. • Agreement ~ were staff involved in the development? • Reasonable ~ for the ability of the students. • Measurable ~ can be observed, tested. • Key Concepts ~ important to the service area, program. • Clarity ~ precise in description. • Singular ~ not “bundled” (shown by use of conjunctions, commas).

18. Critique & Correct RCC Outcomes

19. Group Task: Let’s Look at SLOs and SAOs

22. Part II:Navigating Methods for Assessing Student Learning Outcomes in Student Services

23. The Assessment Process

24. Good Assessment Techniques • Valid • Reliable • Actionable • Efficient and cost-effective • Engaging to students and others • Interesting to faculty and other stakeholders • Triangulated

25. Types of Assessment Direct assessments- provide students the opportunity to show what they know. They “…prompt students to represent or demonstrate their learning or produce work so that observers can assess how well students’ texts or responses fit institutional or program-level expectations.” (Maki, p. 88) Indirect assessments -“…capture students’ perceptions of their learning and the educational environment that supports that learning, such as access to and the quality of services, programs, or educational offerings that support their learning…By themselves, results of indirect methods cannot substitute for the evidence of learning that direct methods provide. They can, however, contribute to interpreting the results of direct methods…” (Maki, p. 88, 89)

26. Types of Assessment (cont) Formative Assessment-is designed to assist the learning process by providing feedback to the learner, which can be used to identify strengths and weakness and hence improve future performance. Formative assessment is most appropriate where the results are to be used internally by those involved in the learning process (students, teachers, curriculum developers). Summative Assessment-is used primarily to make decisions for grading or determine readiness for progression. Typically summative assessment occurs at the end of an educational activity and is designed to judge the learner’s overall performance. In addition to providing the basis for grade assignment, summative assessment is used to communicate students’ abilities to external stakeholders, e.g., administrators and employers.

27. Types of Assessment (cont) Qualitative research - uses open-ended questions to gain an in-depth understanding of the questions being explored. Common qualitative techniques include focus group discussions, mini-groups, and in-depth interviews either face-to-face or via telephone. Quantitative methods - are distinguished by their emphasis on numbers, measurement, experimental design, and statistical analysis. Researchers typically work with a small number of predetermined response categories to capture various experiences and perspectives of individuals. Often emphasis is on analyzing a large number of cases using carefully constructed instruments that have been evaluated for their reliability and validity (Patton, 1990). Techniques include questionnaires, structured interviews, and tests.” (Palomba and Banta, p. 337) Both qualitative and quantitative methods bring valuable information to light, but they yield different types of results. Review the type of information needed when selecting a method, and also consider whether qualitative and quantitative research could be used together.

28. Common Assessment Methods • Performances and Staff Observations • Presentation, or demonstration • Surveys • Attitudes and perceptions of students and staff • Narrative • Staff and student journals, interviews, focus groups • Cumulative • Portfolios and capstone projects • Database-Tracked Academic Behavior • Grades, graduation, lab usage, persistence

29. Your Charge: Lets Practice Assessment

30. Your Task… Work with the members of your service area to: • Develop an assessment plan: • Consider types of evidence. • Utilize existing assessment tools if appropriate. • Consider partnering with other student service areas. • Close assessment loop. • Create an annual timeline for assessment. • Delineate people responsible for the various segments of the assessment plan.

31. RCCD Student Services Assessment of SLO’s and SAO’s • Decide on how to assess your SLO’s & SAO’s • Can have more than one assessment for any of the SLO’s or SAO’s • Consider assessment tools that are easy for your area to use • Create own assessment tools, if necessary • Decide where the assessment will happen • Decide who will conduct the assessment • Decide when it will be conducted

32. Checklist: Means of Assessment • Specific means of measurement identified. • Overall score, rating or response level set. • Sub-score minimum of a lower scale in any given area established. • Number of students/clients who meet score given. • Administration when and by whom defined. • Results judged on what basis stated. • Results judged by whom stated.

34. ASSESSMENT WORKSHEET

35. Definitions • Student Learning Outcome - Defines whatstudents should be able to do, think, or know. • Service Area Outcome - Defines how the service area performs. • Audience - Who is your program/service targeting? • Circumstance/Condition Intervention - Where or under what circumstances/ conditions will the learning occur? • Benchmark data - Data that serves as a standard by which others may be measured that serves as a basis for evaluation or comparison • Significant Degree - Target performance level that states how much learning will be enough? • Data Collection – What assessment methods and tools will be used to measure learning or performance? • Person (s) Responsible – Identify titles of persons responsible for carrying out SLO/SAO. • Timeline for data collection – Set a goal for data collection. • SLO/SAO Data Analysis – Summarize the outcome(s) of the study. • Recommended Improvement – How will you use the outcome(s) for service area improvement?

36. Implement Assessment Plans (for SAO’s & SLO’s) Spring

37. Implementing Assessment Results • Discuss and disseminate • Make appropriate program improvements • Integrate with program review and strategic planning process

38. RCCD Student Services Assessment of SLO’s and SAO’s • Assess the objectives and collect the data: • Conduct the Assessment • Organize the data • Summarize your findings • Disseminate results • Decide what to do next based on the findings • Happy with the results? You congratulate yourself and select another SLO/SAO to measure… • Not happy with the results? What can you do in your area to make changes?

39. Thank You&Good Luck!