Articulating and Mapping Outcomes in Student Affairs

1. Articulating and Mapping Outcomes in Student Affairs Dr. Marsha Watson Director of Assessment Dr. Kenny Royal Assistant Director of Measurement & Analysis Dr. Julie Johnson Assessment Specialist

2. The LEARNING Initiative Dual Track Implementation Strategy

3. Institutional Expectations • Academic units articulate and assess student learning outcomes aligned with the institution’s academic expectations • Where are the institution’s other expectations – like “tolerance,” “teamwork,” and “the ability to function in an increasingly diverse world” taught and assessed? • Is “taught” even the right word?

4. Shifting Perspectives • Substitute “learned” for “taught” – and the question becomes: Where are these other outcomes learned and assessed? • This shifts our perspective to: • Student demonstration emphasis instead of topic-covering • Ability to do or apply supersedes knowing • Shared responsibility for learning • Site of learning becomes less specific; boundaries become more porous

5. After Learning Reconsidered • Learning is not exclusively classroom-based • Many important outcomes are: • Not learned in the classroom • The result of processes outside the classroom • “Learning” is a process based on three interdependent student experiences: • Understanding academic content and processes • Student development • Identity formation

6. Developmental Paradigms • Piaget: 4 stages of cognitive development • Perry’s 9 stages of cognitive development • http://www.uky.edu/TASC/ED/perry.php • Kegan: 6 stages of psychological development • Kohlberg: 6 stages of moral development • Expands on Piaget’s two-stage theory of moral thought

7. After Learning Reconsidered • Responsibility for learning: • Exists outside the classroom • Doesn’t always take the same form • Some entities on campus produce learning, some facilitate learning, some support learning • Thus, responsibility for assessing learning exists outside the classroom as well

8. Learning in Non-Classroom Settings Critical thinking Working with others Global competence Communication Adaptability Preparation for life-long learning Ethical behavior

9. Co-Curricular Learning Outcomes • The cognitive and/or affective competencies we intend students to be able to acquire with what they have learned • Achievement of outcomes indicates institutional and departmental effectiveness • Examples of co-curricular learning outcomes

10. Co-Curricular Learning Outcomes • Must be: • Intentional • Carefully planned and designed • Coordinated • Carefully implemented • Part of the structure of a student’s experience • Assessed

11. Co-Curricular Assessment • The “Effectiveness Model” • Focus on student performance as a result of – or even within – a planned, structured co-curricular activity • Can be based on observation or objective measures • Requires carefully designed and consistent measuring practices

12. Formulate Statements of Intended Learning Outcomes Discuss and Use Assessment Results to Improve Learning Develop or Select Assessment Measures Formative-Based Revisions Create Experiences Leading To Outcomes

13. Activity #1 (15 minutes) • The Evidence Inventory • Working in groups, identify direct and indirect student learning data you are already collecting • Report back

14. Questions to Ponder as You Plan • What action/experience are you assessing? • What specific outcome is it meant to develop? • How are you assessing that development? • What specific information about that outcome’s development do you want to know? • What will you do with the information you gather?

15. Co-Curricular Assessment Example: Group discussion of USC mini-case study (10 minutes)

16. 15 Minute Break!!

17. DesignBackward Intended Learning Outcomes of the Student Affairs Division Intended Learning Outcomes of the Institution Intended Learning Outcomes of the Activity Intended Learning Outcomes of the Program Intended Learning Outcomes of the Department Deliver Forward Revised: Huba, M.E. and Freed, J.E. (2000).  Learner-Centered Assessment on College Campuses: Shifting the Focus from Teaching to Learning (p. 108). Allyn & Bacon, Needham Heights, MA.

18. Activity # 2: Dr. Johnson • Review: Handout #4 (Design Backward/Deliver Forward) • Review University of Kentucky’s Mission/Vision • Review Values/Outcomes of Student Affairs • Review New Student and Parent Programs • Review K-Week • Discuss Learning Outcomes and K-Week Activity Mapping

19. Planning for the Long Term • Mapping and Coordinating • If the learning outcome is important, a single exposure isn’t enough • Map activities/experiences to outcomes, from first year to last year • Determine how first-year experiences are different from last-year ones • What difference is expected in student response? • Plan to assure student’s development of outcomes from the first year to the last • Consider how co-curricular activities might be coordinated with classroom instruction

20. Basic Mapping Template I = Outcome is introduced R = Outcome is reinforced E = Outcome is emphasized

21. Regular (Periodic) Assessment • One-shot assessment produces haphazard results that are usually insufficient for planning improvement • Tie assessments to logical stages of development, based on an outcomes map • Be consistent in approach to assessing • Options: • Standardized instruments • Self-generated tools

22. Questions to Ask about Evidence • Is it relevant to the area’s stated mission and function? • Does it measure what we want it to measure? • Does it deal in some way with outcomes? • Is the information derived useful? • Can the information be used to improve either function or learning?

23. Developing Measures of Effectiveness • Intentional Planning • Determine areas of responsibility: what office/function might be a logical place to contribute to particular learning outcomes? • Plan the outcomes-based purpose of the activity • Design non-passive activities • Ex: Watching a film plus discussion; International Days as more than food, costumes, dance … • Develop/design outcome-focused opportunities for processing

24. Surveys and Questionaires • Limitations: • Self-reporting, unvalidated opinion • Response rates • “Opportunistic” data • Skewed samples

25. Surveys and Questionaires • Make them tools to (indirectly) assess learning • Use learning outcomes as basis for at least some questions • Validate by cross-referencing outcomes with different populations • employers, alumni, graduates, current students, etc. • Emphasize the learning outcomes in design and analysis

26. Self-Generated Tools • Observations • Expert judgments • Student self-reflection • Peer assessments • Group/team discussion • Employer/supervisor judgments

27. University-Wide Assessment • Ideally, should use the same rubrics or measures • all who use them should have the same understanding of its elements • When using the same tools is not possible, it is essential that there be a way to extract information that is usable at the institutional level while still serving the needs of the functional area

28. Activity #3 • Refer to Handout #8 (Activity #3 Worksheet) • Working in groups, revise/refine two co-curricular learning outcomes • Identify appropriate: • student level (freshman, sophomore, etc) • stage of cognitive development • Design a measurement(s) for each outcome • Report back the large group • Map the refined outcomes to learning activities/experiences • Handout #9

29. One last thing … Please complete the workshop evaluation Thanks!