Creating SLO Assessment Plans Presented by ORIE Team Summer 2013

1. Academy for Planning, Assessment, and Research Creating SLO Assessment Plans Presented by ORIE Team Summer 2013

2. Table of Contents • Defining Student Learning Outcomes • Creating PLO or CLO Statements • A. Bloom’s Taxonomy • B. Habits of Mind • III. Selecting Measures • A. Direct vs. Indirect Measures • B. Program Level Measures vs. Course Level Measures • IV. Selecting Evaluation Methods • A. Score Weighting Guide • B. Rubrics • VI. Setting Performance Criteria for Individual Students • VII. Specifying Target Outcome for Group Success in meeting the SLO

3. Defining Student Learning Outcomes The student learning outcome is a statement that explains what the student is learning, including the accumulated and demonstrated knowledge, skills, abilities, behaviors, and habits of mind, as a result of actively participating in the course or program of study. Program Learning Outcomes answer the question: What will students know or be able to do when they exit the program? Course Learning Outcomes answer the question: What will students know or be able to do when they complete the course?

4. Creating Student Learning Outcome Statements • Student Learning Outcome Statements • Should align with the purpose of the program or the course; • Should be specific and • measurable, using action • verbs. • Bloom’s Taxonomy-- • Three Learning Domains: • Cognitive • Affective • Psychomotor

5. Bloom’s Taxonomy SLO Examples: Students will be able to Describe history, purpose, and scope of physical therapy Identify the social, political, economic and cultural influences and differences that affect the development process of the individual. Knowledge: Recall data or information cite, count, define, describe, draw, identify, know, label, list, match, name, outline, point, quote, read, recall, recite, record, recognize, repeat, reproduce, select, state, tabulate, tell, trace, underline.

6. Bloom’s Taxonomy Comprehension: Understand the meaning, translation, interpolation, and interpretation of instructions and problems. State a problem in one's own words. comprehend, compute, convert, defend, differentiate, discuss, distinguish, estimate, explain, express, extrapolate, extend, generalize, give examples, infer, interpret, paraphrase, predict, report, restate, review, rewrite, summarize, translate. Examples of SLO: Students will be able to Distinguish important aspects of the western moral theories, from the virtue theory of ancient philosophers to the modern theories of act ethics. Explain georeferencing of photos or images to maps. Summarize the principles of magnetism.

7. Bloom’s Taxonomy Application: Use a concept in a new situation or applies what was learned in the classroom into novel situations in the work place. apply, calculate, change, compute, construct, demonstrate, determine, discover, dramatize, employ, examine, illustrate, interpret, locate, manipulate, modify, operate, predict, prepare, produce, relate, show, solve, use. SLO Examples Students will be able to Implement plan of care for patients and families within the legal, ethical, and regulatory parameters. Apply appropriate organizational skills for selected types of speeches Conduct basic laboratory experiments involving classical mechanics.

8. Bloom’s Taxonomy Analysis: Separates material or concepts into component parts so that its organizational structure may be understood. Distinguishes between facts and inferences. analyze, appraise, break down, compare, contrast, diagram, deconstruct, differentiate, discriminate, distinguish, identify, illustrate, infer, outline, relate, select, separate. SLO Examples: Students will be able to Analyze and design electrical and electronic circuits and systems, using Knowledge of mathematics and basic sciences. Analyze self-practice in relation to the roles of the professional nurse.

9. Bloom’s Taxonomy Synthesis: Builds a structure or pattern from diverse elements. Put parts together to form a whole, with emphasis on creating a new meaning or structure. categorize, combine, compile, compose, create, devise, design, explain, generate, integrate, modify, organize, plan, rearrange, reconstruct, relate, reorganize, revise, rewrite, summarize, tell, write SLO Examples: Students will be able to Integrate the pathophysiological assessment findings to formulate a field impression. Coordinate human information and material resources in providing care for patients and their families.

10. Bloom’s Taxonomy Evaluation: Make judgments about the value of ideas or materials. appraise, compare, conclude, contrast, criticize, critique, defend, describe, discriminate, evaluate, explain, interpret, justify, relate, summarize, support SLO Examples: Students will be able to Use critical thinking and a systematic problem-solving process for providing comprehensive care. Analyze and evaluate economic events that apply to the preparation of financial statements.

11. Habits of Mind Arthur Costa and BenaKallick (2000)’s Habits of mind: Persisting, thinking flexibly, and striving for accuracy Examples of Habits of Mind: • Develop and use effective time management skills • Follow directions correctly • Appreciate negatives and failures as opportunities to learn and discover choices SLO Examples: Students will demonstrate time management skills, adhering to all deadlines for assignments, tests, and projects. Students will demonstrate effective therapeutic patient communication/relationship.

12. Selecting Measures A. Direct vs. Indirect Measures To measure student learning outcomes, use the direct measures primarily, but also use indirect measures for triangulation of data Direct Measures: Refer to the type of data that can be used to directly measure students’ knowledge and skills Examples: Pre-test, post-test, comprehensive subject matter exam, licensure exam, portfolio, thesis, writing assignment, internship evaluation, capstone assignment, and performance demonstration (recital, art exhibit, or science project)

13. Selecting Measures Indirect Measures: Refer to the type of data that can be used to infer student learning or achievement Examples: Surveys, Interviews, Focus Group Studies, Document Analyses, or Audit Measures (course grades, retention/graduation/job placement rates)

14. Selecting Measures • B. Program Level Measures vs. Course Level Measures PLO Measures: Exit exam (standardized, licensure/professional exams) Thesis, Dissertation Capstone Course Paper or Project Portfolios CLO Measures: Exams, Tests Rubric-Based Assignments: Projects Essays Portfolios Presentations Performance Demo.

15. Selecting Measures C. How to Select Measures?

16. Examples of Measures Aligned with SLOs

17. Examples of Measures Aligned with SLOs

18. Selecting Evaluation Methods Aligned with Measures

19. Selecting Evaluation Methods Aligned with Measures

20. Creating a Rubric • A rubric is a systematic scoring guide to evaluate student performance. • Three Essential Parts of a Rubric: • A Set of Indicators of Learning • A Scale • A Set of Descriptors

21. Creating a Rubric • Indicators fora rubric-based assignment (essay, project, speech, performance) should be aligned with the instructional objectives. If an instructor’s objectives of teaching an essay are related to helping students compose an essay that is focused, coherent, organized, etc., then these features should serve as indictors of learning. • Example of Indicators for an Essay for an English Class: • Focus • Coherence • Organization • Sentence Structure • Word Choice

22. Creating a Rubric A scale indicates points to be assigned in scoring a piece of work on a continuum of quality. High numbers are typically assigned to the best work. Scale examples: Needs Improvement (1)... Satisfactory (2)... Good (3)... Exemplary (4) Beginning (1)... Developing (2)... Accomplished (3)... Exemplary (4) Needs work (1)... Good (2)... Excellent (3) Novice (1)... Apprentice (2)... Proficient (3)... Distinguished (4)

23. Creating a Rubric Descriptors are used to describe signs of performance at each level; the description needs to include both strengths and weaknesses (Weaknesses should be described particularly in lower levels of performance). Examples of Descriptors: Word Choice 4-- Vocabulary reflects a thorough grasp of the language appropriate to the audience. Word choice is precise, creating a vivid image. Metaphors and other such devices may be used to create nuanced meaning. 3-- Vocabulary reflects a strong grasp of the language appropriate to the audience. Word choice is accurate, but may be inappropriate in a couple of places. 2-- Vocabulary reflects an inconsistent grasp of the language and may be inaccurate or inappropriate to the audience. 1-- Vocabulary is typically inaccurate and inappropriate to the audience. Word choice may include vague, non-descriptive, and/or trite expressions.

24. Example of a Rubric (A Snapshot)

25. Setting Criteria for Meeting SLO To determine whether students have successfully met the learning outcomes, we need to specify what score or rating an individual student needs to get in order to be considered as meeting the learning outcome. Examples: Measure: Examon the history and purpose of Physical Therapy Evaluation Method: Score Weighting Guide with scores assigned to each exam question Criterion for Meeting the SLO: A score of 70 or above on the exam Measure: Essaydescribing the scope of physical therapy Evaluation Method: Grading Rubric specifying levels of quality in identified areas Criterion for Meeting the SLO: A score of 3 or above on a scale of 1-4

26. Specifying Target Outcomes To determine whether students as a group have successfully met the learning outcomes, we need to specify what % from the group should meet the learning outcome. Example I: Measure: Examon the history and purpose of Physical Therapy Evaluation Method: Score Weighting Guide with scores assigned to each exam question Criterion for Meeting the SLO: A score of 70 or above on the exam Target Outcome: 85% of the students will score 70 or above on the exam.

27. Specifying Target Outcomes To determine whether students as a group have successfully met the learning outcomes, we need to specify what % from the group should meet the learning outcome. Example II: Measure: Essay describing the scope of physical therapy Evaluation Method: Grading Rubric specifying levels of quality in identified areas Criterion for Meeting the SLO: A score of 3 or above on a scale of 1-4 Target Outcome: 70% of the students will score a 3 or above on their essays

28. Recap: SLO Assessment Plan

29. PLO Assessment Plan: (Example)

30. CLO Assessment Plan: (Example)

31. Do’s and Don’ts When Creating an SLO Assessment Plan Do’s • When creating PLO or CLO statements, DO include both lower level and higher level of cognitive skills: not only knowledge, comprehension, but also application, analysis, evaluation, and synthesis. • When selecting assessment measures for PLOs, DO include both course-embedded assignments and program level measures (thesis, exit exams, or national exams, or graduate portfolios, or internship project, or capstone course project). • Encourage faculty participation in the design of the Assessment Plan. • Try to use common assessment assignments or tools, but allow for flexibility.

32. Do’s and Don’ts When Creating an SLO Assessment Plan Don’ts Don’t be overly ambitious—Create only 5 to 7 PLOs or CLOs, and use only 7-10 pieces of data!

33. Academy for Planning, Assessment, and Research