Outcomes assessment Basics

1. Outcomes assessmentBasics Dan Stroud May 25, 2018

3. A Vision for AssessmentCommitment to the Outcome • To provide sufficient support and guidance to help you realize the dividends for the time/effort invested • Enhanced learning • Improved programs/degrees • Greater communication about teaching/learning among faculty • To create a culture of learning, where striving to enrich our students’ learning is ALWAYS what is PROJECTED

4. Some Guiding Assumptions… • Teaching and learning can be improved through systematic inquiry • Assessment is always a work in progress, and it’s ok if things don’t go perfectly • Assessment is about lessons learned in the efforts to enhance learning/teaching • Goal of the Assessment Annual Report = • To demonstrate concerted effort on the part of faculty to examine student outcomes and make appropriate adjustments to improve program

5. FIVE “Big Picture” questions to ask When conducting assessment • How do you define a successful student in your program? • What have you learned about your students’ learning? • What Evidence do you have that proves their skill level? • How are you using that evidence to improve Student understanding in your program? • After implementing a plan of action, how did it change the resulting skill level?

6. Assessing Our University’s (& Your Department’s) Assessment Efforts Compliance Commitment External Questions Internal Questions Number & Amount Quality & Utility Reporting Interpreting Collecting it Using it Accreditation Learning

7. Assessment Components for Academic Degree Programs • Mission statement • Learning Outcomes (usually 4-6) • Remember: SMART • Specific • Measurable • Attainable • Relevant/Results-Oriented • Time-bound

8. Measurements Complete Measurements Process • What instrument? why? • formative and summative assessment? • direct and indirect measure? • When possible, Multiple measures can enhance assessment and make it more meaningful to program • How conduct measurement? • which students? when measured? where? • how administered? by whom? • often good to use smaller samples of students; capstone courses • How collect and store data? • Who analyzes data? how? when?

9. Achievement Targets • What kind of performance do you expect from your students Learning? • What should the Outcomes be? • What is the desirable level of performance for your students ? • Rubrics can clarify this (see the next slides) • What percentage of students do you expect to achieve this?

10. Using Rubrics • A rubric is: “a set of criteria and a scoring scale that is used to assess and evaluate students’ work” (Cambell, Melenyzer, Nettles, & Wyman, 2000). • Addresses performance standards in a clear and concise manner (which students appreciate!) • Clearly articulates to students the areas of improvement needed to meet these standards • To find examples, Google rubrics for your discipline

11. Example of a Rubric Foreign Languages and Literatures Assessment Tool for Oral Proficiency Interview adapted from “Interpersonal Mode Rubric Pre-Advanced Learner” 2003 ACTFL

12. How to build a rubric • Answer the following questions: • Given your broad course goals, what determines the extent of student understanding? • What criterion counts as EVIDENCE of student learning? • What specific characteristics in student responses, products or performances should be examined as evidence of student learning?

13. Developing a rubric helps you to clarify the characteristics/components of your Learning Outcomes: For example: Can our students deliver an effective Public Speech? eye contact gestures volume sources transitions style rate poise examples verbal variety appearance evidence conclusion organization attention getter

14. More rubric help • AACU Rubrics • http://www.aacu.org/value/rubrics • Rubrics from Susan Hatfield (Accreditation Mentor) • www.winona.edu/air/rubrics.htm • Rubistar • http://rubistar.4teachers.org/

15. Findings What do the data tell you? • Part I: specific findings • Compare new data to achievement targets • Did students meet or deviate from expectations? • Important: Include specific numbers/percentages when possible • Do not use course grades or pass rates.

16. Findings (cont.) • what do the data tell you? • Part II: general findings • What lessons did your faculty learn from this evidence about your students? • What broader implications do you draw about your program? • Ex: curriculum, admissions, administration, policies, requirement, pedagogy, assessment procedures, and so on • Conversations • The more people involved, the better!

17. Action Plans • Concrete Steps for Change • list of specific innovations that you would like to introduce in AY 2013-14 to address lessons learned in AY 2012-13. • Again, in curriculum, admissions, administration, policies, requirement, pedagogy, assessment procedures, and so on • Resources? Time Period? Point Person? • It is best to have documentation of the changes made through these Action Plans (e.g., in syllabi, the course catalogue, meeting minutes)

18. Don’t Forget the Assessment Annual Report • Part I: Detailed Assessment Report • “Assessment Narrative • All items (mission -> action plans) submitted in Assessment Management System folder Assessment Deadline December 1 Annually

19. Assessment Plan Narrative • Part II: Timeline/Account of Activities • “Assessment Plan Narrative” • In 1-2 pages, tell the story of all the work and careful consideration you and your colleagues accomplished in your assessment work this year (Ex.: meetings, mentoring, experiments, setbacks, lessons learned) • Plug into your Assessment Folder for that year • Please follow the four outlined questions (see next slide)

20. Four Questions for the Assessment Narrative • Process: Please describe the specific activities and efforts used to design, implement, and analyze your assessment plan during this academic year. This narrative might be organized chronologically, listing meetings, mentoring sessions, and experiments at each stage of the developmental process including the names of people involved in various capacities, with each event given one paragraph. • Positives: Please describe what was most useful about the assessment process, or what went well.  What did you learn about your faculty, students, or program through this experience?  • Challenges: Please describe the challenges you encountered in terms of the development or implementation of your assessment procedures, as well as the lessons you learned from this experience and your efforts or plans for overcoming them. This section might be organized topically. • Support: Please describe your program’s experience during the past year with the support and administrative structures in place at MSU for Assessment:  the Provost’s Office, the University Assessment Committee, the Institutional Research and Assessment office, and so on.

21. Submission: December 1st • No edits allowed after 1st of December • After this date, the Peer Review process will commence with feedback returned by approximately March 1st to the programs. • Also After this date, information gathered and reported can be loaded into the next year’s assessment folder. • The previous year’s Narrative should be held in the new folder until end of year reporting is complete for that cycle.

22. After December 1st • Assessment entries for AY 2013-14 begin • Assessment Cycle runs from Approximately August 15 (End of Summer II for a 12 month cycle. • Necessary to implement the Action discussed in the Narrative and recorded in the previous year’s assessment plan. • Update mission statements, goals, learning outcomes, and measurements based on feedback from UAC. Items in the new assessment folder will carry over from last year unless changed, with the exception of previous year’s documentation. • Begin Entering new findings and formulating new action plans.

23. Questions?

24. Contact Information • For assistance with assessment, please contact • Dr. Dan Stroud, Assessment Specialist at 397-4742 or daniel.stroud@mwsu.edu.