1 / 55

Academic Program Workshop for Creating an Assessment Plan

Academic Program Workshop for Creating an Assessment Plan. Facilitated by Pamela Baker, PhD, Director, CET&L Laurah Turner, PhD, Assistant Director, CET&L. Session Outcomes. Identify and explain to colleagues the purpose and process of assessment.

min
Download Presentation

Academic Program Workshop for Creating an Assessment Plan

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Academic Program Workshop for Creating an Assessment Plan Facilitated by Pamela Baker, PhD, Director, CET&L Laurah Turner, PhD, Assistant Director, CET&L University of Cincinnati, Center for the Enhancement of Teaching & Learning

  2. Session Outcomes • Identify and explain to colleagues the purpose and process of assessment. • Identify and evaluate the components of an assessment plan. • Develop a framework for the ideal graduate of your program. • Draft strong program-level learning outcomes. • Understand alignment of outcomes at the program, course, and unit-level. • Conduct a preliminary audit of program-level assessments. • Draft a curriculum map for your program that identifies specific courses where program learning outcomes are addressed. • Develop a plan to gather & analyze assessment data to determine how well student learning matches expectations(with a view to identify action to be taken based on findings). University of Cincinnati, Center for the Enhancement of Teaching & Learning

  3. Session Timeline • 9:00-9:15  Introductions and Program Overview • 9:15-9:30  Identify & Explain the Purpose of an Assessment Plan • 9:30-9:45  Identify & Evaluate the Components of an Assessment Plan • 9:45-10:05  Develop a framework for the Ideal Graduate • 10:05-10:45  Identify Strong Program Outcomes • 10:45-10:55  BREAK • 10:55-12:00  Review Program Outcomes • 12:00-12:30   LUNCH • 12:30- 1:15  Conduct a Preliminary Audit of Program Level Assessment • 1:15-2:15  Draft a Curriculum Map • 2:15:2:25  BREAK • 2:25-3:25  Develop a plan to gather and analyze assessment data (with a view to identify action to be taken based on findings). • 3:25-3:30  Q & A University of Cincinnati, Center for the Enhancement of Teaching & Learning

  4. Overview: Identify & Explain the Purpose & Process of Assessment • What is the purpose of program assessment within the context of Semester Conversion & Higher Ed? • What is assessment of a program? • Program Effectiveness (e.g. the program will graduate 90% of students, & 70% of those students will earn degrees within five years) • Program Outcomes (i.e. emphasis on demonstrating what students can do as a result of completing a program) • What is the common purpose of assessment? • A tool to make meaningful improvements and appropriate interventions • A tool to ensure efficient allocation of resources • A way to highlight strengths within a curriculum • Reaccreditation University of Cincinnati, Center for the Enhancement of Teaching & Learning

  5. Assessment, Continued SHORTER TERM • Ascertain whether what our students have learned is good enough • Identify assessment methods & measures • Identify whether these assessment tools are assessing what it is we expect our students to learn • Identify whether required courses/experiences within the curriculum align with the program level outcomes • Determine whether our students are learning at the appropriate scope, depth, and level LONGER TERM • Use long-term findings to make more significant curriculum revisions (beyond simple fine-tuning of programs) • Use assessment evidence to make informed decisions about your program within the context of college and institutional priorities • Identify external benchmarks to evaluate the learning of our students • Sharing evidence & best practices Adapted from Linda Suskie, “Why Are We Assessing?,” Inside Higher Ed (October 26, 2010), adapted from her talk at the 2010 Assessment Institute. (Dr. Suskie serves as Vice President of the Middle States Commission on Higher Education.) & Susan Hatfield, “Assessing Your Program-Level Assessment Plan,” Idea Paper #45, The Idea Center. University of Cincinnati, Center for the Enhancement of Teaching & Learning

  6. Anatomy of an Assessment Plan • Program Outcomes that can be measured, assessed, or observed in some way • Curriculum/Program Map connections identified between courses and program-level outcomes as well as clear levels of learning at each level (emerging, strengthening, achieved) • Methods/Measured articulated, which include direct & indirect examples of student learning University of Cincinnati, Center for the Enhancement of Teaching & Learning

  7. Anatomy of an Assessment Plan, Continued • Assessment Infrastructure comprised of program faculty charged with overseeing the execution of the assessment plan, with efforts linked to college and institutional assessment efforts & goals • Findings for several years explained, patterns and trends identified (data collected & reviewed on an annual basis) • Use of Findings discussed among faculty; learning outcomes and pedagogical strategies reviewed and revised based on assessment data University of Cincinnati, Center for the Enhancement of Teaching & Learning

  8. ACTIVITY 1: Envisioning the Ideal Graduate The Task: Brainstorm the “ideal” graduate of your program. Be sure to describe how you imagine your ideal graduates once they become professionals in their field. • What does an “ideal” graduate of your program look like? • How will your graduates think and behave? • What are 3-4 important things your students will learn to do in your program? • What do students in your program need to learn to prepare them for a job? • What would you like your students to be doing consistently 5 years from now? University of Cincinnati, Center for the Enhancement of Teaching & Learning

  9. ACTIVITY 1: Continued • If these category listings are not applicable to your program, please insert alternative headings. University of Cincinnati, Center for the Enhancement of Teaching & Learning

  10. Program Outcomes • Outcomes are achieved results or consequences of what was learned (i.e. evidence that learning took place). • Outcomes should be framed in terms of the program and not individual courses or students. • Outcomes are student-centered and describe what a student should be able to do or understand upon graduation. • Outcomes should describe the abilities, knowledge, values and attitudes expected of students after completion of the program. • Outcomes must be stated in such a way that they can be measured or observed by more than one assessment method. University of Cincinnati, Center for the Enhancement of Teaching & Learning

  11. How Program Outcomes Fit Within a Curriculum Represent broad statements incorporating areas of inter-related knowledge and skills developed throughout a program, across courses and experiences. Represent the measurable and observable knowledge, skills, abilities, or attitudes students should have by the end of the course. Course outcomes should align with the program outcomes. Represent the measurable and observable knowledge, skills, abilities, or attitudes students should have by the end of the unit. Unit outcomes should align with the course outcomes. Represent the knowledge, skills, abilities, or attitudes that students should achieve by the end of an individual class session. Class outcomes should align with the unit outcomes. University of Cincinnati, Center for the Enhancement of Teaching & Learning

  12. How Program Outcomes Fit Within a Curriculum Represent the measurable and observable knowledge, skills, abilities, or attitudes students should have by the end of the course. Course outcomes should align with the program outcomes. University of Cincinnati, Center for the Enhancement of Teaching & Learning

  13. How Program Outcomes Fit Within a Curriculum Represent the measurable and observable knowledge, skills, abilities, or attitudes students should have by the end of the unit. Unit outcomes should align with the course outcomes. Represent the knowledge, skills, abilities, or attitudes that students should achieve by the end of an individual class session. Class outcomes should align with the unit outcomes. University of Cincinnati, Center for the Enhancement of Teaching & Learning

  14. Unit & Class-Level Outcomes • Move from content covered to student action • Consider how in-class activities can model or reinforce learning outcomes • Make course calendar more explicitly supportive of learning outcomes University of Cincinnati, Center for the Enhancement of Teaching & Learning

  15. How Program Outcomes Fit Within a Curriculum Represent broad statements incorporating areas of inter-related knowledge and skills developed throughout a program, across courses and experiences. University of Cincinnati, Center for the Enhancement of Teaching & Learning

  16. What are Program Outcomes? Program outcomes represent broad statements that incorporate many areas of inter-related knowledge and skills developed over the duration of the program through a wide range of courses and experiences. They represent the big picture, describe broad aspects of behavior, and encompass multiple learning experiences. University of Cincinnati, Center for the Enhancement of Teaching & Learning

  17. Program Outcomes… • are achieved results or consequences of what was learned (i.e. evidence that learning took place) • should be framed in terms of the program and not individual courses or students • are student-centered and describe what a student should be able to do or understand upon graduation • should describe the abilities, knowledge, values and attitudes expected of students aftercompletion of the program • must be measurable or observable by more than one assessment method University of Cincinnati, Center for the Enhancement of Teaching & Learning

  18. Bloom’s Taxonomy Action Verbs Universtiy of Cincinnati, Center for the Enhancement of Teaching & Learning

  19. Program Outcomes, Examples: Undergraduate • Examples: • Too generic: Students completing the Engineering program will be practiced in design skills. • Specific/Assessable: Engineering graduates will demonstrate knowledge of math, science, and engineering fundamentals. Specifically, the student will have the ability to: • Demonstrate general design principles. • Use fundamental engineering techniques, skills and tools for engineering practice. • Analyze and interpret data to produce meaningful conclusions and recommendations. University of Cincinnati, Center for the Enhancement of Teaching & Learning

  20. Program Outcomes: Undergraduate Bachelor Degree in Business (BBA) Upon completion of the BBA Program, students will be able to: • Apply functional and cross-functional knowledge to critically assess business problems • Use analyses to inform and develop integrative solutions that improve business outcomes • Express ideas clearly, logically and persuasively in both oral and written formats • Recognize ethical and social responsibility issues in a business environment and know how to apply a process of ethical inquiry • Show how operating in a global market creates business opportunities and challenges University of Cincinnati, Center for the Enhancement of Teaching & Learning

  21. Program Outcomes: Undergraduate Bachelor Degree in Health Science (HLSC): Upon completion of the HLSC Program, students will be able to: • Plan, execute, evaluate, adjust physical activities and programs appropriate for self and clients • Provide complete, understandable and accurate information within content areas to clients and community • Develop and/or disseminate scientific information to the general public and the academic community • Apply information from various basic and applied science disciplines in a manner that provides for efficient, effective and safe physical activities University of Cincinnati, Center for the Enhancement of Teaching & Learning

  22. Program Level OutcomesCategories for Graduate Level • Need to be progressively more advanced in academic content • Need to address discipline knowledge and research or professional practice • Need to address technology skill development (if appropriate) • Need to be informed & aligned with professional organization standards, licensing bodies, or specialized accreditors • Professional development of graduate students University of Cincinnati, Center for the Enhancement of Teaching & Learning

  23. Example Student Learning Outcomes at the Graduate Level Upon completion of the physics PhD program, students will be able to: • identify and describe the broad fundamental concepts of physics; • analyze physics problems using fundamental physics concepts and appropriate mathematical models; • solve physics problems using analytic, computational, observational, or experimental methods; • present clearly organized oral explanations of physics topics at a level appropriate to the audience at hand; • explain and analyze at an expert level the concepts and the state of current research in a subfield of physics; • compose and defend written presentations of physics research in a format and at a level appropriate for journal publication; • conceptualize, design, and implement new approaches for solving research problems in physics, including analytic, computational, observational, and experimental methods, as appropriate. Universtiy of Cincinnati, Center for the Enhancement of Teaching & Learning

  24. Example Student Learning Outcomes at the Graduate Level • Professional Development • Ethics Universtiy of Cincinnati, Center for the Enhancement of Teaching & Learning

  25. Activity 2: Define Student Learning Outcomes • Focused on the learner, rather than on the content covered • Attainable, rather than aspirational • Describe observable and demonstrable skills and knowledge • Reflect how the learner will use course content now and in the future • Indicate specific and measurable elements that will be assessed using multiple assessment methods • Identify in the curriculum where the PO will be introduced, developed and achieved University of Cincinnati, Center for the Enhancement of Teaching & Learning

  26. ACTIVITY 2b: Peer Review of Program Outcomes • Gallery Walk University of Cincinnati, Center for the Enhancement of Teaching & Learning

  27. Program Outcomes and Your Program Assessment Plan • PO’s are the foundation of an assessment plan The Process of Assessment University of Cincinnati, Center for the Enhancement of Teaching & Learning

  28. Activity 3: Curriculum Mapping Matrix • To identify where key learning outcomes are addressed in the curriculum • To identify whether courses are aligned with the curriculum • Specifically, to determine/identify whether course-level student learning outcomes align with program-level outcomes • Note: You’ve already completed the process of mapping course-specific student learning outcomes to program outcomes, when you completed the P-1. University of Cincinnati, Center for the Enhancement of Teaching & Learning

  29. Activity 3: Continued Instructions • List the program learning outcomes identified in the P-1 in the column labeled “Program Learning Outcomes.” • List the required courses and experiences identified in the P-1 across the top row. NOTE: Do not list courses or experiences controlled by external academic units. Only list courses and experiences that are controlled by your program. If you identify a program outcome, you must be able to assess it. • Identify which course(s) and/or experiences utilize each program learning outcome and at what level the program learning outcome is addressed (e.g. emerging, developing, achieved) University of Cincinnati, Center for the Enhancement of Teaching & Learning

  30. Universtiy of Cincinnati, Center for the Enhancement of Teaching & Learning

  31. Universtiy of Cincinnati, Center for the Enhancement of Teaching & Learning

  32. Identify & Describe Corresponding Program Outcomes & Assessment Methods: Conduct a Preliminary Audit Circle any of the measures used in your academic program University of Cincinnati, Center for the Enhancement of Teaching & Learning

  33. Why Not Just Use Course Grades?

  34. Why aren’t course grades adequate indicators for program assessment? • Course letter grades are insufficient for program assessment because: • Letter grades are nominal values reflecting a summary of a student’s performance. • Grades are not an exact measurement of what students have and have not learned. • Grades do not measure achievement of program outcomes, learning of specific skills, or the capacity of the curriculum to achieve program outcomes.

  35. Example: Outcomes for an Undergraduate Economics Program • Students should be able to identify, explain, and use economic concepts, theories, models; and data-analytic techniques. • Students should be able to deliver effective presentations in which they combine visual communication design with oral arguments and/or the written word. • Students should use investigative skills necessary for conducting original economic research and participating effectively in project teams. • Students should be able to apply their economic tools to formulate positions on a wide range of social and economic problems and engage effectively in policy debates.

  36. Example Curriculum Map for an Undergraduate Economics Program

  37. Letter grades are nominal values reflecting a summary of a student’s performance

  38. PO 1. Students should be able to identify, explain, and use economic concepts, theories, models; and data-analytic techniques PO 2. Students should be able to deliver effective presentations in which they combine visual communication design with oral arguments and/or the written word.

  39. PO 2: Students should be able to deliver effective presentations in which they combine visual communication design with oral arguments and/or the written word. • The oral presentation assignment is worth 25% of the course grade. • It is possible for one student to get an exceptional grade on that assignment, and another to fail, with both students receiving a “C” for the final course grade. • The OVERALL course grade does not measure: • students’ demonstrated oral presentation skills • the capacity of the curriculum to achieve PO2

  40. Why aren’t course grades adequate indicators for program assessment? • Course letter grades are insufficient for program assessment because: • Grades reflect the evaluation practices, policies, and criteria of individual instructors. • Faculty teaching the same course may teach different material. • Faculty teaching the same course may emphasize different course outcomes.

  41. Grades reflect the evaluation practices, policies, and criteria of individual instructors Instructor 2 Instructor 1 Difference of 70 points between two sections of the same course!

  42. Why aren’t course grades adequate indicators for program assessment? • Course letter grades are insufficient for program assessment because: • Letter grades are nominal values reflecting a summary of a student’s performance. • Grades are not an exact measurement of what students have and have not learned. • Grades do not measure achievement Program outcomes, learning of specific skills or the capacity of the curriculum to achieve PO2 • Grades reflect the evaluation practices, policies, and criteria of individual instructors. • Faculty teaching the same course may teach different material. • Faculty teaching the same course may emphasize different course outcomes.

  43. What CAN Be Used for Assessment? • Specific grades that reflect a specific program outcome • An exam • Assignment • Oral presentation • Then the corresponding scores can be used to represent students' learning achievement on the program outcome, which can serve as data for program assessment.

  44. Conditions for Using Course Products as Assessment Measures: • All sections of the required course must produce the course product (e.g. an oral presentation) • Faculty must agree on product structure (e.g. directions, requirements) • Faculty must agree on how the products should be evaluated: • Minimum performance standards • Rubric Average Program Score: 78/100

  45. Final Message Course products (cases, papers, presentations, exercises) may be used as program assessment measures, but course grades, by themselves, cannot.

  46. Activity 4: Conduct a Preliminary Audit of Assessment Methods Utilized in Your Program • Using the “Program Assessment Inventory” (also available on the CET&L Website), read through the assessment measures and their associated descriptions. Check off any of the measures currently used in your academic program. University of Cincinnati, Center for the Enhancement of Teaching & Learning

  47. Activity 5: Complete the Assessment Measures Alignment Matrix University of Cincinnati, Center for the Enhancement of Teaching & Learning

  48. Example Assessment Measures Alignment Matrix University of Cincinnati, Center for the Enhancement of Teaching & Learning

More Related