1 / 65

Assessing Gen-Ed and Institutional Learning Goals: Innovative Efforts

Assessing Gen-Ed and Institutional Learning Goals: Innovative Efforts. Barbara E. Walvoord , Ph.D. Professor Emerita , University of Notre Dame Consultant in Assessment, Teaching and Learning, and Writing Across the Curriculum 45 Huckleberry Lane, Easthampton, MA Phone: 413-203-5086

sitara
Download Presentation

Assessing Gen-Ed and Institutional Learning Goals: Innovative Efforts

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Assessing Gen-Ed and Institutional Learning Goals: Innovative Efforts Barbara E. Walvoord, Ph.D. Professor Emerita, University of Notre Dame Consultant in Assessment, Teaching and Learning, and Writing Across the Curriculum 45 Huckleberry Lane, Easthampton, MA Phone: 413-203-5086 walvoord@nd.edu

  2. Outline: Six Areas of Challenge and Innovation for Gen-Ed Assessment • Definitions; Basic, No-Frills Plan • 1. Teaching and Faculty Engagement • 2. High-Impact Practices • 3. Systems for Information and Action • 4. Using Information: The “Stomach” • 5. Rubrics and Evaluation of Student Work • 6. Keeping it Simple

  3. Definitions: What Is Gen-Ed? • Gen Ed GOALS: Everyone works on them. • May be a subset or a more specific version of institution-wide goals • Gen Ed CURRICULUM: Every course that students can use to fulfill their general-education requirements • Community colleges: Your Associates Transfer degree is someone else’s gen-ed

  4. The Basic, No-Frills System of Gen-Ed Assessment 1. Goals 2. Information • Direct (sample of students’ work and/or standardized test) • Indirect (student survey, student retention/ success, etc.) 3. Action • Forums for discussion • A system for information to flow into decisions

  5. Outline: Six Areas of Challenge and Innovation for Gen-Ed Assessment • Definitions; Basic, No-Frills Plan • 1. Teaching and Faculty Engagement • 2. High-Impact Practices • 3. Systems for Information and Action • 4. Using Information: The “Stomach” • 5. Rubrics and Evaluation of Student Work • 6. Keeping it Simple

  6. Bottom Line: TEACHING! • In the presence of a reasonable curriculum, what makes a difference for student learning is TEACHING, broadly defined. That is, how the instructor manages student-instructor and student-student interaction and arranges the educative experiences--in class, out of class, online.

  7. Corollary for Bottom Line • If you do not engage the faculty, you will not change teaching. • Thus, your gen-ed assessment system should be planned at every step for maximum faculty engagement and impact on teaching. • Innovation: Replace “faculty on board” with “collaborative steering.”

  8. Gen-Ed Revision -- TEACHING • Many institutions spend too much time changing titles, number, and stated learning goals of required courses. • Innovation: Gen-ed reform focusing on faculty development, not (or not only) courses. • Innovation: Gen-ed courses are required/ encouraged to use research-based pedagogical approaches: e.g. active learning. • Innovation: Gen-ed reform institutes research-based “high-impact practices.”

  9. Questions for Discussion • How do your institutions try to affect teaching and faculty engagement? • What was the focus of your most recent gen-ed reform? Did it affect teaching? • What are the implications of requiring certain types of pedagogy in gen-ed courses: e.g. active learning? • Where do “writing-intensive” and similar courses fit?

  10. Outline: Six Areas of Challenge and Innovation for Gen-Ed Assessment • Definitions; Basic, No-Frills Plan • 1. Teaching and Faculty Engagement • 2. High-Impact Practices • 3. Systems for Information and Action • 4. Using Information: The “Stomach” • 5. Rubrics and Evaluation of Student Work • 6. Keeping it Simple

  11. Research-Based Seven Principles for Good Practice in Undergraduate Education Good practice … • Encourages contact between students and faculty • Develops reciprocity and cooperation among students • Encourages active learning • Gives prompt feedback

  12. Research-Based Seven Principles for Good Practice in Undergraduate Education Good practice … 5. Emphasizes time on task 6. Communicates high expectations 7. Respects diverse talents and ways of learning Chickering and Gamson, 1987, widely available online.

  13. Innovation: Research-Based High-Impact Educational Practices • First-Year Seminars/Experiences • Common Intellectual Experiences • Service Learning/Community-Based Learning • Learning Communities • Writing-Intensive Courses

  14. Research-Based High-Impact Educational Practices, cont. • Collaborative Assignments/Projects • Undergraduate Research • Diversity/Global Learning • Internships • Capstone Courses and Projects Kuh, 2008. See AACU.org. Also http://www.neasc.org/downloads/aacu_high_impact_2008_final.pdf

  15. High Impact Practices for Community Colleges • Academic goal setting and planning • Orientation • Accelerated for fast-track developmental education • First-year experience • Student success course • Learning community

  16. High Impact Practices for Community Colleges, cont. • Experiential learning beyond the classroom • Tutoring • Supplemental instruction • Assessment and Placement • Registration before classes begin • Class attendance • Alert and intervention www.ccsse.org/docs/Matter_of_Degrees_2.pdf.

  17. Seven Principles, High-Impact Practices, and YOUR Assessment • Seven Principles and High-Impact Practices can guide you. • They are powerful because they change factors that affect learning: engagement, interactions among instructor and students, and the arrangement of educative experiences.

  18. BUT • High-impact practices are not assessment of learning. • AND, to get the benefits, you have to do them well. • Use published research about how to make practices most effective. • Use assessment in your own setting to inform your practices.

  19. Questions for Discussion • How do your institutions implement and assess high-impact practices? • How do you use assessment in connection with high-impact practices?

  20. Outline: Six Areas of Challenge and Innovation for Gen-Ed Assessment • Definitions; Basic, No-Frills Plan • 1. Teaching and Faculty Engagement • 2. High-Impact Practices • 3. Systems for Information and Action • 4. Using Information: The “Stomach” • 5. Rubrics and Evaluation of Student Work • 6. Keeping it Simple

  21. SYSTEMS for Information and Action • You Need a SYSTEM for Gen-Ed, not just a set of isolated actions. • NEXT: Diagram shows asystem by which assessment information flows through the institution to inform action at every level. • Diagram is in your handout.

  22. Innovation: Institutional System for Gen-Ed Assessment Administration, faculty committees Start reading at the bottom boxes, which show common types of assessment information. STOMACH: Assessment Committee, IR, etc. Aggregate/analyze data; recommend 1 Dept, group 6 2 Scorers 5 4 3 Student affairs; academic support Instructor Stand. tests IR: surveys Student work

  23. Institutional System for Gen-Ed Assessment Administration, faculty committees Black arrows with numbers show pathways for assessment information. STOMACH: Assessment Committee, IR, etc. Aggregate/analyze data; recommend 1 Dept, group 6 2 Scorers 5 4 3 Student affairs; academic support Instructor Stand. tests IR: surveys Student work

  24. Institutional System for Gen-Ed Assessment Administration, faculty committees Fat green arrows show feedback loops where resources and policies flow back to influence student learning STOMACH: Assessment Committee, IR, etc. Aggregate/analyze data; recommend 1 Dept, group 6 2 Scorers 5 4 3 Student affairs; academic support Instructor Stand. tests IR: surveys Student work

  25. Institutional System for Gen-Ed Assessment Administration, faculty committees You do not need all possible sources of information. Keep it simple. Gather only what you can use. STOMACH: Assessment Committee, IR, etc. Aggregate/analyze data; recommend 1 Dept, group 6 2 Scorers 5 4 3 Student affairs; academic support Instructor Stand. tests IR: surveys Student work

  26. Institutional System for Gen-Ed Assessment Administration, faculty committees Make your own version of this diagram, with your own offices and details. STOMACH: Assessment Committee, IR, etc. Aggregate/analyze data; recommend 1 Dept, group 6 2 Scorers 5 4 3 Student affairs; academic support Instructor Stand. tests IR: surveys Student work

  27. Questions for Discussion • What is your institution’s system?

  28. Outline: Six Areas of Challenge and Innovation for Gen-Ed Assessment • Definitions; Basic, No-Frills Plan • 1. Teaching and Faculty Engagement • 2. High-Impact Practices • 3. Systems for Information and Action • 4. Using Information: The “Stomach” • 5. Rubrics and Evaluation of Student Work • 6. Keeping it Simple

  29. Innovation: A Better “Stomach” Administration, faculty committees STOMACH: Assessment Comm., Deans, IR, etc. Aggregate/analyze data; recommend 1 Dept, group 6 2 Scorers 5 4 3 Student affairs; academic support Instructor Stand. tests IR: surveys Student work

  30. Models for the “Stomach” Disbursement Model: “Stomach” members work to ensure use of data at every level. Requirement Model: Provost and others require assessment data for budget and policy Retreat Model: Retreat (leaders or entire camps) to discuss 5-8-page summary of relevant data, how to use it in their own areas, and what the institution should work on.

  31. Questions for Discussion • What people/offices make up your “stomach”? What functions do they perform? • Which model(s) do you use? • How well is your “stomach” working to ensure that information about learning is aggregated, analyzed, distributed, and used for action?

  32. Outline: Six Areas of Challenge and Innovation for Gen-Ed Assessment • Definitions; Basic, No-Frills Plan • 1. Teaching and Faculty Engagement • 2. High-Impact Practices • 3. Systems for Information and Action • 4. Using Information: The “Stomach” • 5. Rubrics and Evaluation of Student Work • 6. Keeping it Simple

  33. Rubrics and Evaluation of Student Work: Paths 1-4 Administration, faculty committees STOMACH: Assessment Committee, IR, etc. Aggregate/analyze data; recommend 1 Dept, group 6 2 Scorers 5 4 3 Student affairs; academic support Instructor Stand. tests IR: surveys Student work

  34. Advantages and Problems • The further to the left, the more faculty involvement. • The further to the right, the more inter-rater reliability for institution-wide quantitative data. To Stomach 1 Dept, group 2 Scorers 4 3 Instructor Stand. tests Student work

  35. Innovations: Try to solve the problems, by… • 1 & 2: • Providing institution-wide information, externally trusted • 3 & 4: • Ensuring validity, faculty involvement To Stomach 1 Dept, group 2 Scorers 4 3 Instructor Stand. tests Student work

  36. Path 1: Instructor-Group Analysis: PROS and CONS • Can involve many instructors. • Instructors piggy-back onto grading. • No one else reads the student work. • Rich discussion, collaboration. • Engages faculty with their OWN work • Modest software requirements because reports, not scores, are aggregated. To stomach 1 Dept, group Instructor Student work

  37. Path 1: Instructor-Group Analysis: Pros and Cons, cont. • Challenges if common rubric: • Validity: what is being measured? • Inter-rater reliability • Challenges if own rubrics: • Comparability at institutional level To stomach 1 Dept, group Instructor Student work

  38. Innovations for Path 1: Instructor-Group Analysis • Innovation: Own rubrics, but within categories (e.g. critical thinking). Scores are aggregated for categories (Prince George’s CC). • Innovation: Reports, not scores, are aggregated. Report: what we found, what we did, what we recommend institution should work on. To stomach 1 Dept, group Instructor Student work

  39. Innovations for Path 1, cont. • Innovation: Sample of depts/groups document improvement in learning. Sample results are extrapolated to institution. • Innovation: Triangulate with survey or standardized test. To stomach 1 Dept, group Instructor Student work

  40. Path 1: Instructor - Group: Examples • Raymond Walters College (2 year) of the University of Cincinnati. Each program/department holds an end-of-year meeting in which faculty each present one assignment that assesses “critical thinking,” a rubric, scores, and instructor’s action. Departments/programs take action, and also report in a common format to the Academic Assessment Committee, which makes recommendations to the Chief Academic Officer about common needs and institution-wide actions. All record-keeping is done in Word. Walvoord, Bardes, and Denton in Banta, ed, 2007.

  41. Path 1: Instructor - Group: Examples • “Medium-sized public university.” Selected faculty report to gen-ed “area committees,” which aggregate reports and recommend action to Gen Ed Council, which informs departments about their gen-ed courses. Gerretson, H. & Golson, E. (2005). Synopsis of the use of course-embedded assessment in a medium sized public university’s general education program. Journal of General Education, 54(2), 139-149.

  42. Path 1: Instructor - Group: Examples • Juniata College. Center for Teaching holds numerous faculty workshops and discussion groups where faculty conduct and share assessment and improvement of student learning. Strong influence of the Scholarship of Teaching and Learning (SOTL). www.learningoutcomesassessment.org/Juni ataCaseStudy.pdf.

  43. Path 1: Instructor-Group: Examples, cont. • La Guardia Community College. Extensive workshops and faculty seminars support a strong e-portfolio system. www.learningoutcomesassessment.org/documents/LaGuardiaCC.pdf.

  44. Path 1: Instructor-Group: Examples, cont. • Washington State University. Extensive faculty workshops involve faculty in developing and using/adapting common rubrics for critical thinking. Some faculty conduct classroom research to show improvement in student learning when faculty use the rubrics and teaching strategies developed in the workshops. These studies can be aggregated. Kelly-Riley in Banta, ed., 2007.

  45. Path 2: Instructor Reports Directly to Stomach Administration, faculty committees STOMACH: Assessment Committee, IR, etc. Aggregate/analyze data; recommend 1 Dept, group 6 2 Scorers 5 4 3 Student affairs; academic support Instructor Stand. tests IR: surveys Student work

  46. Path 2: Instructor Reports: OPTIONS • Instructor uses own OR common assignment. • Instructor uses own OR common rubric. • Instructors submits rubric scores • AND/OR report: What I found, what I am doing, what the institution should work on. • Student responses may OR may not be included. To Stomach 2 Instructor Student work

  47. Path 2: Instructor Reports: PROS and CONS • Bypasses department or group. • Saves instructor meeting time. • Leaves instructor isolated, without community discussion. • Inter-rater reliability problems. • Requires software to aggregate individual instructor scores/reports. • May not include assignment or student response. Thus scores interpreted in isolation. To Stomach 2 Instructor Student work

  48. Path 2: Instructor Reports: Examples • Prince George’s Community College. Each instructor uses a course-specific rubric to enter scores into a database. Each cell of the rubric is assigned a point value, so the same rubric can be used to calculate the student’s grade. In the software program, each row of the rubric is connected to a course outcome, which is connected to program and gen-ed outcomes. Thus rubric scores can be aggregated to provide scores for each outcome. http://learningoutcomesassessment.org/Documents/Occasional%20Paper%20FINAL.pdf

  49. Path 2: Instructor Reports: Examples • North Carolina State University. Gen-ed instructors report to the Assessment Office how they have assessed student work that addresses common gen-ed goals, and how they have used information for changes. Reports can be aggregated to determine, for example, what goals faculty find most difficult for students, and what faculty are working on. Assessment Office also conducts a few focused studies, e.g. common math exam questions and common rubric scores for first-year writing. DuPont in Bresciani, ed., 2007

  50. Path 3: Institution-Wide Sample/Portfolios Scored Administration, faculty committees STOMACH: Assessment Committee, IR, etc. Aggregate/analyze data; recommend 1 Dept, group 6 2 Scorers 5 4 3 Student affairs; academic support Instructor Stand. tests IR: surveys Student work

More Related