Assessing Undergraduate Program Curriculum Renewal: Geography & WGSS

1. Assessing Undergraduate Program Curriculum Renewal: Geography & WGSS Exploring the Redesign & Assessment Process in Geography & Women’s, Gender and Sexuality Studies

2. Academic Program Coordinator for Women’s, Gender and Sexuality Studies (WGSS) Stotlar.1@osu.edu Professor & Interim Chair for Geography Munroe.9@osu.edu Hi, we’re Jackie Stotlar &Darla Munroe

3. WGSS had a curriculum and assessment problem. Our assessment plan couldn’t help us pinpoint issues with the curriculum.

4. Geography BA/BS degrees • Geog BA • Urban-regional-global Studies • Environment and Society • Air Transport (social science option) • Geog BS • GIS / Spatial Analysis • Physical Geog

5. Geography Goals

6. Development Process

7. Development Process

8. Geography Overarching Themes • Global, spatial, geographical, human-environment • Systems • Process and pattern • Interconnection • Justice

9. New Geog goals, all programs, AY 19-20 • Goal A: Human, Environmental, and Spatial Systems • The successful student will be able to use holistic, interdisciplinary approaches to understand the patterns and processes in complex social, operational and environmental systems. • Goal B: Knowledge Generation and Advancement • The successful student will be able to understand historical context and the ongoing development of research practices in their field. • Goal C: Communication and Engagement • The successful student will be able to share and receive knowledge by engaging with diverse audiences, participants, and stakeholders. • Goal D: Critical Thinking • The successful student is able to interpret current and developing problems by examining their contexts. Based on the critique of problems, the successful student can then generate critical questions about the world around them, and is open to continuous reevaluation while intellectually engaging with different points of view and lived experiences. • Goal E: Data and Methods • The successful student will be able to apply analytical, computational, and qualitative methods, as appropriate, to acquire and transform data and evidence into actionable knowledge supporting ethical scholarship and decision-making. • Goal F: Professional Development • The successful student will be prepared to seek a personally inspiring and sustaining career path, with passion, confidence, and creativity.

10. 6 Program Goals of WGSS Goal 1: The successful student will be able to question common-sense, dominant assumptions about what seems “natural,” “timeless,” “universal,” “human,” and “normal,” by critically speaking, thinking, writing, and reading. Goal 2: The successful student will be able to work and learn compassionately and collaboratively. Goal 3: The successful student will be able to understand “feminisms” as interdisciplinary, creative, theoretical and social movements. Goal 4: The successful student will be able to understand and critically engage categories of social difference (such as but not limited to gender, race, class, language, ethnicity, sexuality, nation, empire, geography, and (dis)ability) as intersectional, always shifting, and shaped by hierarchies of power. Goal 5: The successful student will be able to understand the interconnections between the local and the global. Goal 6: The successful student will be able to recognize, analyze and critique their position and identity in society, thereby understanding the potential to imagine themselves and act creatively as feminist agents of social change.

11. Outcomes for WGSS Program Goal 1 Goal 1: The successful student will be able to question common-sense, dominant assumptions about what seems “natural,” “timeless,” “universal,” “human,” and “normal,” by critically speaking, thinking, writing, and reading. Goal 2: The successful student will be able to work and learn compassionately and collaboratively. Outcome A: Collaborate with people across difference. Outcome B: Engage with texts in an open and critical manner. Outcome C: Interact productively and openly across difference. Outcome A: Interrogate a variety of dominant narratives especially as relating to sex, gender, sexuality, disability, race, ethnicity, nation, class, etc. Outcome B: Evaluate counter-narratives that challenge dominant assumptions. Outcome C: Analyze texts. Outcome D: Articulate clear and cohesive thoughts through writing. Etc…

12. WGSS Program Goal 1, Outcome A • Goal 1: • The successful student will be able to question common-sense, dominant assumptions about what seems “natural,” “timeless,” “universal,” “human,” and “normal,” by critically speaking, thinking, writing, and reading. • Outcome A: • The successful student will be able to interrogate a variety of dominant narratives especially as relating to sex, gender, sexuality, disability, race, ethnicity, nation, class, etc. Assessment usually comes in here.

13. Proficiencies for Outcome A of WGSS Program Goal 1 Goal 1: The successful student will be able to question common-sense, dominant assumptions about what seems “natural,” “timeless,” “universal,” “human,” and “normal,” by critically speaking, thinking, writing, and reading. • Outcome A: • Interrogate a variety of dominant narratives especially as relating to sex, gender, sexuality, disability, race, ethnicity, nation, class, etc. • Basic Proficiency 1: Identify variances within and across cultures and contexts as they have developed across time and spaces. • Basic Proficiency 2: Recognize cultural assumptions and common knowledge as socially constructed in historical, cultural, political, scientific, religious, aesthetic, contexts. • Intermediate Proficiency 1: Detect dominant narratives. • Intermediate Proficiency 2: Analyze the role of power in upholding core cultural assumptions. • Advanced Proficiency 1: Map dominant narratives to their particular historical, cultural, political, scientific, religious, aesthetic origins. • Advanced Proficiency 2: Evaluate the connections between core cultural assumptions and various forms of injustice. • Specialized Proficiency: Envision a more equitable and just society.

14. WGSS Program Goal 1, Outcome A, Basic Proficiency 1 • Goal 1: • The successful student will be able to question common-sense, dominant assumptions about what seems “natural,” “timeless,” “universal,” “human,” and “normal,” by critically speaking, thinking, writing, and reading. • Outcome A: • The successful student will be able to interrogate a variety of dominant narratives especially as relating to sex, gender, sexuality, disability, race, ethnicity, nation, class, etc. • Basic Proficiency 1: • The successful student will be able to identify variances within and across cultures and contexts as they have developed across time and spaces. Now assessment starts here! This will be linked to various course assignments.

15. Curriculum Mapping • Core courses were mapped first. • Faculty are mapping existing courses as they teach them. • Newly proposed courses are mapped from start.

16. Building the Assessment Plan Steps Used to Build New Assessment Plan: 1) Determined how to break up the goals across the cycle (which goals in which year) a. Goals 1, 2, 4 in year one to break up goals 4 and 5 (WoC and GTF goals) 2) Examine locations of proficiencies across assessed courses (core and WoC/GTF) in order to determine allocation of proficiencies per course a. Maximize range of level assessed (B, I, A for each outcome) b. Assign to appropriate course based on workload distribution, centrality to the course, and importance of proficiency in assessing overall outcome

17. Linking Proficiencies to Program Assessment • 1.a. Interrogate a variety of dominant narratives especially as relating to sex, gender, sexuality, disability, race, ethnicity, nation, class, etc. • (B-1) Identify variances within and across cultures and contexts as they have developed across time and spaces. • (B-2) Recognize cultural assumptions and common knowledge as socially constructed in historical, cultural, political, scientific, religious, aesthetic, contexts. • (I-1) Detect dominant narratives. • (I-2) Analyze the role of power in upholding core cultural assumptions. • (A-1) Map dominant narratives to their particular historical, cultural, political, scientific, religious, aesthetic origins. • (A-2) Evaluate the connections between core cultural assumptions and various forms of injustice. • (S) Envision a more equitable and just society. Which courses contain which proficiencies? 1110, 2550 1110, 2550, 3575 1110, 2550, 3575 1110, 2550, 4575 2550, 4575 3575, 4575 Note: S proficiencies not assessed

18. Linking Proficiencies to Program Assessment Select specific proficiencies and courses to minimize labor • 1.a. Interrogate a variety of dominant narratives especially as relating to sex, gender, sexuality, disability, race, ethnicity, nation, class, etc. • (B-1) Identify variances within and across cultures and contexts as they have developed across time and spaces. • (B-2) Recognize cultural assumptions and common knowledge as socially constructed in historical, cultural, political, scientific, religious, aesthetic, contexts. • (I-1) Detect dominant narratives. • (I-2) Analyze the role of power in upholding core cultural assumptions. • (A-1) Map dominant narratives to their particular historical, cultural, political, scientific, religious, aesthetic origins. • (A-2) Evaluate the connections between core cultural assumptions and various forms of injustice. • (S) Envision a more equitable and just society. 1110, 2550 1110, 2550, 3575 1110, 2550, 3575 1110, 2550, 4575 2550, 4575 3575, 4575 Note: S proficiencies not assessed

19. Linking Proficiencies to Program Assessment • 1.a. Interrogate a variety of dominant narratives especially as relating to sex, gender, sexuality, disability, race, ethnicity, nation, class, etc. • (B-2) Recognize cultural assumptions and common knowledge as socially constructed in historical, cultural, political, scientific, religious, aesthetic, contexts. • Assessed during year one in 1110 • (I-1) Detect dominant narratives. • Assessed during year one 3575 • (A-2) Evaluate the connections between core cultural assumptions and various forms of injustice. • Assessed during year one in 4575 • Different proficiencies or additional proficiencies can be selected cycle to cycle.

20. Proposed WGSS Assessment Plan

21. Creating Metrics for Assessment Plan Benchmarking Use Guide A rubric score of 4 out of 5 or 80% is used as the benchmark for each proficiency. Target range is the goal percentage of assessed students who meet or exceed a rubric score of 4 on a proficiency. If the target range is at or exceeds the percentage listed, we are confident students are adequately demonstrating the proficiency. Flag range is the percentage at which we become concerned that students are not adequately demonstrating proficiency. Example: we expect that 75% of students in a 1000-level class will earn a 4 our of 5 on any basic proficiency covered in the course.

22. Testing Assessment Plan: WGSST 1110 • To test plan and record assessment data for AY18-19, decided to test ALL proficiencies (55 in total) in AU18 offering of WGSST 1110 – Gender, Sex & Power. • Turned out to be less work than anticipated. All proficiencies assessed across 6 assignments: • Two response essays, two quizzes, midterm, and final paper. • Student responses that demonstrated upper tier, middle tier, and lower tier were collected to aid in the creation of rubrics moving forward.

23. WGSST 1110 Assessment Results Goal 1:The successful student will be able to question common-sense, dominant assumptions about what seems “natural,” “timeless,” “universal,” “human,” and “normal,” by critically speaking, thinking, writing, and reading. 1.a. Interrogate a variety of dominant narratives especially as relating to sex, gender, sexuality, disability, race, ethnicity, nation, class, etc. (B-1) Identify variances within and across cultures and contexts as they have developed across time and spaces. (80%) (B-2) Recognize cultural assumptions and common knowledge as socially constructed in historical, cultural, political, scientific, religious, aesthetic, contexts. (75%) (I-1) Detect dominant narratives. (70%) (I-2) Analyze the role of power in upholding core cultural assumptions. (60%) 1.b. Evaluate counter-narratives that challenge dominant assumptions. (B) Recognize and describe counter-narratives.(75%) (I-1) Explain the production of counter-narratives. (65%) 1.c. Analyze texts. (B-1) Recognize that different schools of thought inform reading practices. (80%) (B-2) Analyze texts using a feminist lens. (80%) (A-3) Use texts as evidence for research ideas. (85%) 1.d. Articulate clear and cohesive thoughts through writing. (B) Demonstrate the components of clear writing. (75%)

24. Discussion of Assessment Results • Discussion of results in WGSS Undergraduate Studies Committee already demonstrated the power of the new process. • Concern expressed around student writing abilities. • Larger number of students speaking English as a second language in 1110. • Mirrored patterns of less polished writing in other faculty taught courses. • Noted that the proficiencies around goal 4 were much harder for students to grasp than anticipated. • Recommendations • 1) Encourage faculty lead courses to have a writing workshop day. • 2) Encourage all instructors to grant extra credit/make mandatory the use of the writing center. • 3) Track goal 4 proficiencies in WGSST 1110 during AY2021 offerings to observe trends; re-write if necessary.

25. Final Summary • Backwards design builds assessment into the curriculum. • Built-in assessment allows for rapid closure of the feedback loop. • Even large scale assessment is not as onerous as it seems.

26. Questions? Comments? Provocative Anecdotes? Please share! Jackie Stotlar, Stotlar.1@osu.edu Darla Munroe, Munroe.9@osu.edu Thank you!