Dr. Jenn Berg Assistant Professor of Mathematics Dr. Christopher Cratsley

1. Liberal Arts and Sciences Outcomes Assessment: Closing the loop or spiraling in the right direction? Dr. Jenn Berg Assistant Professor of Mathematics Dr. Christopher Cratsley Interim Director of Assessment Dr. Elizabeth Gordon Assistant Professor of Geo/Physical Sciences Dr. Joe Moser Assistant Professor of English

2. Closing the loop or spiraling in the right direction? Assess Student Learning Use data to improve instruction Assess Student Learning Use data to improve instruction & assessment

3. LA&S Assessment at FSU • The Liberal Arts and Sciences (LA&S) assessment process carried out at Fitchburg State University from 2008-2010. • The actions taken by the Fitchburg State University LA&S council in 2011 to reframe the objectives, reinterpret the role of LA&S courses, and revise the assessment system. • Proposed Fitchburg State University LA&S course approval process that has emerged from the work of the LA&S council in 2011 including the rubrics utilized to assess the Fitchburg State University LA&S objectives.

4. LA&S Curriculum Objectives • Aesthetic Appreciation • Communication • Problem Solving and Synthesizing • Ethical Reasoning • Citizenship

5. The Process of LA&S Assessment • In the old days of 2008-10, we solicited student work as needed to match learning objectives. • Faculty on the LA&S committee (and whomever else we could corral) performed assessment of a small sample of student work that matched a particular objective. • Two to three faculty members scored each piece of student work on a 1-3 (insufficient, sufficient, or proficient) scale. • Final assessment data stayed with the LA&S committee.

6. Example: Communication

7. Communication results Spring 2010 – Nine student artifacts evaluated Fall 2010 – Eleven student artifacts evaluated; Honors course

8. Example: Problem solving • Work is correct • Work is organized • Work is complete • Uses formulas properly, where and when appropriate • Creates graphs, tables and/or statistics to summarize data • Gives clear, precise and relevant explanations, identifies causes and proposes solutions • Uses appropriate methodology to collect data • Integrates information from outside sources • Uses numerical data to defend argument(s) • Explains patterns or trends in observations, data, graphs and/or tables • Identifies pros and cons of argument(s), including biases and/or limitations • Analyzes outcomes from multiple perspectives • Applies content knowledge, methods and/or results to new situations

9. Problem solving results • Separation of math and science rubrics

10. Problem solving results • Separation of math and science rubrics • Many unused criteria

11. The Problems of LA&S Assessment • Our sample sizes tended to be small. • In some cases, due to a dearth of available student work and time constraints, the faculty performing the assessment did not have the assignments to which the pieces of student work were adhering. • Some of our rubrics were either too vague or too detailed and thus hard to apply. • Faculty buy-in was difficult to achieve, which undermined the assessment process, which led to further difficulty with faculty buy-in, which further undermined the process, which led to . . .

12. Questions? Assess Student Learning Use data to improve instruction & assessment

13. January 2011

14. The Vision Project Key Outcomes • COLLEGE PARTICIPATIONCollege-going rates of high school graduates • COLLEGE COMPLETIONGraduation and success rates of the students we enroll • STUDENT LEARNINGAcademic achievements by our students on campus-level and national assessments of learning • WORKFORCE ALIGNMENTAlignment of our degree programs with key areas of workforce need in the state's economy • ELIMINATION OF DISPARITIESAchievement of comparable outcomes among different ethnic/racial, economic and gender groups

15. Implementation Plan • Seek to become a LEAP State and establish a Massachusetts LEAP Team. • Develop a Resource Plan that outlines direct and indirect resources needed to achieve the goals of the Vision Project. • Engage campus provosts to provide leadership for faculty engagement and campus assessment activity including LEAP-related analysis. • Establish a network of faculty and professional staff assessment leaders to engage in collaborative activities including determining system-wide learning outcome measures within LEAP framework.

16. Assessment Afternoons • 4/6  LEAP Learning Outcomes and the Fitchburg State University LA&S Curriculum • 4/13 Assessing Art Appreciation • 4/20 Assessing Communication Skills • 4/27 Assessing Problem Solving • 5/4 Assessing Ethical Reasoning • 5/11 Assessing Citizenship • 5/23 LA&S AND YOU: Linking Liberal Arts and Sciences Assessment to what is happening in the departments and courses.

17. LEAP Essential Outcomes Knowledge of Human Cultures and the Physical and Natural World • Through study in the sciences and mathematics, social sciences, humanities, histories, languages, and the arts Intellectual and Practical Skills, including • Inquiry and analysis • Critical and creative thinking • Written and oral communication • Quantitative literacy • Information literacy • Teamwork and problem solving Personal and Social Responsibility, including • Civic knowledge and engagement—local and global • Intercultural knowledge and competence • Ethical reasoning and action • Foundations and skills for lifelong learning Integrative and Applied Learning, including • Synthesis and advanced accomplishment across general and specialized studies

18. Assessment afternoons • Who came? • What were our goals? • What actually happened?

19. LA&S Curriculum Objectives • Aesthetic Appreciation • Aesthetic Appreciation and Aesthetic Expression • Communication • Oral Communication • Written Communication • Problem Solving and Synthesizing • Problem Solving through Quantitative Literacy • Problem Solving through Inquiry and Data Analysis • Ethical Reasoning • Ethical Reasoning when Making Choices • Citizenship • Citizenship through Critical Analysis of Events

20. The Arts rubric

21. The Arts rubric revision

22. Communication rubric revisionsStudents will speak, read, write and listen to create and understand meanings using a variety of media. They will recognize how to participate in or lead groups to accomplish goals. Written communication • Controlling idea • Organization and Development • Sources and Evidence • Academic Discourse • Mechanics/Presentation Oral communication • Controlling idea • Organization and Development • Supporting Material • Oral and Physical Delivery • Mechanics/Presentation

23. Problem-Solving rubric revisions Among these strategies, students will analyze and interpret data as a means to evaluate arguments and make informed choices. • Topics Selection • Information from sources • Appropriate methodology • Creates graphs, tables, etc. • Explains patterns or trends • Numerical data to defend • Identifies pros and cons • Applies content knowledge Form problem-solving strategies and evaluate their effectiveness. • Work correct and complete • Uses formulas properly • Creates graphs, tables, etc. • Explains patterns or trends • Gives clear explanations • Applies content knowledge

26. Ethical Reasoning rubric revisionStudents will recognize the ethical issues involved in human actions and be able to formulate a set of principles and virtues which can be brought to bear in personal and public decision making. • Moral Reasoning • Statement of Position • Ethical Issue Recognition • Application of Ethical Perspectives/Concepts • Development of Rationale • Ethical Self-Awareness • Evaluation of Different Ethical Perspectives/Concepts

27. Citizenship rubric revisionStudents will articulate the relationships among local, national and global concerns, interests and needs. They will recognize possibilities and opportunities to enact positive change on an individual or group level. Explanation of Event Evidence Student’s Position Diversity of Communities and Cultures Analysis of Knowledge

28. Questions? Assess Student Learning Use data to improve instruction & assessment

29. NEEAN Summer Institute Through facilitated dialogue, participants will: 1. Explore options for institutionalizing assessment to improve student learning; 2. Learn how to improve the reporting process; 3. Consider ways to enhance faculty development and target resources to continuous improvement; 4. Become familiar with helpful tools and methods to assess and improve student learning; and 5. Discuss how to best analyze data.

30. Affecting change through governance • What are our goals: • Increase faculty awareness/involvement • Increase the variety of data we collect • Increase faculty input on the process • Make LA&S-ness primary in course design

31. Means of effecting change • Alter the course approval process to ask for: • Which objective(s) the course addresses • Description of an assignment (aligned with an LA&S rubric) whose student work could be submitted to the LA&S council • Option to indicate need for new rubric

32. Pros and cons of this idea • Pros: • Achieves many of our goals • Increase faculty awareness/involvement • Increase the variety of data we collect • Increase faculty input on the process • Make LA&S-ness primary in course design • Cons • Expect faculty resistance

33. LA&S Assessment Cover Sheets Please indicate yes or no for each of these boxes and then make a final yes/no judgment on whether scorers should assess this criterion. Faculty member or department name _________________ Course #__________Course Name _________________ Assignment name __________# students in course ______ Date ___ % of grade □< 3% □3-5% □6-10% □11-20% □> 20%

34. Final Summary Slide • Assessing student work has value even when the process is imperfect because the data allows us to improve our assessment methods. • A meaningful assessment system has to take into account what faculty are actually doing with their students in the classroom. • Assessment may be more meaningful and effective if it is integrated into the process of course design and development.

35. Questions? Assess Student Learning Use data to improve instruction & assessment

36. Thanks to: • The Fitchburg State University LA&S Council • The Davis Foundation • New England Educational Assessment Network • Dr. Ben Railton and Dr. Eric Budd past chairs of the LA&S Council • Fitchburg State University Office of Academic Affairs