Writing and Using Assessment Plans to Enhance Teaching, Learning, & Curriculum

1. Writing and Using Assessment Plans to Enhance Teaching, Learning, & Curriculum November 10, 2006

2. Today’s Objectives • Understand some reasons why we need to assess student learning • Identify and discuss each component of an assessment plan • Brainstorm about some ways to make assessment useful and meaningful • Brainstorm about appropriate methods to use when assessing student learning • Introduce ways that the assessment process can impact teaching, learning, curriculum, and other things important to us in the university community

3. External Pressure to Assess • Federal government pressures to assess—NCLB Act, Spellings Commission Reports, etc. • State pressures will not dissipate, given budget constraints • Regional accreditors (for us, the Middle States Association) require it • Local constituencies want it—they want to know how well universities and colleges are “adding value” to a student’s education • The public increasingly expects it • Even grant funders want it—assessment systems submitted with grant applications, assessment results included with grant reports, etc.

4. Example of Regional Accreditor’s Review: Middle States Association Suggestion for Binghamton University

5. Despite External Pressures, There Are Real Advantages to Program Assessment • Enables programs to answer external requests for information • Empowers faculty, not bureaucracies to make decisions about curriculum, instruction, and learning • Enhances grant application process • Empowers faculty to reflect on student teaching and learning in a way that is non-threatening • Enhances faculty’s ability to publish • Improves student learning

6. Why An Assessment Plan? • Facilitates periodic, not episodic assessment of student learning and program outcomes • Serves as template for conversations about student learning • Leads to improvement in student learning • Enables faculty to play central role in managing student learning • Facilitates information gathering for program review process • Makes it easier to do something about what assessments have to say

7. Whose Needs Might We Consider When We Develop Assessment Plans?

8. Speaking Different Languages: Assessment Communities In Practice University Community • Federal Guidelines • State Guidelines • North-Central Association Guidelines • Public Expectations and Norms Regulatory Community • Mission Statement • General Education • Objectives • Constituent Expectations and Norms • Faculty Expectations • Dispositional Expectations • Professional Standards • Advisory Board • Expectations • Professional Organization Norms and Expectations • Employer Expectations Program Community Assessment Plan

9. Reflect on some of the different audiences that might be interested in or that might require assessment information about student learning. What are some of those audiences? University Community: Example: Faculty Regulatory Community: Example: Federal Government Program (or Discipline) Community: Example: National Council on Accrediting Teacher Education (program accreditor)

10. Faculty Involvement Is Crucial • The focus of assessment is not measurement, but how assessment information is used by faculty • Only faculty are qualified to make decisions about how to use assessment information for enhancing teaching, learning, curriculum, etc. • This is an opportunity, not an obstacle

11. Defined Student Learning Objectives Evidence-Based Program Enhancements—(Curriculum, Learning, Teaching, etc.) Faculty Meaningful Measurement (Qualitative, Quantitative, Triangulated) Focused Reflection/Discussion Assessment As A Faculty-Based Process

12. Components of An Assessment Plan • Student Learning Objectives • Assessments • Plan for Faculty Discussion About Assessments and How they Lead to Recommendations to Further Enhance Student Learning Objectives • Plan to Implement Recommendations in Respect to Curriculum, Instruction, etc.

13. Identifying Student Learning Objectives

14. Defining Learning Objectives • Knowledge—what do we want students to know when they graduate (i.e., content knowledge, etc.) • Skills—ability to perform specific tasks, think in certain ways, etc.; what should a graduating student be able to do? • Competencies—ability to perform specific tasks “in real time,” or “authentically” [knowledge + skills  competencies]; also, what are some values, attitudes, behaviors we feel are important for graduates to have?

15. Listing Student Learning Outcomes By Knowledge, Skill, and Competency • Think about some of the knowledge, skills, and competencies, you would like a student from your program to graduate with • List them • Discuss with a partner at your table.

17. Assigning Relevant Assessments

18. Selecting Appropriate and Meaningful Assessments • At least one should be a “direct assessment” of student learning, meaning that they should involve observations of actual student performance • Indirect assessments —those that include student opinions about a programs’ ability to deliver on the student learning objectives, etc. can also be very helpful • Using a combination of these—two or three—might provide meaningful information; “triangulation” is increasingly being required by evaluators • It is rarely a surprise to find out that departments and programs initially feel that they never directly assess student learning, only to find out after discussion that they have been doing so, only informally

19. “Not everything that can be counted counts, and not everything that counts can be counted.” -Albert Einstein

20. “Data is the plural of anecdote” -Ronald Coase

21. Helpful Tips • The process of selecting appropriate assessments is usually dependent on a number of factors —experience in process, resource limitations, etc. • While the preference is to move toward a comprehensive assessment system, reality suggests that the development of an assessment system is very much developmental; it takes time to develop such a system. • Although the prior point is important to keep in mind, most program and regional accreditation organizations are beginning to expect 3-5 years of periodically collected assessment information • Therefore, begin incrementally. Start with assessments that create a balance between need to conserve resources and need to maximize the meaning gleaned from the process • The most important question to ask at this point: “How will this information provide faculty with legitimate information that will affect learning, teaching, and curriculum?” • A “shameless plug:” Ask the Assistant Provost for Curriculum, Instruction, & Assessment for assistance!

22. Outcomes Assessments from the Perspective of Different Communities Assessment Plan University Community Examples: State Licensure Exams (Passage Rates) General Education Syllabi Reviews Regulatory Community Examples: Student Satisfaction Surveys Portfolio Assessments Graduate Record Exam (GRE) Math Scores Focus Group Interviews Unstructured Student Interviews Examples: Capstone Course Case Study Reviews Advisory Board Input GRE Subject Exam Scores Disciplinary Standards Community

23. Encouraging Faculty Discussion

24. Defining When Faculty Will Discuss Information • Faculty (or an assessment committee) should be given information in advance to consider • Focus of meeting is to discuss what assessments say about student learning objectives and any recommendations that might stem from such discussion; it should not be a “rubber stamping” of findings or solely an opportunity to complain—instead, the focus should be upon recommendations • It is important to focus on communicating results to faculty, and tracking ways that information is used

25. Tip on Producing An Effective Meeting: Try to Put Assessment Results on One Page, If Possible

27. Implementing Faculty Recommendations, and Affecting Learning and Teaching

28. Ideas for Using Assessment • Use information to pursue department or program-level initiatives or projects on teaching • Incorporate assessment information into curriculum discussions and recommendations • Have periodic discussions or reflective discussions on teaching using assessment information • Use for grant applications • Use in budget requests • Use in fundraising letters or alumni functions • Use to gather further information about student learning from instructors on a periodic basis

29. Ways to Track Effect of Faculty Discussions/Recommendations • Encourage inclusion in annual report (section under teaching effectiveness) • Encourage submissions of faculty narratives —how were faculty discussions and recommendations used in course design, selection of courses, etc.? • Track how recommendations were enacted —curriculum process, department/program initiatives, course sequencing, equipment requests, etc. • Not so important to make an academic study out of this, or to make this too huge of a process, but documentation is helpful when writing annual reports on assessment, curriculum, & instruction

30. Future Directions and Vision • Will ask for report on four questions at end of each academic year • Focus is not upon “reporting for reporting’s sake,” but instead upon how we can assist • Focus will be upon serving faculty—in helping them in accomplishing the last two of the four core questions • Focus primarily upon impact—how has process impacted learning, teaching, and curriculum processes

31. Conclusion • Four foci: • Objectives • Assessments • Faculty Usage • Impact

32. All assessment is a perpetual work in progress. -- Linda Suske, Unknown , May 3, 2005

33. In assessment, "the perfect is the enemy of the good." Let's keep striving for the good. -- Tom Angelo, Unknown , Unknown

34. Questions? Comments?