Report of the Ad Hoc Working Group on Assessment of Learning Outcomes

1. Report of the Ad Hoc Working Group on Assessment of Learning Outcomes N. John Cooper Dean, School of Arts and Sciences & College of General Studies Convener, Ad Hoc Working Group on Assessment of Learning Outcomes Council of DeansNovember 9, 2006

2. Shared Values • Commitment to excellence in instruction that spans regional campuses, professional schools, and liberal arts schools • Commitment to importance of evaluating the outcomes of instructional programs • Commitment to the use of outcome evaluations in strategic planning processes

3. Move to Implementation Commitment to importance of evaluating the outcomes of instructional programs Commitment to excellence in instruction Commitment to the use of outcome evaluations in strategic planning processes

4. Leads to Creation of New Culture Culture of Assessment

5. Processes of that Culture Must Be • Consistent with University goals • Comprehensive • Documented • Meaningful to others • Sustainable

6. To Be Consistent with University Goals Each school’s and campus’ goals for student learning outcomes must be consistent with the University’s goals for all of our graduates, that they are able to: • Think critically and analytically • Gather and evaluate information effectively & appropriately • Understand and apply basic, scientific, & quantitative reasoning • Communicate clearly and effectively • Use information technology appropriate to their discipline • Exhibit mastery of their discipline • Understand and appreciate diverse cultures • Work effectively with others • Have a sense of self, responsibility to others, and connectedness to the University

7. To Be Consistent with University Goals • Assessment must be based on student learning outcomes • As articulated by Middle States in Standard 14, Assessment of Student Learning: “Assessment of student learning demonstrates that the institution’s students have knowledge, skills, and competencies consistent with institutional goals and that students at graduation have achieved appropriate higher education goals.”

8. Faculty Must Be Engaged in Development of Learning Outcomes Assessment • What attributes, skills, and knowledge do you expect graduates in your major(s) to acquire that are characteristic of the discipline? • What attributes, skills, and knowledge do you expect your graduates to acquire that are hallmarks of your program at Pitt? • What qualitative and/or quantitative evidence can you collect on an ongoing basis to show how well your graduates are meeting these goals? Questions asked of Arts and Sciences Department Chairs and Program Directors for 2006 Departmental Activity Reports:

9. Student Outcomes May Be Measured Directly • Course papers • Course assignments • Exhibits • Performances • National disciplinary exams • Performance on licensure or professional exams • Authentic assessments

10. Student Outcomes May Be Measured Indirectly • Job placements and placement rates • Student surveys • Graduate follow-up surveys • Focus groups • Exit interviews

11. Especially Rich Opportunities to Measure Student Outcomes • Capstone courses • Senior projects • Undergraduate research projects • Thesis/dissertation • Internships • Portfolios of work and reflections on that work • Embedded questions in assignments /examinations in specific courses in the curriculum

12. Approaches to Measurement of Outcomes • Program faculty review assignments from capstone courses and assess how well the students have achieved desired outcomes • Program faculty review approved dissertations and assess how well the students have achieved desired outcomes • Faculty external to the course score embedded standardized questions

13. To Be Comprehensive Assessment processes expected for: • All degree or certificate-granting programs listed in the graduate and undergraduate bulletins • At a minimum each major at each degree level should be assessed, e.g., • Associate of Science in Nursing • Bachelor of Science in Civil Engineering • Master of Arts in History • Doctor of Philosophy in English • MD, JD, MPPM, MAT, EdD, MSW • Etc. • School- and campus-level general education curricula

14. To Be Documented Need Standard Minimal Components: • An articulation of program mission/goals which reflects national disciplinary norms as well as any unique features of the University of Pittsburgh program • Identification of three to five prioritized learning outcomes • Identification of methods of assessment and determinations of when in the curriculum—and how—learning outcomes will be assessed • Standards of comparison/targets for results of assessment

15. To Be Meaningful to Ourselves and to Others • Assessment must include direct evidence • If a specific course is used, there must be periodic validation external to the course and instructor • A process of faculty and administrative review that ensures results are used for program improvement

16. To Be Sustainable • Must be respectful of faculty, administrative, and staff resources • Do not have to measure every outcome every year using every student • Must be embedded in annual planning process • Must be seen to drive change

17. Distributed Responsibility Is Critical to Sustainability • Program faculty are responsible for the development and administration of the assessment processes of individual programs in accordance with the appropriate programmatic or departmental governance structure • Department chairs are responsible for coordinating the assessment process for departmentally-based programs; deans & campus presidents are responsible for school- and campus-based programs • Schools & regional campuses are responsible for developing internal procedures for documenting program assessment • Deans & campus presidents are to report annually to the Provost on their assessment activities and relevant results as part of their planning processes

18. Processes of a Culture of Assessment Must Be • Consistent with University goals • Comprehensive • Documented • Meaningful to others • Sustainable • STARTED!

19. Proposed Time Table To ensure that we are prepared to report to our accrediting agency, Middle States, in a timely manner: • Deans & campus presidents will be asked to submit documentation of their assessment processes • for each degree program as part of their annual planning documents in March 2007 • for general education programs as part of their annual planning documents in March 2008 if not embedded in degree program • Initial assessments should be conducted and results reported • for each degree program in AY 2008 • for general education programs in AY 2009 if not embedded in degree program