Building a Program Assessment Plan

1. Building a Program Assessment Plan Jack McGourty Associate Dean Fu Foundation School of Engineering and Applied Science Columbia University

2. Workshop Objectives • Discuss assessment in relation to ABET Engineering Criteria 2000 • Provide a structured process to follow to support accreditation requirements • Familiarize participants on how to define departmental/program/course objectives, outcomes, and assessment strategies • Start defining program objectives, strategies, outcomes, and assessment processes • Discuss next steps

3. What is Assessment? “An ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance.” American Association for Higher Education

4. Accreditation Processes • Assessment an integral component • Criteria designed to • promote a seamless linkage from the institution’s mission to program objectives • involve all relevant constituents in planning • foster cross-institutional collaboration • create a systemic approach to assessment • lead to continuous improvement activities

5. The Four Strategies 1. Initiate structured process leading to comprehensive assessment & continuous improvement programs 2. Facilitate faculty (& student) development 3. Develop assessment "toolbox" 4. Identify/Align enabling institutional practices

6. Assessment Process Step #1 Define Objectives, Strategies, & Outcomes Step #2 Identify Assessment Processes Step #5 Apply Results Continuous Improvement Step #3 Develop/Pilot Assessment Processes Step #4 Implement/Expand Assessment Processes

7. Actions: Facilitated work session to establish common language and expectations Structured process using pre-designed workbooks to define program plan Multiple sessions optimal Document final results Problems: Real "Buy-in" hard to achieve Momentum difficult to sustain Define Objectives, Strategies & Outcomes

8. Activity • Defining Departmental/Program Objectives • Workbook provides a process to help identify and define: • Program Objectives • Strategies and Actions • Outcomes • ABET 2000 Criterion 3. (a-k) • Assessment Methods

9. Program Objectives What do you want to accomplish by the end of this program? What will the students know and be able to do? • Question: Our students will have a series of “hands-on” design experiences throughout the curriculum so that they will be able to effectively apply design principles in a variety of project situations. Example:

10. Strategies/Actions What strategies and actions will the department and its faculty take in order to achieve this program objective? Question: Create an industrial partner program where practicing engineers work with student project teams on real industrial design problems in both our COOP program and senior capstone course. Example:

11. Learning Outcomes What student learning outcomes (critical knowledge, skills, and behaviors) will students acquire as a result of these strategies? What will they know and be able to do? • Question: Example: Students will be able to clearly articulate the design process and apply it to real industrial projects

12. ABET Criteria 2000 Which ABET Criteria (a-k) are met if these outcomes are achieved? • Question: Example: ABET Criteria: C,E,F, & K.

13. Assessment Method What are the methods to assess the stated outcome? How will assessment results be applied? Question: Competency based survey completed by industrial partners rating students on specific design skills. The results of the survey will be provided to students for developmental purposes and to faculty for continuous improvement of the curriculum. Example:

14. Activity • Each group takes 10 minutes. • Reviews objectives with strategies, outcomes, and methods. • Audience provides feedback after listening to the complete plan

15. Review Questions? • Are the objectives clear and the right level of specificity (too general or specific)? • Are the strategies realistic (Can the actions be implemented in this institution)? • Are the outcomes measurable (at least documented)? • Are ABET Criteria - student outcomes (a-k) adequately covered in all objectives? • How are assessment results to be used for continuous improvement?

16. Suggested Process (Early Steps) • Charter Assessment Committee • Establish departmental/program teams • Define program objectives, strategies, outcomes, & assessment methods (Departmental Workbook) • Assessment Committee reviews workbooks for clarification & synergy • Program objectives, strategies, and outcomes sent to faculty for review & comment

17. Suggested Process (continued) • Incorporate faculty input and publish final program-level documents • Implement program-level strategies • Identify critical curriculum/course areas • Begin faculty/course level definition process (Faculty Workbook) • Implement Course-level strategies • Continuously evaluate & update program/course objectives

18. Next Steps

19. Contact Information Jack McGourty, Ph.D. Associate Dean Columbia University jm723@columbia.edu