Marvin W. Peterson & Thomas E. Perorazio The National Center for Postsecondary Improvement, Project 5.2 The Universi

1. Institutional Strategies for IntegratingStudent Assessment with Academic Planning and Management Marvin W. Peterson & Thomas E. Perorazio The National Center for Postsecondary Improvement, Project 5.2 The University of Michigan Ross A. Griffith, Wake Forest University American Association for Higher Education Assessment Conference Denver, CO - June 25, 2001

2. Session Purposes • Discuss Initial Findings from the Case Study Phase of NCPI Research Project on Institutional Support for Student Assessment • Provide Detailed Examples of Two Institutions That Have Integrated Assessment Into their Academic Management Processes • Outline the Lessons Learned Applicable to Assessment Practitioners

3. The National Center for Postsecondary Improvement Main Project Web Site http://www.stanford.edu/group/ncpi/ University of Michigan - Project Five http://www.umich.edu/~ncpi/

4. NCPI Project Area 5: Student Learning and Assessment 5.1 State Policies and Regional Accreditation Practices of Assessment for Student Learning 5.2 Institutional Support for Enhancing Student Assessment and Performance • 5.3 Academic Programs and Students: • Educational Reform and Innovation in Teaching, Learning, and Assessment Practices • Metadata Project: Review of National Student, Faculty, and Institutional Data Resources

5. Project 5.2’s Purpose To Examine How Postsecondary Institutions Support and Promote the Uses and Impacts of Student Assessment Information for Institutional Improvement

6. Overview of Project 5.2 • Phase One • Review & Synthesis of the Literature • Developed a Conceptual Framework of Institutional Support • Phase Two • National Survey of Institutions • Comprehensive Inventory Based on Conceptual Framework • 1394 Usable Responses -- 55.1% • Phase Three • Case Studies of Institutions with Comprehensive Student Assessment Efforts

7. Phase I:Conceptual Frameworkof Organizational and Administrative Support for Student Assessment Institutional Approach to Student Assessment Institution-wide Support for Student Assessment Uses & Impacts of Student Assessment External Influences on Student Assessment Assessment Management Practices & Policies Assessment Culture and Climate Institutional Context

8. Phase II:National InventoryResearch Questions • What Approaches to Student Assessment Have Institutions Adopted? • What Organizational and Administrative Support Patterns and Assessment Management Practices for Student Assessment Have Institutions Developed? • How Have Institutions Used Student Assessment Information and What Impacts Has It Had? • How Do Patterns of Student Assessment Approach, Organizational and Administrative Support Patterns and Practices, and Uses and Impacts of Student Assessment Vary by Institutional Characteristics? • How Do External Groups Influence Institutional Adoption of Various Measures of and Approaches to Student Assessment? • How Are External Influences on Institutional Approaches to and Organizational and Administrative Support Patterns and Practices for Student Assessment Related to the Use and Impact of Student Assessment Information?

9. Phase III: Case Study Research Research Purposes 1) To Examine the Internal Dynamics of Institutional Support for Student Assessment 2) To Identify Institutional Strategies for Supporting, Promoting & Integrating Student Assessment 3) Assess the Culture & Climate for Using Student Assessment Data for Academic Improvement • Seven Institutions With Extensive Involvement in Student Assessment • Review of Response Patterns to National Inventory of Institutional Support for Student Assessment (ISSA) • Campus Site Visits by Research Teams • Collection & Synthesis of Documents, & Records • Interviews & Focus Groups With Faculty, Administrators, & Students

10. Integrating Student Assessment with Academic Management for Institutional Improvement Institution’s Academic Management Approach • Assessment • Leadership • Support • Coordination • Assessment • Purpose • Structure • Approach Links to Management Process/ Improvement Use in Academic Decision Making

11. Discussion of Institutions • Brief Institutional Storyline • Institution’s Academic Management Approach • Primary Mission & Governance Pattern • Key Academic Management Processes • Student Assessment Strategy • Purposes of Assessment • Assessment Structure • Approach to Assessment • Leadership, Support, & Coordination • Links to Academic Management Processes • Utilization in Academic Decision Making • Summary - Culture for Student Assessment

12. The Story of Institution Developed a Quality Initiative Based on a Baldridge Model Assessment Adopted as a Means of Academic Improvement Assessment Activity Gradually Integrated into Culture of Quality and Improvement for Many Years

13. Primary Mission & Governance Pattern • Vision • Emphasis on Culture of Quality; Sets Goals • Statement of Mission • Focuses on Continuous Improvement • Committed to Using Assessment to Enhance Performance • Cultural Core Values • Define the University; Drive the Mission & Vision • Decision Drivers • Priorities Regarding Essential Functions

14. Strategic Planning Council Seven Step Planning Process Program Management Student Affairs KQIs Office of Assessment, Information, & Analysis Approach to Academic Improvement Key Academic Management Processes

15. Assessment Purpose -- NMSU Initiating Conditions Internal- President’s Focus on Culture of Quality (early 1980s) External- State Mandated Assessment by Governor (late 1980s) President’s Early Efforts Resisted • Culture of Quality • Focused on Academic Improvement in Undergraduate Education • Adapted From Baldridge National Quality Award Criteria

16. Assessment Structure -- NMSU Formal, University-wide Strategy Uses Baldridge Quality Award Criteria As an Assessment Framework • Comprehensive Approach -- Cognitive and Affective Learning • College and Post-college Experiences • Collected by by Office of Assessment, Information, and Analysis (OAIA) Seven Step Planning Process

17. Assessment Approach -- NMSU Quantitative & Qualitative Data Gathered From Recruitment Until Post-graduation • Battery of Cognitive Measures During College Years • Regular Collection Schedule • Data on Needs or Expectations of Students Using Local and National Surveys & Listening Posts Assessments are required--Explained in UG Catalog • Student progress tracked and enforced through restrictions on registration and final transcripts

18. Leadership, Support, & Coordination • Leadership • Long-term, Continuous Presidential Promotion • Strong Support by Provost Through OAIA • Other Groups: • President’s Cabinet & Deans, Baldridge Category Council • Faculty Senate Assessment Committee • Student Success Task Force • Faculty Provide Significant, but Hesitant, Support • Coordination • OAIA-- Compiles and Distributes Assessment Data • Formal Grid Details Methods, Uses, & Frequencies for Assessment Activity

19. Links to Academic Management Process/Improvement • Strategic Planning Council • Representatives From Administrative, Academic, and Student Affairs • Assessment Info Used: • To Evaluate Organizational Performance on Strategic Initiatives (SI) • The Development of New Key Quality Indicators (KQI) for Each SI Annual Departmental Reports & Course Reviews • Primary Link Between Action Planning & Assessment • Deans Uses Assessment Data to Identify Areas for Improvement • Focus Is on Student Performance and Experience Data • President's Cabinet Reviews Reports & Provides Feedback to Unit Leaders

20. Links to Academic Management Process/ Improvement • OAIA Produces Three Types of Academic Information: • 1) Outcomes and Performance; • 2) Student and Stakeholder Satisfaction Ratings; and • 3) Operational Effectiveness • Measurement of of Key Quality Indicators • Seven Step Planning Process • To Meet Stakeholder Requirements • Data Is Distributed Through Dashboards and Profiles • Tracking Systems to Indicate Deviations From Intended Targets • Can Be Accessed Through Campus Network

21. Utilization in Academic Decision Making-- NMSU • Planning • Phase IReview of Mission and Analysis of Progress on Current Goals • Phase IIIdentification of Strategic Initiatives • Phase IIIAction Planning and Development • Actions Plans Are Developed to Improve Performance on • Key Quality Indicators • Purpose Is to Increase Organizational Performance • Assessment Information Part of KQIs in Dashboards & Profiles • Critical to Developing Agenda for Planning Efforts

22. Utilization in Academic Decision Making • Program Management • Annual Reports & Reviews • Benchmarking • Stretch Goals • Quality • Student Affairs-Academic Affairs Alliance • Development of Student Life & Experiences Surveys/Studies • Links to Educational uses of Assessment Information • NMSU Applies Annually for Baldridge Quality Award

23. Summary-- Culture for Assessment • Strong, Central Administrative Support for Quality Approach & Assessment • Baldridge Quality Criteria Drive Assessment Framework • Integration of Assessment into Quality, Planning, and Program Management Processes • Data-Driven System & Reports • Key Coordination Role Played by OAIA -- • Commitment to Assessment • As a Means of Visibility for Quality Emphasis • Increasingly for Institutional Improvement

24. The Story of Assessment Data Essential to Cycles of Institutional Planning, Program Review, & Evaluation Assessment Information Used by Decision Makers at All Levels to Evaluate Progress Towards Goals Administration Expects that Assessment Activity Will Occur for the Purpose of Academic Improvement

25. Primary Mission & Governance Pattern • No Formal Statements Regarding Assessment • WFU Administration believes Assessment is a • Necessary Management Tool to Maintain & • Improve its Quality and Reputation • Obtaining Data Crucial to Decision Making

26. Institutional Strategic Planning Evaluation Committee Institution-wide Assessment Departmental Review Student Life Office of Institutional Research Approach to Academic Improvement Key Academic Management Processes

27. Assessment Purpose -- WFU • Initiating Conditions • Internal- Foundations for Institutional Cycles of Planning Initiated by President & Vice-president • External- Accreditation Focus on Institutional Effectiveness • Evaluate Strategic Plan • The Plan for the Class of 2000 • Assessment Now a Management Tool in a Continual Process of Planning, Implementation, and Evaluation • Program Planning & Evaluation Committees Use Assessment Data • To Address Major Campus Needs • Ongoing Strategic Planning

28. Assessment Structure -- WFU Three Level, Decentralized Process • 1) University-wide-OIR- Oversees Assessment of the Process • Administers National & Local Surveys to Students & Faculty • Analyzes, Stores, & Disseminates Data • Works With Evaluation Committee & Other Depts. • Only Office to Have Dedicated Funds for Assessment • 2) Departmentally Based- Program Review Process • Primary Planning Unit on Campus • 13-month Process, 7 Year Cycle for All Programs • Review of Curriculum & Department/ Program Performance • $2,500 Budgeted for Each Review • 3) Student Life-Focuses on its own programs • Focus is on student experiences & satisfaction • Important for Intellectual Climate Planning

29. Assessment Approach-- WFU Primarily Quantitative Focus on Satisfaction & Experiences • Little Emphasis on Cognitive or Affective Measures • Some Determined by Dept. In Departmental Review Institution-wide Surveys of Students, Faculty, & Alumni • National Surveys Administered by Office of Institutional Research • Also Some Locally Developed Instruments & Course Evaluations Departments Conduct Own Assessment of Programs & Satisfaction • Significant Autonomy & Discretion Granted to Evaluate Their Students

30. Leadership & Support • Institutional Support from Executive • President’s and Provost’s Seminars, Conferences, & Retreats • Executive Council & Reynolda Cabinet Receive Reports & Use the Information in Strategic Planning

31. Coordination • OIR Has the Primary Role In: • Administering Institution-wide Student Assessments • Coordinating Program Planning • Conducting Data Analysis OIR Shares Its Information With Departments Upon Request Student Life Provides Its Data to Departments Information Systems Office Sends Course Evaluation Results to Department Heads Online OIR Role in Accreditation Allows It to Maintain Focus on Institutional Effectiveness Criteria

32. Information Flow/ Link to Improvement • Data Driven Approach to Management • Measuring Progress on Strategic Plan Goals Results in Steps Toward Improvement • Annual Reports by Departments to Deans • Must State Progress on Goals and Set New Ones OIR & Departments Report to Evaluation Committee (EC) for the Strategic Plan Reviews Departmental Reports on How SA Info Was Used to Monitor Progress on Goals • EC Monitors 27 Areas of Quality • EC Members Serve As Liaisons to Other Committees and Departments

33. Utilization in Academic Decision Making • Institutional Strategic Planning • Prior Data Streams Produced the Development, Implementation • and Evaluation of Plan for the Class of 2000 • Recent Student Life Data Critical to Next Wave of Planning on • Intellectual Climate • Program Review & Departmental Evaluations • One Full Cycle Completed • Departmental and Curricular Changes, including Program • Restructuring or Elimination • Departments use OIR Data & their own to Improve Teaching & Learning

34. Utilization in Academic Decision Making • Student Life • Units Engage in Planning & Evaluation • Many Changes to Services Made Based on Satisfaction Survey Results • Evaluation Committee • Assesses Laptop Initiative and Effect of Technology on Academics • Purpose Remains Measurement of Progress Towards Academic & Educational Goals

35. Summary-- Culture for Assessment • Academic Management is Well-designed • Integrated Institutional Planning, Program Review, & Academic Evaluation Approach that is Improvement Driven • Student Assessment Integrated into All Academic Management Processes • Strong, Central Support for Data Driven Management • Intensive Involvement of Departments in Program Review • Key Support & Coordination Roles Played by OIR • Commitment to Assessment is Embedded in Culture, but Not as an End in Itself • Wide Support for Student Assessment

36. Conclusions

37. Lessons & Insights • Need a Clear Understanding of: • The Institution’s Academic Management Approach & its Key Elements • The Role of Assessment in Their Various Elements • Emphasis is on Integrating Student Assessment into Key Elements & Academic Decision Making Arenas: • Assessment Used at All Levels of Institution • Tied to Academic Improvement • Embedded in Academic Management Policies & Practices • Assessment Data Key to Management & Evaluation Processes

38. Lessons & Insights • An Effective Student Assessment Strategy Requires: • Clearly Understood Purpose for Assessment • Committed, Upper-Level Leadership • Coordination Across Support Offices & Departments • Flow of Information to Decision Makers at All Levels • Link of Data to Problem Identification and Goal Setting • Building a Culture that Values Assessment Requires: • Long-term Effort Linking Assessment to Academic Management Processes Concerned with Academic Improvement

40. Wake Forest Case Study Visit September 2000 • Wake Forest Honored to Be Selected for the Case Study • Faculty, Students and Administrators Were Most Cooperative in Participation of Interviews and Focus Groups • Interviews Included President, Vice Presidents, Deans, Associate Provost, Advising Director, Director of Teaching & Learning Center, Director of Institutional Research, Chief Information Officer, Faculty Senate President, Chair of Curriculum Committee, Chairs of Academic Departments and Members of Evaluation Committee • Focus Groups Included Four Faculty Members and Four Departmental Chairs Active in Assessment and 10 “Good” Students

41. Lessons Learned • Considerable Assessment With Decided Outcomes Took Place at Institutional Level, e.g. Program Review and Surveys of Students and Faculty • Assessment at Departmental Level Is Decentralized and Varies Considerably • Qualitative Assessment Could Be Emphasized More at the Institutional Level • Combination of SACS 1995-97 Self Study and 1992-95 Strategic Plan Strengthened Assessment at Wake Forest