International Assessment and Retention Conference, St. Louis, MO, June 8, 2007

1. Learning Connections 24/7: Assessing Learning across Curricular, Co-Curricular and Student Development Programs International Assessment and Retention Conference, St. Louis, MO, June 8, 2007 Margaret Bloom, Vice Provost Christine Krueger, Core of Common Studies Mark McCarthy, Student Affairs Susan Mountin, Manresa Project

2. Marquette University • Research Extensive • Private, faith-based • Mission driven “Marquette University is a Catholic, Jesuit university. Our mission is the search for truth, the discovery and sharing of knowledge, the fostering of personal and professional excellence, the promotion of a life of faith, and the development of leadership expressed in service to others.”

3. Learning Assessment at Marquette The Marquette assessment system is focused on two essential learning outcomes: • Institutional learning outcomes which address the integrated learning expected of every student upon graduation. • Program (undergraduate major and graduate degree) learning outcomes expected of every student at program completion.

4. Framework for Undergraduate Learning Assessment College Competencies Discipline Competencies Values, Commitments & Leadership Required Courses Learning Objectives Services/Programs Core Courses Learning Objectives

5. Governing Principles of the Marquette Assessment System • Common Definitions and Assessment Process Each program and unit uses the common MU assessment system vocabulary and has an assessment plan that contains the specified MU common assessment system components. • Program and Unit Flexibility and Decision-Making Each program faculty/unit staff define learning outcomes, performance indicators, identify and use measures, and evaluate their data.

6. MU Assessment System Components MARQUETTE MISSION CONSTITUENTS LEARNING OUTCOMES FALL 2005 MEASURABLE PERFORMANCE INDICATORS ACTIONS TO IMPROVE LEARNING Fall 2007 FEEDBACK FOR CONTINUOUS IMPROVEMENT DIRECT & INDIRECT MEASURES Spring 2007 EVALUATION INTERPRETATION OF EVIDENCE August 2007 ASSESSMENT COLLECTION, ANALYSIS OF EVIDENCE Late Spring 2007 Modified from Rogers, G. Rose-Hulman Institute of Technology, 2005

7. MU Assessment Systems: Integrated Levels of Assessment LearningOutcomes Actions to ImproveLearning Performance Indicators Institutional Evaluate Measures Collect & Assess StudentAffairs Undergraduate Major Business Administration Student MissionRelated Programs Business Core Major Coursework Core of Common Studies

8. Broad Participation a Key FactorFaculty and Staff Directly Involved in Development of the Marquette Assessment System Institutional Level University Assessment Committee Board of Undergraduate Studies Assessment Steering Team 36 Faculty/Staff/Administrators Program Assessment Leaders 108 Faculty Core Curriculum Review Committee 15 Faculty Student Affairs and Co-Curricular Program Directors 16 Directors

9. Marquette (Institutional) Undergraduate Learning Outcomes Upon completion of an undergraduate degree at Marquette, students are able to… • Utilize critical thinking and reflect to effect positive change in themselves, others, and their communities. • Exercise just, responsible, and competent leadership in professional, intellectual, and societal contexts. • Apply the knowledge and skills of an academic discipline, program, or profession to a career or graduate study. • Communicate in modes appropriate to various subjects and diverse audiences. • Pursue an integration of knowledge into a comprehensive, transcendent vision of life. • Act for social justice within the diverse, global human family.

10. Majors and Degree Programs 58 Majors 35 Masters Degree Programs 17 Doctoral Degree Programs 3 Professional Degree Programs

11. Majors and Graduate Programs • Each Academic Program’s Assessment Plan indicates direct and indirect measures. • Located at www.marquette.edu/assessment/plans/index • Direct measures often include senior-level papers, case studies, or projects. • Indirect measure: University Senior Survey contains up to 5 items for students to self-rate on their specific program learning outcomes. 1st Annual Program Assessment Reports due Sept. 31, 2007

12. Core of Common Studies Assessment and Integration

13. University Core of Common Studies Integrated Learning Outcomes Upon completion of the Core of Common Studies, students are able to… • Apply the perspectives, concepts and traditions of multiple disciplines to personal, professional, intellectual and societal challenges. • Communicate in modes appropriate to various subjects and diverse audiences. • Pursue an integration of knowledge into a comprehensive, transcendent vision of life. • Act as responsible members of the global human family, with knowledge of, and respect for, individuals and cultures in their diversity.

14. Pilot Core Assessment On-line Integrated Assessment Instrument Integrated foundational competencies for responsible leadership • Reading comprehension of brief text offering multidisciplinary analysis of global problem • Interpretation of quantitative and qualitative evidence via multiple-choice questions seeking best answer • Communication of diverse content to diverse audiences via 3 randomly assigned tasks: • Summary for senator • Letter to editor of non-US student newspaper • Graph illustrating data extrapolated from reading

15. Pilot Core Assessment Results • 25 student volunteers (17 completed) • Preliminary Results • Quantitative/qualitative evidence questions (12%-71% outstanding) • Prose communication (80% > proficient) • Graphic communication (20% proficient) • Understanding the Results • Student focus group • Actions • Refinement of instrument and rubrics • Implementation for incoming seniors • Math across the curriculum??

16. Student Affairs Assessment Counseling Center Office of Residence Life Recreational Sports Office of Student Development Student Health Services

17. Student Affairs DivisionShared Learning Outcomes As a result of their experiences with the Student Affairs services and programs, students are able to… • More clearly understand who they are in regards to mind, body and spirit. • Utilize their skills to create positive change in themselves, others and their communities. • Collaborate with others to accomplish goals. • Engage with others who are different from themselves.

18. Office of Student DevelopmentLearning Outcomes After participating in student development programs students are able to… • Use their talents to benefit others. • Interact effectively with people from diverse backgrounds and lifestyles.

19. Population Based Assessment • Multi-Institutional Study of Leadership (Spring 2006) • Based on the Social Change Model of Leadership (HERI, 1996) • Two groups participated in the study • Random sample of MU undergraduate students • Select sample of students involved in Student Affairs sponsored leadership and service experiences.

20. MSL Initial Findings Related to Student Development Outcomes No mean difference reported between the random and student affairs involved groups on pre-test scales for: • Cognitive Development • Leadership Efficacy • Appreciation of Diversity Significant mean differences found between groups on the scales for: • Citizenship, • Leadership Efficacy, • Diversity Discussions • Appreciation of Diversity

21. Program Based Assessment Leadership Summit One-day urban experience followed by keynote address on integrating faith and learning in action and reflection activities Assessment Results • 90% stated that as a result of participation I… • Am more likely to take action to address needs and problems in the community. • Have a greater appreciation for different perspectives. • Am more likely to educate others on issues affecting the Milwaukee community.

22. Key findings related to outcomes Leadership Summit assessment results • 80% stated that as a result of participation I… • Better understand the importance of reflection. • Learned about methods that leaders use in creating change. Learning Themes Identified in statements of reflection • Deeper understanding of the broader Milwaukee community. • Increased commitment to serve others. • Greater understanding of themselves and incorporation of values such as faith, justice, compassion, empathy and hope into work and life.

23. Assessment Challenges • Whom to assess • Controlling for pre-college experiences and perceptions • Alignment of program goals with assessment strategies • Impact of singular experiences on student development • Complexity of developing and using direct measures

24. Integrative ProgramsMission Related Programs Manresa Project

25. Mission Related ProgramsThe Manresa Project • After participation in Manresa Project activities students are able to… • Understand the difference between discernment and decision-making. • Practice Ignatian Spirituality as a component of their faith and life choices. • Understand Jesuit Global Mission ideals. • Explore individual meaning and purpose in life (vocation).

26. Assessment Tools Used Direct measures • Spiritual Autobiography • Site-supervisor observations • Course assignments • Weekly group reflection Indirect measures • Surveys: Multi-Institutional Study of Leadership (MSL) • Interviews • Focus groups

27. The Manresa Project Indirect Assessment Results 2006 • Used the MSL data comparing Marquette students with students at other Catholic universities • Marquette students significantly higher on items (outcomes) • Developing understanding of social justice • Strengthening spiritual life • Defining values • Becoming more independent • Knowing how “God has called me” to use my gifts and talents • Integrating service into life

28. The Manresa ProjectAssessment Challenges • Program has many diffuse opportunities for engagement • First year reading • Manresa supported courses • Sponsored speakers • Seminars, workshops and conferences • Retreats • Data collection difficult • Relies on external partners (faculty, site supervisors,etc.) • Direct contact with small number of students, but a broad impact across the Marquette community

29. Assessing 24/7 Learning: Unanticipated Cultural Changes • Faculty working across disciplines on integrating global understandings: • Shared Futures Project with AAC&U • Assessment has been an integrating influence across campus • Inclusion of Mission based learning in the common understanding of Marquette student learning