Multiple Perspectives on Directing Assessment in Student Affairs

1. Multiple Perspectives on Directing Assessment in Student Affairs 2008 International Assessment & Retention Conference Scottsdale, AZ June 13, 2008 Presented by: A. Katherine Busby, PhD Director of Student Affairs Assessment and Planning University of Alabama Ellen Meents-DeCaigny, PhD Director, Student Affairs Assessment, Research and Communications DePaul University

2. Overview In assessment one size does not fit all… • Describe the context in which we work • Present and discuss 4 scenarios related to assessment • Culture • Metrics • Learning Outcomes • Closing the Loop • Offer insights based on our contexts and experiences

3. Student Affairs Assessment at University of Alabama • Office formed in 2006 • Staff includes director and one graduate assistant • Modest, director controlled budget • Director serves as “internal consultant” • Reports to Associate VP • Decentralized assessment across division • Leadership appreciates, but does not dictate assessment • Active Student Affairs Assessment Council • Emerging partnerships with other assessment professionals

4. UA Model of Assessment

5. Student Affairs Assessment at DePaul University Part-time Coordinator position created 2004 Full-time position created 2007 1 full-time staff member & 2 graduate assistants report to Director Budget has not yet been established Director reports to VP Division-wide approach to assessment Student Affairs Assessment Committee Strong partnerships with Institutional Planning and Research and Teaching, Learning and Assessment Centralized approach to assessment at institution

6. DePaul’s Integrated Model of Assessment Division of Student Affairs Mission Success Factors: Programs, services & collaborations in support of mission Department Key Activities Measures: Cost Magnitude Satisfaction Learning Outcomes Department Assessment Question Measures: How are students learning, engaged orinvolved?

7. Culture: The Human Element “Successful assessment is not primarily a question of technical skill but rather one of human will” (Angelo, 1999). Establish a support structure Build shared trust Build shared motivation Develop a shared language Establish key partners in assessment Develop shared guidelines and expectations

8. Building Culture at DePaul “Assessment Initiatives flounder because they’re headed-up by people who lack the time and clout to accomplish what is necessary” (Cohen and Kotter, 2002). Establish a Support Structure: Chief Student Affairs Officer: Vice President of Student Affairs Senior Level Sponsor: Associate Vice President of Student Development Assessment “Champion”: Director of Assessment, Research and Communications

9. Building Culture at DePaul Build Shared Trust: Hire a knowledgeable, trusted and well-respected “champion” Spend time gathering buy-in Divisional meetings Department meetings Individual meetings Build Shared Motivation: Focus on the benefits of assessment Meet individuals and departments at their level Discuss ALL motivations related to assessment

10. Building Culture at DePaul Establish key partners in assessment: Find institutional partners involved in assessment and research Select Assessment Committee members interested in the initiative Develop shared guidelines and expectations: Present the assessment model Discuss the logic behind the assessment model Distribute the model, timeline and report templates in a timely fashion

11. Building Culture at DePaul Develop a shared language: A challenging process! Present terminology related to the model Define important terms learning outcomes process vs. outcomes assessment Develop data definitions

12. Culture:What I do can’t be assessed. Although most student affairs professionals realize that assessment is a part of their work life we still hear the cries of “what I do can’t be assessed.” Sometimes these cries come from someone who does not buy into assessment, but other times these cries come from those who fear the assessment unknown.

13. A few keys points about culture Helps to have a point person who is focused on assessment Helps to have VP and upper level leadership supporting the process and serving as the “champion” Make the process manageable and provide support for the process Demonstrate how the information will be used (and follow through!) and incorporate it into other areas of the division (annual reports) Determine which battles to fight

14. Learning Outcomes:More than just a well phrased sentence. Developing and utilizing learning outcomes is easier said than done. Some student affairs professionals cling to their “operational outcomes”, indicators, and measures of satisfaction (which are important in their own way) but fall short when it comes to learning.

15. Developing Learning Outcomes at UA • Audience • Behavior • Condition • Degree of Achievement (may be omitted) R. Heinich, M. Molenda, J. Russell, S. Smaldino (2002). Instructional Media and Technologies for Learning, 7th Edition. Englewood Cliffs: Prentice Hall, Inc.

16. Developing Learning Outcomes at UA To write a learning outcome, follow the formula: ConditionAudience Behavior Degree

17. DePaul’s Outcome Evaluation Tool Key Activity #1: (Name) Keeling and Associates, LLC, 2006

18. A few key points about learning outcomes • Shoot, ready, aim OR ready, aim, shoot • Understand and utilize different types of outcomes • Embrace learning outcomes as an iterative process

19. Metrics: What is the purpose? Student affairs practitioners often want information particular to their program or office. However, a good assessment plan may call for that same information to be used at the unit/department or division level. Make the most out of your data by using carefully designed metrics.

20. DePaul’s Use of Metrics Metrics collected for departmental and divisional purposes Metrics enhance data collected through assessment projects Challenges: Developing common terminology Developing common methods of measurement Reporting metrics in ways that are useful to external audiences

21. UA’s Use of Metrics • Utilizing existing student data • Beginning to identify metrics across units

22. A few key points about metrics Developing common terminology is key to examine data across the division Careful collection of data will allow stories to be told at programmatic level, unit level, and division level Appropriately used metrics affords the opportunity to get the right information to the right person for the right reason

23. Closing the Loop: How do we use the data? A successful assessment does not mean that the results met or exceeded our expectations. Rather it means that we used the results to inform our work and improve the college experience for our students.

24. Closing the loop at DePaul • Provide assistance with data analysis • Help make connections between departments and across the division • Help connect results to institutional and national data • Continue to find ways to share results: • Departmental and Divisional Annual Reports • Assessment Celebration • Presentations in and outside the division • Share departmental changes that result from assessment

25. Closing the loop at UA • Consultation after the analysis • Distribution of reports to targeted audiences

26. A few keys points about closing the loop • Assist staff through interpreting the results (even disappointing results) • Maintain focus on improvement • Demonstrate for staff how they can utilize results of national surveys • Share information in multiple formats

27. Contact us Katie Busby Director, Student Affairs Assessment and Planning Box 870116 Tuscaloosa, AL 35487-0116 205-348-7939 205-348-2591 Kbusby@sa.ua.edu Ellen Meents-DeCaigny Director, Student Affairs Assessment, Research & Communications 25 E. Jackson Blvd, Suite 1400 Chicago, IL 60604 312-362-7298 312-362-5671 emeentsd@depaul.edu