Assessment Basics: Implementing and Sustaining a Comprehensive, Outcomes-Based Assessment Plan

1. Assessment Basics: Implementing and Sustaining a Comprehensive, Outcomes-Based Assessment Plan October 19, 2006 Melissa Canady Wargo, Director Office of Assessment NCCEA Annual Conference NCCEA Annual Conference Waynesville, NC

2. Questions for today’s session: What is assessment and why should we care about it? How do I begin? What do I need to be doing? How can I use assessment to improve my program? Others??? NCCEA Annual Conference Waynesville, NC

3. Definition of Assessment “ … any effort to gather, analyze, or interpret evidence, which describes institutional, departmental, or agency effectiveness.” (pp. 3-4) * Schuh, J. and Upcraft, M. (2001). Assessment Practice in Student Affairs: An applications manual. San Francisco: Jossey-Bass. NCCEA Annual Conference Waynesville, NC

4. Why Assess Programs & Services? • To improve programs through informed decision-making • To document the quality of service or extent of student learning for all program constituents (faculty, administration, students, alumni, accrediting agencies, public officials, etc.). NCCEA Annual Conference Waynesville, NC

5. Types of Assessment • Needs Assessment • Environmental Assessment • Cost Assessment • Outcomes Assessment NCCEA Annual Conference Waynesville, NC

6. Why Outcomes-Based Assessment? • Shifts focus from services provided to results accomplished • Defines expected end result, assessment of which allows for focused improvement • Allows YOUto determine what constitutes an effective program NCCEA Annual Conference Waynesville, NC

7. What is outcomes-based assessment? Program, Division, University Mission, Vision, Strategic Priorities NCCEA Annual Conference Waynesville, NC

8. How do I go about assessing my program? • Engage all staff in assessment conversations • Commit to support assessment activities • Be willing to use assessment information to make programmatic changes NCCEA Annual Conference Waynesville, NC

9. How do I begin? Develop comprehensive assessment plan: • Define program purpose • Set program goals • Articulate intended outcomes • Identify assessment methods • Implement data collection/analysis strategies • Establish process to disseminate and act on results NCCEA Annual Conference Waynesville, NC

10. Formulating Program Mission 3 Key Questions to Address in Purpose Statement: • What is program? • What does program provide? • Whom does the program serve? NCCEA Annual Conference Waynesville, NC

11. Sample Mission/Purpose Statement The mission of the Career Services office is to enhance the academic, career and personal development of students by helping them better understand themselves and their environments, and to gain the knowledge and skills necessary for academic success, wise career selection, and satisfying interpersonal relationships. To that end, Career Services will consult with University faculty and staff regarding student needs and organizational development issues; and collect and disseminate information about changing student needs and the effectiveness of our services. NCCEA Annual Conference Waynesville, NC

12. Sample Mission/Purpose Statement University Career Services, an integral part of the educational process, assists students and alumni in assessing their career possibilities, setting their personal goals and achieving their objectives toward becoming productive citizens in the global community. While assisting its clients in identifying professional employment opportunities, University Career Services also provides the university community with insights into the ever-changing world of work to help develop realistic ways to better educate tomorrow’s leaders. (Texas Christian University) NCCEA Annual Conference Waynesville, NC

13. Formulating Program Goals • Focus on what the program provides to achieve the mission and outcomes. • Keep goals brief and general • Describe an ideal career/coop/internship program for your type of institution NCCEA Annual Conference Waynesville, NC

14. Sample Program Goals • To challenge and support students as they engage in self-exploration. • To respond to the critical needs of the University community. • To provide students with the information, experience and personal attention they require to make informed career choices commensurate with individual strengths and aspirations. NCCEA Annual Conference Waynesville, NC

15. Formulating Outcomes Type of Outcomes: • Program (Operational) • Learning NCCEA Annual Conference Waynesville, NC

16. Program Outcome • Statement of what a program intends to do, achieve, or accomplish through certain activities or experiences; i.e., what a program accomplishes for its students, faculty/staff or institution. NCCEA Annual Conference Waynesville, NC

17. Sample Program Outcomes • Placement Rate • # of Coop/Internships • Coop/Internship Participation Rates • # of Employers at Job Fair NCCEA Annual Conference Waynesville, NC

18. Learning Outcome • Statement of what a constituent should know, think, be able to do, or value as a result of participation in/completion of your program. NCCEA Annual Conference Waynesville, NC

19. Types of Learning Outcomes* • Complex Cognitive Skills • Knowledge acquisition • Intrapersonal development • Interpersonal development • Practical Competence • Civic Responsibility • Academic Achievement • Persistence * Adapted from Upcraft, M. (1999). “Assessing Student Learning.” In Stage, F., Watson, L., & Terrell, M. (eds.), Enhancing Student Learning: Setting the campus context. NCCEA Annual Conference Waynesville, NC

20. Learning Outcomes Most Applicable to Career/Coop/Internship Programs • Intrapersonal Development: formation of autonomy, values, identity, self-esteem, maturity • Interpersonal development: valuing diversity, establishing relationships, relating to others • Practical Competence: career prep, time/money management, self-sufficiency NCCEA Annual Conference Waynesville, NC

21. Formulating Outcomes Think SMART* • Specific • Measurable • Attainable • Results-Oriented • Time-Bound * Adapted from UCF Assessment Handbook NCCEA Annual Conference Waynesville, NC

22. Sample Learning Outcomes • Career counseling participants will be able to describe the steps involved in making career decisions. • Describe at least three job search strategies based on their career objective. • Explain to a potential employer how their background and skills match the employer’s needs. • Co-op students will exhibit the maturity and professionalism commensurate with the work environment. NCCEA Annual Conference Waynesville, NC

23. Group ExercisePart I Develop a learning outcome appropriate for a career/co-op/internship program: • Think about skills/knowledge/satisfaction related to student development. • Remember SMART NCCEA Annual Conference Waynesville, NC

24. Formulating Assessment Methods • What evidence will allow us to determine whether the outcome has been met? • Are those evidence/data attainable? • How often should they be collected? • Who is responsible for collection, analysis, reporting? • When will the collection, analysis, reporting occur? NCCEA Annual Conference Waynesville, NC

25. Types of Assessment Measures • Direct: • An assessment method designed to directly measure what a participant knows or is able to do (i.e., requires demonstration of the skill or knowledge). Most applicable to learning outcomes. • Indirect: • An assessment method that indirectly measures what a target subject knows, can do or values (i.e., what a subject perceives or is satisfied with). Most applicable to process/program outcomes. NCCEA Annual Conference Waynesville, NC

26. Assessment Measures Evaluation of: • Experiential activities (e.g. internships, voluntary activities, etc.) • Student work samples (e.g., case study responses, essay responses, etc.) • Collection of products demonstrating extent of learning (portfolios) • Surveys (student, alumni, etc.) • Exit interviews • Employment rates • Retention/graduation data • Demographics of student population • Focus groups NCCEA Annual Conference Waynesville, NC

27. Group ExercisePart 2 Develop two assessment methods appropriate to the outcome. Consider: • What data are needed? • Where will data be collected? • Are there mechanisms in place already? • How often are data collected? • How will data be disseminated and discussed? NCCEA Annual Conference Waynesville, NC

28. Discuss your results!Act on them!Celebrate your improvements! NCCEA Annual Conference Waynesville, NC

29. How can this process be managed more effectively? • Capitalize on activities already in place • Locate and use resources from around the University • Use your time and resources efficiently NCCEA Annual Conference Waynesville, NC

30. Questions/Comments ? NCCEA Annual Conference Waynesville, NC