What Are Outcomes and Why Should I Want to Assess Them?

1. What Are Outcomes and Why Should I Want to Assess Them? Dr. Jim Wilson, AVP Academic Affairs Wilmington University Dana Santoro, Data Analyst – Institutional Research Wilmington University

2. Class of 2011 • What Berlin wall? • They never “rolled down” a car window • They have grown up with bottled water • Wal-mart has always been a larger retailer than Sears and has always employed more workers than GM • High definition televisions have always been available • China has always been more interested in making money than in re-education • They never saw Johnny Carson live on television • Food packaging has always included nutritional labeling Courtesy of Beloit College Public Affairs

3. Things Have Changed! • Higher Education Research Institute (HERI) • UCLA 40yr Trends Presentation

4. Things Have Changed!

5. Education: Costs Wilmington University Tuition 30 credits, Undergraduate 1971 Full year tuition: $1,500 Today Full year tuition: $8,400 (New Castle, UG)

6. Education: Some Changes • Today • Student-centered learning • The “Net Generation” • Distance Learning, Online databases • 1967 • Teacher-centered learning • Classroom, Books, Library card catalogs • Al Gore was only 20 years old Courtesy of Beloit College Public Affairs

7. Education: Student Learning • A new “paradigm for higher education” (Suskie, 2006.) • Learning Centered • Assessment feedback helps faculty understand • What is and is not working effectively • How to improve the curriculum as well as teaching and learning strategies

8. Wilmington University Mission

9. What is Assessment of Student Learning? • “Good assessment must begin in the classroom and end there.” (Wolvord & Anderson, 1998, p. 149)

10. What is Assessment of Student Learning? • “Systematically gathering, analyzing, and interpreting evidence to determine how well student learning matches our expectations.”(Suskie, 2006, p.3.)

11. What is Assessment of Student Learning? • “Assessment is first and foremost a tool for faculty members to use as they do their very best to teach their students well.” (Middle States, 2003)

12. Assessment Is A Friend Who… • Honestly tells us how we are doing • Helps to clarify what we are doing and why • Helps us to make informed decisions • Improves student learning • Helps us to improve • Brags about how well we are doing

13. Assessment Could Be a Foe! • Federal requirements • Regional Accreditation • Legislators • Parents and Community groups • Students • Professional organizations

14. SummaryWhat is Assessment of Student Learning? • Process of gathering & evaluating information • Quantitative and qualitative • From multiple sources • To enhance student learning

15. Student Outcomes Assessment Framework

16. Three Assessment Areas • Student Learning • Teaching Effectiveness • Student satisfaction with their academic experience

17. Wilmington University ACADEMIC AFFAIRS ASSESSMENT PLAN COLLEGE MISSION Assessment of Teaching Effectiveness Assessment of Student Satisfaction Assessment of Student Learning Outcomes Graduation Competencies Benchmarks Benchmarks Division / Program Mission Professional Standards -------- PROGRAMCOMPETENCIES • IDEA Results • Faculty Evaluations • GPA Reports • Enrollment Data • ACT Survey Results Benchmarks License & Certification Exams College Outcomes Course Work Major Field Exams CECRAM Rubrics, Test Embedded Questions, Clinical Evaluations Data Collection, Analysis, Reporting, Decisions,Actions

18. Program Improvement Cycle

19. Language of Assessment • Direct evidence • Direct evidence of student learning indicates whether or not a student has a command of specific content • Examples: Homework assignments, research papers, rubrics • CECRAM (Course-Embedded Criterion-Referenced Assessment Measures)

20. Language of Assessment • Indirect Evidence • Indirect evidence of student learning is correlational – data exists which indicates that students are probably learning. • Examples: Course grades, time spent on service learning or homework

21. Language of Assessment • Formative assessment • Ongoing assessment that is intended to improve an individual student’s performance. • Purpose: to improve course content, provide feedback to faculty.

22. Language of Assessment • Summative assessment • Provides a true gauge of “outcomes” of student learning. • Data is typically gathered near the end of program completion • Examples: course-embedded project/test/portfolio, clinical evaluations, standardized exams

23. WU Language • “Good enough” • The process is accurate and truthful (Suskie, 2004, p. 302) • “Closing the loop” • Outcomes data has been analyzed and appropriate changes have been made.

24. Elements of Assessing Learning • Articulated expectation of student learning at various levels. • A plan that describes student learning assessment activities. • Evidence that student learning assessment information is used. • Documented use of student learning assessment as a part of institutional assessment.

25. Background Information • Academic Affairs Assessment Plan adopted in 2003 • Plan called for evaluation in 2006

26. MSCHE Outcomes Assessment Guidelines Assessment should be: • Useful • Cost-effective, simple • Reasonably accurate and truthful • Organized, systematized and sustained. Suskie, L. (2005)

27. Major Changes Related to Outcomes Assessment • Five Year Data Collection and Implementation Cycle • Revision of Graduation Competencies • Representative Sampling • Formal Reporting Cycle • Five Year Program Review

28. How to maintain momentum? • A 5 year cycle will be utilized • Year 1 – Implement findings from program review • Year 2 – Data collection • Year 3 – Reflection, take appropriate action and collect “other kinds” of data • Year 4 – Data collection • Year 5 – Program Review Report

29. What are Examples?(of closing the loop) Program: B.S., Finance • Weakness: Undergraduate Finance majors were missing some important finance skills • Source: ETS Major Field Exam • Action • Adjust the content of FIN 305 • Add a new requirement, FIN 306 • Competency • Disciplined Inquiry • Finance program competencies

30. What are Examples?(of closing the loop) Program: All UG Education programs • Weakness: Students were unclear of required skills • Source: Portfolio review • Actions: • Curriculum changes • Provide candidates with exemplars • Ensure understanding of required skills • Competency • Disciplined inquiry

31. What are Examples?(of closing the loop) Program: M.S., Community Counseling • Weakness: Appraisal Techniques • Source: CECRAM, CPCE*, NCE* & surveys • Response: Integration of assessment with diagnosis and treatment planning. • Psychopathology course will precede the appraisal course *Standardized exams

32. What are Examples?(of closing the loop) Program: B.S., Nursing • Weakness: Student teaching projects - measurable objectives were not reliably clear • Source: CECRAM • Actions: • Add content and a teaching video to NUR 425 – Community Health Practitioner • Added teaching content to NUR 330 • Revised curriculum and developed a new course: NUR 323 - Nurse as Teacher

33. Conclusion What is outcomes assessment of student learning? Why should I support the assessment process?

34. References • Caffo, Betty, (2003). Striving for Excellence. Faculty Senate Presentation, Wilmington College. • Middle States Commission on Higher Education. (2006). Characteristics of excellence in higher education. Philadelphia: Author. • Suskie, L. (2004). Assessing student learning: A common sense guide. Bolton, MA: Anker. • Walvoord, B.E., and Anderson, V.J. (1998). Effective grading: A tool for learning and assessment. San Francisco, CA: Jossey-Bass.