THE INSTITUTE ON INSTITUTIONAL EFFECTIVENESS & ASSESSMENT

1. THE INSTITUTE ON INSTITUTIONAL EFFECTIVENESS & ASSESSMENT “SELECTING MEASURES FOR EDUCATIONAL (a.k.a.: STUDENT LEARNING) OBJECTIVES” University Library Multi-Purpose Room, Second Floor Wednesday, November 14, 2007 1:00 p.m. – 3:00 p.m. Eva J. Harris Coordinator, Institutional Effectiveness & Assessment eharris@susla.edu (318) 429-7220

2. Assessment …. another definition Assessment is an ongoing process aimed at an understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance. Tom Angelo, AAHE Assessment Forum PAR-IEA Harris, Eva J. (11-07)

3. Evaluation versus Assessment • Evaluation – Performance is measured – A score or label is assigned – Judgment is rendered (comparison with others) – Praise or condemnation ensues • Assessment – Performance is measured – Comparison is made with past results – Improvement is made (key: continuous Improvement) PAR-IEA Harris, Eva J. (11-07)

4. Purpose: What Does Outcomes Assessment Accomplish? • Ensures that units are examining their services and programs. • Documents outcomes assessment and internal improvement efforts for the Chancellor, Vice Chancellors and Accrediting Agencies. • Permits each unit to demonstrate how well it is doing. PAR-IEA Harris, Eva J. (11-07)

5. Purpose: What Does Outcomes Assessment Accomplish? • Identifies internal improvements based on fact, not perception. • Allows unit staff to focus on implementing improvements they feel are high priority. PAR-IEA Harris, Eva J. (11-07)

6. Effective Outcomes Assessment Plans should include: • Mission & Goals • Objectives • Defined Outcomes & Measures (Direct & Indirect) • Assessment Methods • Communication of Outcomes Assessment Results • Decision & Internal Improvement Implementation Remember the cycle! PAR-IEA Harris, Eva J. (11-07)

7. What are measurable student learning outcomes? • Means determining if intended learning has actually occurred • Student learning includes the full breadth of education • Acquisition of Skills • Mastery of concepts • Growth in Life Perspective Can students analyze market trends? express themselves creatively? PAR-IEA Harris, Eva J. (11-07)

8. Benefits of Formulating Learning Outcomes • Form the basis of assessment at the course, program, and institutional levels. • Provide the direction for all institutional activity. • Inform students about your intentions and expectations. PAR-IEA Harris, Eva J. (11-07)

9. What determines if the desired learning has occurred? • Objectives that are clearly defined in measurable terms. ……… • Evaluating titration ability…….. PAR-IEA Harris, Eva J. (11-07)

10. PAR-IEA Harris, Eva J. (11-07)

11. The Outcomes Assessment Cycle: an ongoing process for the Unit • Questions to be asked by Division Head/Unit staff • What are we trying to do? (Mission) • How will we do it? (Objectives) • How well are we doing it? How do we know? (Outcomes Assessment) • How do we use the information to improve? (Improvement Results) PAR-IEA Harris, Eva J. (11-07)

12. PAR-IEA Harris, Eva J. (11-07)

13. Direct Measures • Examples: • Capstone courses/experiences • Portfolio assessments • Standardized tests (Major Field Achievement Test, Critical Thinking Ability, Academic Profile, etc.) • Licensure exams and certifications (PRAXIS, FAA Exam) • Local common exams • Written/oral comprehensive exams • Juried reviews of projects, exhibitions, performances • Assessment rubrics • Direct measures (outcomes) are learning assessment tools. • Academic units should use multiple direct measures of learning. PAR-IEA Harris, Eva J. (11-07)

14. Indirect Measures • Examples: • Exit interviews • Focus groups • Alumni, employer, and student surveys • Graduate follow-up studies • Graduation, retention, and transfer rates • Job placement data • Satisfaction surveys • Self-reported data • Indirect measures (outputs) can provide useful information but DO NOT directly assess learning. PAR-IEA Harris, Eva J. (11-07)

15. Course and Program Outcomes A program can be defined as any learning activity and/or major/department/division/service that generates or supports learning. College major programs Vocational Programs General Education Allied Health Student Services A.A. and/or A.S. degrees Library Administrative Services Student or campus safety Financial aid PAR-IEA Harris, Eva J. (11-07)

16. Individual Student Learning Assessment of Student Learning Outcomes Course Learning Outcomes Program Learning Outcomes College Assessment Plan College Learning Outcomes PAR-IEA Harris, Eva J. (11-07)

17. PAR-IEA Harris, Eva J. (11-07)

18. PAR-IEA Harris, Eva J. (11-07)

19. PAR-IEA Harris, Eva J. (11-07)

20. PAR-IEA Harris, Eva J. (11-07)

21. “Closing the Loop” Mission Measurable Outcomes Institutional Effectiveness PAR-IEA Harris, Eva J. (11-07)

22. Documenting Assess, Analyze & Document Program Level Learning Outcomes (SLO’s) PAR-IEA Harris, Eva J. (11-07)

23. PAR-IEA Harris, Eva J. (11-07)

24. Southern University at Shreveport Institutional Effectiveness Process IE Form 2 – Status of Unit Objectives Division: ______ Instruction _______ Student Development _____ Business/Finance ______ Development _______ Chancellor PAR-IEA Harris, Eva J. (11-07) FOR USE by PAR & Planning and Priorities Committee ONLY.

25. Southern University at Shreveport Institutional Effectiveness Process IE Form 3 – Evaluation of Unit Objectives Division: _____ Instruction ______ Student Development _____ Business/Finance _____ Development ______Chancellor PAR-IEA Harris, Eva J. (11-07)

26. Evaluation’s most important purpose is not to prove, but to improve. (Platt) PAR-IEA Harris, Eva J. (11-07)

27. What can you do? • Develop an Assessment Program • Review previous outcomes • Match the previous outcomes with what you are doing NOW. • Write measurable learning outcomes statements. • Choose a means of measuring the learning…. PAR-IEA Harris, Eva J. (11-07)

28. What can you do? • Implement the strategy. • Evaluate the results. • Change something you do. • Plan, Measure, Learn • Plan, Measure…. …well, you get the picture! PAR-IEA Harris, Eva J. (11-07)

29. IN CLOSING Simple. Don’t be too ambitious or you will be overwhelmed by the assessment. Achievable. The outcome/objective has to be both measurable and within your control. Collectable. When possible and if of value, incorporate institutional/departmental/unit data already being collected. Sustainable. Select assessments that can be carried out for multiple years. PAR-IEA Harris, Eva J. (11-07)

30. QUESTIONS ANYONE? PAR-IEA Harris, Eva J. (11-07)