Academic Program Assessment & Evaluation

1. Academic Program Assessment & Evaluation Fred Rovai Jim Downey

2. Background “College standards are becoming diluted and there is a fuzziness about what faculty teach and what is expected from students” (Commission on the Future of Higher Education). “Outcome-based education means starting with a clear picture of what is important for students to be able to do, then organizing the curriculum, instruction, and assessment to make sure that this learning ultimately happens” (Spady, 1994).

3. SACS Comprehensive Standards 3.3.1 – The Institution identifies expected outcomes for its educational programs and its administrative and educational support services; assesses whether it achieves these outcomes; and provides evidence of improvement based on analysis of those results. 3.4.1 – …establishes and evaluates program and learning outcomes. 3.5.1 – …identifies college-level competencies within the general education core and provides evidence that graduates have attained those competencies.

4. Student Learning Outcomes (SLOs) • Skills, knowledge, and attitudes that students are expected to acquire in a program and be able to demonstrate upon course and program completion. • Similar to instructional objectives . • Characteristics • Specific – SLOs should specify what students are to achieve. • Measurable – You should be able to measure the degree to which SLOs are met. • Achievable - Are the SLOs achievable and attainable? • Realistic – Can students realistically achieve the outcomes with available resources? • Time bounded – When are the students to achieve the outcomes?

5. SLOs SLOs are expressed and measured at the course, program, or institutional level. SLOs reflect a shift of focus from “What am I teaching” to “What are my students learning”?

6. Curriculum Alignment

7. Achieved Curriculum • Assessment for Learning • Formative • “In-progress” • Provides corrective feedback • Assessment of Learning • Summative • “After the fact” • High stakes

8. Taxonomy of the Cognitive Domain

9. Taxonomy of the Affective Domain

10. Taxonomy of the Psychomotor Domain

11. Significant Learning Categories • Learning how to learn: self-regulation, organizing time, data-driven decision making, etc. • Motivation/interest/values • Human dimension: acquiring professional behaviors, leadership, teamwork, etc. • Cognition • Integration/connection • Application/problem solving/critical thinking • Skills: using technology/communication, etc.

12. Course Syllabus • Course goals, instructional objectives, and/or SLOs must be consistent with: • University mission and goals • School mission and goals • Program goals and SLOs • National disciplinary norms • Assessments must be identified and linked to each instructional objective or SLO

13. Assessment Dressel, P. (1957) writes: "...a grade (is)...an inadequate report of an inaccurate judgment by a biased and variable judge of the extent to which a student has attained an undefined level of mastery of an unknown proportion of an indefinite amount of material." (p. 6.)

14. Assessment Anderson et al. (1975) provide the classical description of assessment: “Assessment, as opposed to simple one-dimensional measurement, is frequently described as multitrait-multimethod; that is, it focuses upon a number of variables judged to be important and utilizes a number of techniques to assay them – Its techniques may also be multisource … and/or multijudge” (p. 27).

15. Assessing SLOs • Focus is on assessing student: • Skills (psychomotor domain) • Knowledge (cognitive domain) • Attitudes (affective domain) as a result of the educational program. • Purpose: To obtain information that can be used by program faculty to answer the following questions: • Are our students learning what we think is important? • Are they learning what they need to succeed in this field or profession? • Are we improving in our ability to help students learn? • Should our curriculum or teaching strategies be modified? • Are there other techniques or additional resources that would help our students learn more effectively?

16. Sample SLO Assessments Direct Measures Indirect Measures Student surveys Graduate follow-ups Focus groups Exit interviews • Standardized test results • Performance on licensure or professional exams • Embedded assignments in specific courses • Internships • Performances • Portfolios of work samples • Job placement rates

17. Student Learning Assessment Cycle

18. Evidence Requirements • Identification of SLOs and how they are addressed for each program, e.g., curriculum maps • Assessment of each SLO • Efforts to: • improve student learning • make the program more effective based on these assessments

19. Simple Program Curriculum Map

20. Assessment Opportunities • Embedded assignments in specific courses • Traditional vs. authentic, application driven • Student performances • Student thesis/dissertation defenses • Internships • Program comprehensive examinations • Course evaluations (self, peer, student, & supervisor) • Standardized surveys • Focus groups

21. Putting Assessment into Practice at Regent • Assessment schedule will be advanced to address SACS concerns and start building the culture • New assessment matrices will be used • Executive summaries of results will be generated for reporting to school deans and Office of IE • Identify one high-stakes outcome for you program that would be a major issue if not achieved • Begin to think about program outcomes beyond SLOs • Job placement rates • Retention rates • Employer satisfaction • Etc.