Plan

1. Plan Implement Improve Assess

2. Outcomes Assessment The process of providing credible evidence of the outcomes of higher education undertaken for the purpose of improving programs and services within the institution. Banta, T. W.

3. ASSESSMENT . . . “a rich conversation about student learning informed by data.” -- Ted Marchese -- AAHE

4. Assessment of Individual Student Development • Assessment of basic skills for use in advising • Placement • Counseling • Periodic review of performance with detailed feedback • End-of-program certification of competence • Licensing exams • External examiners • CLAST

5. Key Results of Individual Assessment • Faculty can assign grades • Students learn their own strengths and weaknesses • Students become self-assessors

6. A Second Look • Across students • Across sections • Across courses

7. Where is learning satisfactory? • What needs to be retaught? • Which approaches produce the most learning for which students?

8. Group Assessment Activities • Classroom assignments, test, projects • Questionnaires for students, graduates, employers • Interviews, focus groups • Program completion and placement • Awards/recognition for graduates • Monitoring of success in graduate school • Monitoring of success on the job

9. Use of Results of Group Assessment • Program improvement • Institutional and / or state peer review • Regional and / or national accreditation

10. Some Purposes of Assessment 1. Students learn content 2. Students assess own strengths 3. Faculty improve instruction 4. Institutions improve programs/services 5. Institutions demonstrate accountability

11. Outcomes Assessment Requires Collaboration • In setting expected program outcomes • In developing sequence of learning experiences (curriculum) • In choosing measures • In interpreting assessment findings • In making responsive improvements

12. Barriers to Collaboration in the Academy • Graduate schools prepare specialists • Departments hire specialists • Much of our scholarship is conducted alone • Promotion and tenure favor individual achievements -- interdisciplinary work is harder to evaluate

13. Campus Interest in Assessment WHAT WORKS in…. • increasing student retention? • general education? • use of technology in instruction? • curriculum in the major?

14. Good assessment is good research . . . • An important question • An approach to answer the question • Data collection • Analysis • Report -Gary R. Pike (2000)

15. To Foster Collaboration • Name interdisciplinary committees • Read and discuss current literature on learning/assessment • Attend conferences together • Bring experts to campus • Share good practices • Work together on learning communities

16. Most Faculty Are Not Trained as TeachersFACULTY DEVELOPMENT Can Help Instructors: • Write clear objectives for student learning in courses and curricula • Individualize instruction using a variety of methods and materials • Ask questions that make students activelearners • Develop assessment tools that test higher order intellectual skills

17. Cognitive domain categories Knowledge Comprehension Application Analysis Synthesis Evaluation Sample verbs for outcomes Identifies, defines, describes Explains, summarizes, classifies Demonstrates, computes, solves Differentiates, diagrams, estimates Creates, formulates, revises Criticizes, compares, concludes Taxonomy of Educational Objectives(Bloom and Others, 1956)

18. Organizing for Assessment

19. Some General Education Objectives • Differentiate between fact and opinion • Gather, analyze, and interpret data • Apply ethical principles to local, national, global issues • Communicate ideas in writing effectively

20. Learning Outcomes in Science • Define and explain basic principles, concepts, theories of science • Identify characteristics that distinguish math and science from each other and from other ways of obtaining knowledge • Illustrate how developments in science can raise ethical issues • Solve theoretical or experimental problems in science • Evaluate the validity and limitations of theories and scientific claims in interpreting experimental results • Evaluate scientific arguments at a level encountered by informed citizens

21. Critical Assessment Questions 1. What should a major know and be able to do? 2. What curriculum experiences promote student attainment of This knowledge? These skills? 3. Are these experiences taking place? 4. How do we know students are attaining The knowledge? The skills?

22. Planning for Learning and Assessment

23. Some Assessment History 1970 – Alverno NE Missouri 1979 – Tennessee 1985 – VA, NJ, CO 1998 – HE Amendments - Accreditors

24. Purposes for Assessment • Accountability to satisfy external stakeholders • Improvement to make things better internally

25. Some external impetus is necessary to initiate outcomes assessment in higher education.

26. Organizational Levels for Assessment National Regional State Campus College Discipline Classroom Student

27. Licensing/Certification Tests • National Teacher Exam • Commons and specialty areas • Engineer in Training Exam • NCLEX in Nursing • CPA exam in Accounting • Bar exam in Law • NCARB exam in Architecture • Board exams in Medicine, Social Work, Planning

28. Major Field Achievement Tests from Educational Testing ServicePrinceton, New Jersey Biology Chemistry Computer Science Economics Education Engineering Geology History Literature in English Mathematics Music Physics Political Science Psychology Sociology

29. Definitions and Assessment Methods for Critical Thinking, Problem Solving, and Writing By T. Dary Erwin James Madison University for the National Postsecondary Education Cooperative (U.S. Dept. of Education, National Center for Education Statistics) Student Outcomes Pilot Cognitive Working Group Washington, DC 1998 Website: nces02.ed.gov/evaltests

30. Are Standardized Tests the Answer? • Not available in many fields • Do not measure all that is taught • Usually assess knowledge, not performance • May be standardized on unrepresentative norm group • Provide few, if any, subscores • Do not indicate why scores are low

31. Start with Measures You Have • Assignments in courses • Course exams • Work performance • Records of progress through the curriculum

32. Primary Trait Scoring Assigns scores to attributes (traits) of a task STEPS • Identify traits necessary for success in assignment • Compose scale or rubric giving clear definition to each point • Grade using the rubric

33. Narrows and defines topic Produces bibliography Develops outline Produces first draft Produces final draft Presents oral defense Can Develop a Research Paper

34. Bibliography Outstanding – References current, appropriately cited, representative, relevant Acceptable – References mostly current, few citation errors, coverage adequate, mostly relevant Unacceptable – No references or containing many errors in citation format, inadequate coverage or irrelevant

35. Mapping Course Outcomes to Program Outcomes

36. Score 3 2 1 0 Criterion Clear conceptual understanding, consistent notation, logical formulation, complete solution Adequate understanding, careless errors, some logic missing, incomplete solution Inadequate understanding, procedural errors, logical steps missing, poor or no response Problem not attempted or conceptual understanding totally lacking Ball State University Sophomore Competence in Mathematics(Multiple choice responses & supporting work)

37. Assessment in Sociology and Anthropology Focus groups of graduating students • Given a scenario appropriate to the discipline, a faculty facilitator asks questions related to outcomes faculty have identified in 3 areas: concepts, theory, methods. • 2 faculty observers use 0-3 scale to rate each student on each question • GROUP scores are discussed by all faculty • Murphy & Goreham North Dakota State University

38. Entries accurately and vividly record objective observations of site experiences (events, people, actions, setting) Entries convincingly record subjective responses to site experience (thoughts, emotions, values, judgments) Entries effectively analyze/ evaluate your experiences (find insights, patterns, meaning, causes, effects) Journal Evaluation

39. Direct Measures of Learning Assignments, exams, projects, papers Indirect Measures Questionnaires, inventories, interviews - Did the course cover these objectives? - How much did your knowledge increase? - Did the teaching method(s) help you learn? - Did the assignments help you learn?

40. Fast Feedback(at end of every class) • Most important thing learned • Muddiest point • Helpfulness of advance reading assignments for day’s work in class • Suggestions for improving class / assignments Bateman & Roberts Graduate School of Business University of Chicago

41. Student Suggestions for Improvement • Install a portable microphone • Increase type size on transparencies • Leave lights on when using projector • Don’t cover assigned reading in detail • Provide more examples in class

42. Computer and information technology Course learning Writing experience Experience with faculty Art, music, theater Campus facilities Clubs and organizations Personal experiences Student acquaintances Scientific and quantitative experiences Conversations The college environment Estimate of gains College Student Experience Questionnaire(4th Edition)SCALES

43. College Student Experiences Questionnaire (sample item)Library Experience • used library as quiet place to study • used online catalogue • asked librarian for help • read reserve book • used indexes to journal articles • developed bibliography • found interesting material by browsing • looked for further references cited • used specialized bibliographies • read document other authors cited

44. Self-Reports • How time is spent • Studying (IUB) • In all activities (Miami) • Social interactions • Diaries, journals • Portfolios

45. Assessing Student GrowthThe Portfolio - Some Examples of Content • Course assignments • Research papers • Materials from group projects • Artistic productions • Self-reflective essays (self-assessment) • Correspondence • Taped presentations

46. Student Electronic Portfolio • Students take responsibility for demonstrating core skills • Unique individual skills and achievements can be emphasized • Multi-media opportunities extend possibilities • Metacognitive thinking is enhanced through reflection on contents - Sharon J. Hamilton IUPUI

47. Using Electronic Assessment Methods We can • Track progress in assignments • Evaluate contributions to group projects • Conduct immediate process checks to evaluate instruction • Assess the quality of written work

48. Faculty-Developed Exam in Religious Studies Components: • Identification of topic for comprehensive paper in senior seminar • faculty critique • Development of bibliography for paper • faculty critique • Development of outline for paper • faculty critique • Writing of first draft of paper • faculty critique • Writing of final paper • faculty critique according to set of guidelines • critique by external consultants using same guidelines University of Tennessee, Knoxville

49. Authentic Assessmentat Southern Illinois University - Edwardsville • Business - Case Study Analysis with Memo • Education - Professional Portfolio • Psychology - Poster on Research Project • Engineering - Senior Design Project • Nursing - Plan of Care for Patient

50. Responses to Assessmentat Southern Illinois University - Edwardsville Business - More case studies and research • Education - More practice in classroom management • Psychology - Curriculum change in statistics • Engineering - More practice in writing and speaking • Nursing - Simulation lab with computerized patients