Assessment and evaluation

1. Assessment and evaluation Prof. Namir G. Al-Tawil Hawler Medical University College of Medicine namiraltawil@yahoo.com

2. Contents • Objectives • Definitions of assessment and evaluation • Types of assessment • Formative assessment • Summative assessment • Assessment of students’ knowledge • Assessment of students’ performance

3. Objectives • At the end of the lecture, the audience must be able to know: • The differences between formative and summative assessment. • How to assess students’ knowledge. • How to assess students’ performance.

7. Definitions • Assessment • It is the process of data gathering by instructors about their teaching and their students’ learning. • Pre-test, observations, examinations are forms of assessment. • Evaluation • It is the decision making process based on the assessment data.

8. Types of assessments • Diagnostic assessment • Formative assessment • Summative assessment

9. Diagnostic assessment • Helps to identify students’ current knowledge, skills, and capabilities; and to clarify misconception before teaching takes place. • Types: • Pre-tests • Self assessment • Interview

10. Formative assessment • Provides feedback during a teaching process. • It assesses student’s and teacher’s progress. • These assessments are typically not graded. • E.g: if after 2 weeks of the start of the course, you made a test (by clickers for e.g) and you discovered that the majority didn’t answer correctly, then you have to repeat this point, and to correct yourself in the next course

11. Types of formative assessment • Observations during in-class activities. • Homework exercises. • Question and answer sessions (planned and spontaneous). • Conferences between instructor and students. • Student feedback about their instructors.

12. Summative assessment • Takes place at the end of learning process. • It assesses the final product. • Grades are usually an outcome of it. • To decide whether the student is eligible for the next step or not.

13. Types of summative assessments • A mid-term exam. • A final exam. • A final project. • A paper.

14. Criteria of the assessment tool • Validity • Reliability • Educational impact, on teaching and learning • Acceptability to staff, students, and others • Cost A variety of assessment methods are needed to test competency

15. “When the cook tastes the soup, that’s formative evaluation. When the guests taste the soup, that’s summative evaluation” Bob Stake 1991

16. Assessment of knowledge and its application • The most common method for knowledge assessment is the written method. • Several formats are available like MCQs and essays…. • The content of the question is more important than the format. • A variety of formats are preferable.

17. Questions formatsMCQs (A-Type: one best answer) • Single choice questions are more preferred than the true/false (multiple response item). • Anatomy of the item (question): • Stem • Vignette (e.g case study) • Lead-in question (what is the most likely Dx?) • Options • Correct answers • Distracters

18. Tips for designing MCQs • Use Plausible Distracters (wrong-response options) • Use a Question Format (rather than incomplete sentence format) • Keep Option Lengths Similar • Balance the Placement of the Correct Answer • Be Grammatically Correct

19. Avoid Clues to the Correct Answer • Avoid Negative Questions • Use Only One Correct Option • Give Clear Instructions • Avoid the "All the Above" Option • Avoid the "None of the Above" Option • Don't Use MCQs When Other Item Types Are More Appropriate

20. MCQs (R-type: Extended matching items) • Extended matching questions consist of lettered options followed by a list of numbered problems/questions. For each numbered problem/question select the one lettered option that most closely answers the question. You can use the lettered options once, more than once, or not at all.

21. Key features questions • They are short clinical cases or scenarios which are followed by questions aimed at key features or essential decisions of the case. • The questions could be MCQs or open ended questions. • More than one correct answer could be provided.

22. Short answers questions • Open ended questions. • Their answer is usually 1 or 2 words. • Usually used when closed formats are not possible. • Each teacher should correct the same question for all the students. So more reliable scores.

23. Essay questions • Requires more time to answer. • Less reliable scoring. • Using correction scheme (checklist) can improve reliability.

24. Modified essay questions • Is a special type of essay question that consists of a case followed by a series of questions that relate to the case and that must be answered in the sequence asked. • A student answering the first question incorrectly is likely to answer the subsequent questions incorrectly too.

25. Assessment of performance • Categorized into: • In vitro assessment (using simulation) • In vivo assessment (real condition) • In both, the student make a skill or behavior and observed by the examiner.

26. Assessment of performance, cont • Many methods like check-lists and rating scales can be used to report observation. • OSCE and OSPE • Direct observation of procedural skills (DOPS) • Peer assessment • Self-assessment

27. Methods for assessing performance • Checklists • E.g OSCE • Rating scales • Problem of subjectivity and low reliability • Use more than one rater (observer) • OSCE • Short cases to assess clinical competence

28. Long cases (to assess clinical competence) • No more used in many colleges • Of help for formative examinations • Modification done in many places to improve reliability • 360 degrees evaluation • Of help in formative assessment • Multiple evaluators like supervisors, peers, students, administrative staff, and patients. • Time consuming and not practical.

29. Mini Clinical Evaluation Exercises (MINI-CEX) • Of help in formative assessment • They are relatively short observations (15-20 minutes) in which performance isrecorded on a 4 point scale where 1 is unacceptable, 2 is below expectation, 3 is met expectations, and 4 isexceeded expectations. • Several competencies are evaluated: history taking, physical examination, clinicaljudgment, counseling, professionalism and other generic qualities

30. Portfolio • A portfolio is a collection of student work which provides evidence that learning has taken place. • Mainly for formative assessment.

31. Questions • ? • ? • ?

32. Conclusion • Various assessment methods that test a range of competencies are available for examiners • The choice depends on purpose of examination, i.e. either summative, formative or both, in addition to other factors.

33. Thanks for listening