Assessment, Scoring, and Evaluation

1. A Pass/Fail 85% F B Assessment, Scoring, and Evaluation 93% C+ S A- 67% D GRADES Unsatisfactory C

2. We have an exam!!

3. Definitions • Assessment -- The process of measuring something with the purpose of assigning a numerical value. • Scoring -- The procedure of assigning a numerical value to assessment task.

4. Definitions • Evaluation -- The process of determining the worth of something in relation to established benchmarks using assessment information.

6. Reasons for Assessment • Student selection and certification • Instructional Monitoring • For: • program evaluation • Making decisions about different aspects of the educational process, if they provide accurate, authentic, reliable and valid information about educational LEARNING GOALS

7. Formative - for performance enhancement Formal - quizzes, tests, essays, lab reports, etc. Traditional - tests, quizzes, homework , lab reports, teacher Assessment Types

8. Summative - for performance assessment Informal- active questioning during and at end of class Alternative - presentations, essays, book reviews, peers Assessment Types

9. Formative Assessment • Assessment for learning • Taken at varying intervals throughout a course to provide information and feedback that will help improve • the quality of student learning • the quality of the course itself

10. Summative Assessment • Assessment oflearning • Generally taken by students at the end of a unit or semester to demonstrate the "sum" of what they have or have not learned. • Summative assessment methods are the most traditional way of evaluating student work.

11. Summative Assessment • Assessment oflearning • "Good summative assessments--tests and other graded evaluations--must be demonstrably reliable, valid, and free of bias" .

12. Formative ‘… often means no more than that the assessment is carried out frequently and is planned at the same time as teaching.’ (Black and Wiliam, 1999) ‘… provides feedback which leads to students recognizing the (learning) gap and closing it … it is forward looking …’ (Harlen, 1998) ‘ … includes both feedback and self-monitoring.’ (Sadler, 1989) ‘… is used essentially to feed back into the teaching and learning process.’ (Tunstall and Gipps, 1996) Summative ‘…assessment (that) has increasingly been used to sum up learning…’(Black and Wiliam, 1999) ‘… looks at past achievements … adds procedures or tests to existing work ... involves only marking and feedback grades to student … is separated from teaching … is carried out at intervals when achievement has to be summarized and reported.’ (Harlen, 1998)

14. Alternative Assessment • Alternative to what? Paper & pencil exams • Alternatives: • lab work / research projects • portfolios • presentations • research papers • essays • self-assessment / peer assessment • lab practical

15. More Formal Alternatives • Rube Goldberg projects • bridge building / rocketry / mousetrap cars • writing a computer program • research project • term paper • create web page • create movie • role playing • building models

16. Informal CATs (Classroom Assessment Techniques) • Quick-fire questions • Minute paper • 1) What did you learn today? • 2) What questions do you have? • Directed paraphrasing (explain a concept to a particular audience) • The “muddiest” point (What is it about the topic that remains unclear to you?)

17. Authentic Assessment • The National Science Education Standards draft (1994) states, "Authentic assessment exercises require students to apply scientific information and reasoning to situations like those they will encounter in the world outside the classroom as well as situations that approximate how scientists do their work."

18. Assessment Concerns • Validity -- Is the test assessing what’s intended? • Are test items based on stated objectives? • Are test items properly constructed? • Difficulty -- Are questions too easy or too hard? (e.g., 30% to 70% of students should answer a given item correctly)

19. Assessment Concerns • Discriminability -- Are the performance on individual test items positively correlated with overall student performances? (e.g., only best students do well on most difficult questions)

20. Difficulty Level

21. Question Types To evaluate students at Knowledge and Comprehension level - Multiple Choice - True or False - Fill-in-the Blanks

23. To evaluate students at comprehension and application level - Scenario-based questions - Simulation-based activities - Sequencing Other level 1 interactive questions – puzzle, crossword and any other creative question types (initially limited to 10 or 12 types including paragraph) Question Types

24. Define Test Generate Test Components from Test Bank Online Test(Use Case Diagram) Generate Test Main Program Source Code Teacher Build Version of Test Program Modify/Verify Students Test Information View Student Test information/ generate test information Modify, add, delete, test information

25. Allowed to re-enter answer if entered incorrectly Online Test(Use Case Diagram) View answer entered and the correct answer Print Test information Student Request for Test score modification Can Access / Take Online Test Student Can Verify His/Her Online Test Information

26. Evaluation is a ……….. • State of mind, rather than a set of techniques

27. Evaluation Types • Criterion-referenced evaluation -- student performance is assessed against a set of predetermined standards • Norm-referenced evaluation -- student performance is assessed relative to the other students • The “curve” -- sometimes a combination of criterion- and norm-referenced processes

28. Criterion-Referenced Eval’s • Based on a predetermined set of criteria. • For instance, • 90% and up = A • 80% to 89.99% = B • 70% to 79.99% = C • 60% to 69.99% = D • 59.99% and below = F

29. Pros: Sets minimum performance expectations. Demonstrate what students can and cannot do in relation to important content-area standards Criterion-Referenced Eval’s

30. Cons: Some times it’s hard to know just where to set boundary conditions. Lack of comparison data with other students and/or schools. Criterion-Referenced Eval’s

31. Norm-referenced Evaluation • Based upon the assumption of a standard normal (Gaussian) distribution with n > 30. • Employs the z score: • A = top 10% (z > +1.28) • B = next 20% (+0.53 < z < +1.28) • C = central 40% (-0.53 < z < +0.53) • D = next 20% (-1.28 < z < -0.53) • F = bottom 10% (z < -1.28)

32. Pros: Ensures a “spread” between top and bottom of the class for clear grade setting. Shows student performance relative to group. Con: In a group with great performance, some will be ensured an “F.” Norm-referenced Evaluation

33. Cons: Top and bottom performances can sometimes be very close. Dispenses with absolute criteria for performance. Being above average does not necessarily imply “A” performance. Norm-referenced Evaluation

34. Norm-Referenced: Ensures a competitive classroom atmosphere Assumes a standard normal distribution Small-group statistics a problem Assumes “this” class like all others Criterion-Referenced: Allows for a cooperative classroom atmosphere No assumptions about form of distribution Small-group statistics not a problem Difficult to know just where to set criteria Norm and Criterion Compared

36. 1- Oral examinations: Advantages 1. Provide direct personal contact with candidates. 2. Provide opportunity to take mitigating circumstances into account. 3. Provide flexibility in moving from candidate's strong points to weak areas. 4. Require the candidate to formulate his own replies without cues. 5. Provide opportunity to question the candidate about how he arrived at an answer. 6. Provide opportunity for simultaneous assessment by two examiners.

37. 1- Oral examinations Disadvantages 1. Lack standardization. 2. Lack objectivity and reproducibility of results. 3. Permit favoritism and possible abuse of the personal contact. 4. Suffer from undue influence of irrelevant factors. 5. Suffer from shortage of trained examiners to administer the examination. 6. Are excessively costly in terms of professional time in relation to the limited value of the information it yields.

38. 2- Practical examinations Advantages 1. Provide opportunity to test in realistic setting skills involving all the senses while the examiner observes and checks performance. 2. Provide opportunity to confront the candidate with problems he has not met before both in the laboratory and at the bedside, to test his investigative ability as opposed to his ability to apply ready-made "recipes". 3. Provide opportunity to observe and test attitudes and responsiveness to a complex situation (videotape recording). 4. Provide opportunity to test the ability to communicate under Pressure, to discriminate between important and trivial issues, to arrange the data in a final form.

39. 2- Practical examinations Disadvantages • 1. Lack standardized conditions in laboratory experiments using animals, in surveys in the community or in bedside examinations with patients of varying degrees of cooperativeness. • 2. Lack objectivity and suffer from intrusion or irrelevant factors. • 3. Are of limited feasibility for large groups. • 4. Entail difficulties in arranging for examiners to observe candidates demonstrating the skills to be tested.

40. 3- Essay examinations Advantages 1. Provide candidate with opportunity to demonstrate his knowledge and his ability to organize ideas and express them effectively Disadvantages 1. Limit severely the area of the student's total work that can be sampled. 2. Lack objectivity. 3. Provide little useful feedback. 4. Take a long time to score

41. 4- Multiple-choice questions Advantages 1. Ensure objectivity, reliability and validity; preparation of questions with colleagues provides constructive criticism. 2. Increase significantly the range and variety of facts that can be sampled in a given time. 3. Provide precise and unambiguous measurement of the higher intellectual processes. 4. Provide detailed feedback for both student and teachers. 5. Are easy and rapid to score.

42. 4- Multiple-choice questions Disadvantages 1. Take a long time to construct in order to avoid arbitrary and ambiguous questions. 2. Also require careful preparation to avoid preponderance of questions testing only recall. 3. Provide cues that do not exist in practice. 4. Are "costly" where number of students is small.

43. Fill in the Blank • Fill in the blank questions are distinguished by the text box located in within the question text. • Test-takers are only allowed to type one answer for each text box.

44. Match • Matching questions are distinguished by the drop down menus located underneath each question section. Test-takers are only allowed to select one option for each question.

45. Open-Book exam • pen book exams are not an easy option It is important to know what type of information and how much will be allowed into the examination room. • Answers cannot just copy directly from the textbook

46. Examples: Online Self-Assessment • Self-tests: Use test tool to create self-tests (multiple choice, true false) • May wish to track student efforts • Can incentivize use (essentially, use as a learning tool) • Reflection papers: Have students submit brief, focused papers expressing the strengths and assessments of their assignment(s)

47. Private Work is submitted to the teacher only Entire burden of feedback is on teacher Important if assessing at fact level Public Peers can see each others’ work (either in process or completed) Peers may comment on each others’ work Often increases quality of work submitted Public vs. Private