Education 325: Assessment for Classroom Teaching

1. Education 325: Assessment for Classroom Teaching G. Galy, PhD Week 5

2. Let’s look at Evaluation: Teacher-Made Tests Evaluation: A judgment, the determination of worth Common evaluation tools: Exams, Tests, Quizzes

3. Teacher-made Tests

4. Valid and Reliable How do we make our tests valid and reliable?

5. Validity and Reliability

6. Validity: “Measuring what we set out to test.” “So that we can make sound judgments.” “Validity is never 100% achievable.”

7. Process Validity “Getting consistent results” Factors that can impact process validity include: • Directions • Vocabulary • Time • Difficulty • Clues • Scoring (unclear criteria) • Arrangement • Emotion/Feelings • Cheating

8. Content Validity “Alignment” Factors that can impact content validity: • Irregular ‘spot check’ • Gaps (untested) • Alignment (teach it = test it) • Appropriate value and weighting

9. Predictive Validity “Future Performance” • Minimal Competencies: Pass? Fail? (Driver’s License) • Pass: Mastery? 80% • Pattern of achievement and performance? • How confidently can you predict success for this student in the next grade? Next course?

10. Consequential Validity “Ability to accurately measure without influence” • Test anxiety • Competitive stress • Too much teaching to the test, rather than learning

11. Construct Validity “attitudes, values or beliefs” • Creativity • Team Work • Problem-Solving • Safety-mindedness • Higher- order thinking • Does your test REALLY measure this? • How do you measure such things? • How do you accurately measure a mental process?

12. Reliability “produces consistent results…even if given to the same group, a second time” Common characteristics: • Re-test reliability • Inter rater reliability

13. Writing the Test • Write Valid and Reliable Items • Arrange Items in clear, sequential order

14. Alternate Response • Bonus points can distort achievement. T F • Achievementis distorted by ____________. a) Bonus Points b) Avion Miles A B

15. Alternate Response Pro: Con: • Recall level • Not easy to make (No ambiguity) • Quick to complete • Unimportant details tend to be tested • Promotes ‘guessing’ • Not good with complex material

16. Alternate Response: Guidelines #1. Word statement clearly: Poor Example: “Achievement may be distorted by bonus marks.” Better Example: “Achievement is distorted by bonus marks.” Note: Broad generalizations are rarely wrong. CUE: ‘usually’, ‘generally’ ‘often’ ‘sometimes’ cue TRUE CUE: Absolutes ‘all’ ‘never’ ‘none’ ‘only’ ‘always’ cue FALSE

17. Alternate Response: Guidelines 2. Avoid testing trivial or pointless information: Each item should test important learning outcomes. Ask: Is the answer obvious? If so, then it will undermine the content validity of the test.

18. Alternate Response: Guidelines 3. Avoid the use of negative statements and double negatives. “Neither the bonding nor the adhesive surface should be handled with an ungloved hand. Bold, underline or italicize the word ‘not’. Double negatives (not, no, none, never, neither, nor) are confusing.

19. Alternate Response: Guidelines 4. Keep statements short. Avoid marathon statements. “Large bubbles in local anaestheticcarpules, with or without plungers that extend beyond the end of the carpule, may be formed from dissolved gases in the solution. These bubbles are harmless. T F

20. Alternate Response: Guidelines - 5. Avoid double barreled statements. “Parks and protected areas should be maintained for the use and benefit of all British Columbians.”

21. Alternate Response: Guidelines 6.Avoid statements that contain lists. If you do, you won’t be able to determine if they knew the answer or took a guess. “ Stockton, Jordan and Robinson are in the NBA Hall of Fame.”

22. Alternate Response: Guidelines 7. Avoid using trickery. Don’t insert insignificant detail to deceive the student. “The area of a rectangle 4 m x 3 m equals 12 square centimeters.” T F

23. Alternate Response: Guidelines 8. Have an equal number of True and False items. Why? Students have a tendency to guess True more often than False. Skewing the number will unduly favour or penalize a number of students.

24. Alternate Response: Guidelines 9. Keep True and False statements items similar in length. The longer statement is most often True. Largely because it contains additional detail and is more accurate. If possible, keep all items short and the same length.

25. “Fill in the Blank” or Short Answer Guidelines 1. Blank is located at the end of the sentence. 2. Only important words are omitted. 3. No more than two blanks per statement. 4. Each blank asks for a single idea.

26. “Fill in the Blank” Guidelines 5. Blanks must be the same length. 6. Only an ‘informed’ person should be able to complete the answer.