Completion, Short-Answer, and True-False Items

1. Completion, Short-Answer, and True-False Items Educational Assessment of Students Chapter 7 Anthony J. Nitko (Brenda Stewart)

2. Three Fundamental Principles for Crafting Assessments • Focus each assessment task entirely on important learning targets • Craft each assessment task to elicit from students only the knowledge and performance that are relevant to the learning targets you are assessing • Craft each assessment task to neither prevent nor inhibit a student’s ability to demonstrate attainment of the leaning targets you are assessing

3. Short-Answer Items • Require a word, short phrase, number, or symbol response. • Three types of short-answer items: • Question – What is the capital of Texas? • Completion – The capital of Texas is _________. • Association – On the blank next to the state name, write the abbreviation for that state. • Texas TX • California CA

4. Usefulness of Short-Answer Items • Abilities Assessed • Lower-order thinking skills • Recall • Comprehension of information • Higher-level abilities • Make simple interpretations of data and applications of rules • Ability to solve numerical problems in science and mathematics • Ability to manipulate mathematical symbols and balance mathematical and chemical equations

5. Usefulness of Short-Answer Items • Strengths • Relatively easy to construct • Can be scored objectively • Lowers the probability of getting the answer correct by random guessing • Shortcomings • Not free of subjectivity • Cannot anticipate all possible responses • Tends to lower the reliability of the obtained score

6. Crafting Short-Answer Items • Importance of what is assessed • How does it match the test blueprint • Question format is the preferred format for a short-answer item. Focus the item on the specific knowledge sought • Word the items specifically and clearly • Put the blank near the end of the sentence • If possible do not copy statements verbatim

7. Crafting Short-Answer Items • A completion item should omit important words and not trivial words • Limit blanks to one or two • Keep all blanks the same length • Specify the precision you expect in your answer • Avoid irrelevant clues

8. True-False Items • Consists of a statement or proposition the student may judge as true or false • Six varieties: • True-False • Yes-No • Right-Wrong • Correction • Multiple True-False • Yes-No with explanation

9. Usefulness of True-False Items • Advantages • Certain aspects of the subject matter readily lend themselves to verbal propositions that can be judged true or false • Relatively easy to write • Scored easily and objectively • Can cover a wide range of content within a relatively short period • Criticisms • Poorly constructed assess only specific, frequently trivial facts • Ambiguously worded • Answered by random guessing • Encourage students to study and accept only oversimplified statements of truth and factual details

10. Usefulness of True-False Items • Generalizations in a subject area • Comparisons among concepts • Casual or conditional propositions • Relationships between two events, concepts, facts, or principles • Explanations for why events or phenomena occurred • Instances or examples of a concept or principle

11. Usefulness of True-False Items • Evidential statements • Predictions about phenomena or events • Steps in a procedure of process • Computations (or other kinds of results obtained from applying a procedure) • Evaluations of events of phenomena

12. Validity of True-False Item Format • Essence of educational achievement is the command of useful verbal knowledge • All verbal knowledge can be expressed in propositions • Proposition is any sentence that can be said to be true or false • Particular area of knowledge is indicated by success in judging truth or falsity of propositions

13. Guessing on True-False Items:An Ill Wind? • Guessing – getting 50% correct • Blind (complete random guessing) quite unlike informed guessing (based on partial knowledge) • Well-motivated guess blindly only on a small percentage of questions on a test • Difficult to obtain a good score on a test by blind guessing only • A given true-false test has a high reliability coefficient – evidence blind guessing does not seriously affect the scores

14. Guessing on True-False Items:An Ill Wind? • Random guessing – sometimes called blind guessing • Partial knowledge – make an informed guess • Higher than 50-50 chance of success • Measures of partial knowledge • Full knowledge – 100% chance of answering correctly • Percentages of correct answers go down with the number of blind guesses made

15. Suggestions for Getting Started Properly • Create pairs of items, one true and one false • Rephrase evaluative judgments into a comparative statement • Write false statements that reflect the actual misconceptions held by students • Convert a multiple-choice item into two or more true-false items

16. Suggestions for ImprovingTrue-False Items • Importance of what is assessed • How does it match the test blueprint • Assess important ideas, rather than trivia, general knowledge, or common sense • Make sure it is either definitely true or definitely false • Use short statements when possible • Use exact language • Use positive statements and avoid double negatives

17. Suggestions for ImprovingTrue-False Items • Avoid copying sentence verbatim • True and false statements should have approximately the same number of words • Don’t present in a repetitive or easily learned pattern • Do not use verbal clues that give away the answer • Attribute the opinion in a statement to an appropriate source • Focus on one idea

18. Crafting Multiple True-False Items • Looks like a multiple-choice item – followed by several alternatives • No single correct or best answer • Response to each alternative is true or false • Each alternative is scored correct or incorrect

19. Crafting Multiple True-False Items • Advantages • Two or three multiple true-false responses in the time it takes to make one multiple-choice response • Multiple true-false tests created from multiple-choice items have higher reliability than original multiple-choice • Can assess same abilities as straight multiple-choice items that are crafted to assess parallel content

20. Crafting Multiple True-False Items • Advantages • Students believe multiple true-false items do a better job of assessing knowledge than multiple-choice items • Students perceive them to be harder • May be easier to write – not limited to one correct answer • Limitations • Shares some of same limitations as multiple-choice

21. Works Cited • Nitko, A. J., (2004). Educational Assessment of Students, 4th ed. Upper Saddle River, NJ: Pearson Education, Inc.