Welcome to Unit III Recognition Items

1. EDS 596 Assessment Techniques in Counseling Welcome to Unit III Recognition Items

2. Usual Disclaimer After viewing this presentation avoid operating mobile or dangerous equipment This presentation contains graphic images which some viewers may find disturbing. Avoid prolonged contact with skin Not to be taken internally No animals were harmed during the production of this presentation

4. Definition of Test Typically a test is used in an attempt assess or measure some quality or characteristic of an individual. Tests are used to judge, grade, or evaluation someone. “…a task or series of tasks used to collect a sample of human behavior assumed to be representative of a large domain.”

5. Tests Tests are composed of items; Each item is designed to elicit a response (simple or complex) The response is evaluated or compared to a standard and a judgment made (right-wrong/correct-incorrect) Tests are frequently used in the assessment of Achievement, Aptitude or Intelligence. Items typically fall into one of two large families of item forms Recognition and Supply

6. Recognition Items This family of items includes: The alternative response item The Matching Exercise and … The multiple choice/Interpretive Exercise Highly structured Restricted Response Freedom Objectively Scored

7. Some beginning thoughts….a test item represents a link in a chain and is used for one of two purposes…. 1. To assess “mastery” of some body of material or skill….or 2. To measure differences in mastery or levels of achievement among students/clients…..

8. As a link in the chain each item must have one characteristic. Ideally, a client or student who does not know the material should miss it and a student who knows the material or has mastered the material should be able to get it right! The item should discriminate.

9. Recognition Items…… • Advantages: • Permits excellent item sampling • Easy, quick, and objective scoring • Writing is minimized…Primitive Responses • Generally good for measuring simple outcomes… • No skill is needed for scoring • Quick return of scores and scored tests…

10. Recognition Items…… • Disadvantages: • Bright students may be penalized… • Guessing is always possible • Items may encourage rote learning….not always bad… • Sometime used to measure “trivial” material.

11. Recognition Items…… • General Guidelines • Test only for important ideas, information, & skills • Write items as simply and directly as possible • C. Include only ONE correct or best answer. • Sometime used to measure “trivial” material. • Avoid “Chaining” questions • Avoid specific determiners • Never “pull” questions or material from text. • Avoid “trick” questions • Always review items prior to printing test

12. The Alternative Response Item Form:A simple declarative sentence requiring a dichotomous judgment. • Advantages: • Excellent item sampling • Easy to write and score • Good for measuring “simple” outcomes. • Good item for clients with poor reading ability

13. The Alternative Response Item • Disadvantages: • Ease of successful guessing (50%) • Encourages rote learning…not always bad • Inability to measure complex outcomes • Limited to dichotomous material • Susceptible to ambiguity and misinterpretation • Least discriminating of all test item forms.. • May penalize bright and insightful students

14. Guidelines for Alternative Response Items • Insure that items are unequivocally true or false • Avoid the use of specific determiners • Avoid ambiguous, and indefinite terms of degree • Avoid negatively worded items.. • Avoid “double barreled items” • Keep T’s and F’s about the same length • Avoid detectable response patterns • Avoid trivial material

15. Consider this Question… True of False…The sun rose today. Most would agree that the answer is true….it got light this morning..... However, the more thoughtful among us may conclude that it is False…the sun only appears to rise and set as a function of the earth’s rotation on its axis, for example…..

16. Alternative Response Items Poor: T F Because of dramatically increased levels of CO2 emissions from industrialized countries there has been a reduction in the size of the polar ice caps. Better: T F Recently there has been a reduction in the size of the polar ice caps. This is a double barreled item!

17. Alternative Response Items True or False Acceleration of a body due to gravity is not 9,8 m/sec2 This item is worded in the negative...

18. Alternative Response Items True or False Weather and climate always have the same meaning…. This item contains a specific determiner, the word “always”

19. What to do about guessing • Guessing is possible anytime the correct answer is available for selection • A student gets the same point if he/she knows the answer or simply get lucky with a guess • Some students guess far more often than others • Guessing closes the range and makes differences between students harder to see. Any attempt to deal with guessing should be directed toward equalizing the impact of guessing on all students.

20. Two choices are available • Try to discourage guessing by imposing a penalty through the use of a score correction formula • Try to encourage guessing by telling students not to leave any question blank (Probably the best option)

21. The Score Correction Formula Tommy takes a 50 item, Alternative Response test, getting 42 correct. His score, correct for guessing would then be…..42-(8/2-1)….42-(8/1)….42-8=34 • That’s not so difficult, is it?

22. Confidence Weighting True or False? The Avco-Lycoming O-235-C1C produces 130 bhp at 2950 rpm. Please write down your answer….true or false… Now…tell me how confident you are of your answer….write down: 1 if your answer is just a guess 2 if you’re kind of certain and 3 if you’re absolutely sure…..

23. The Matching Exercise Form: Two columns of information the first of which is referred to as the list of items…the second column is referred to as the list of options. The task is to match the items with the appropriate options….This is the best type of item to use when trying to assess mastery of a paired-association.

24. Advantages of the Matching Exercise • Ease of construction • Excellent item sampling • Ease of scoring (if properly constructed) • Best item for paired-associations tasks

25. Disadvantages of the Matching Exercise • Encourages simplistic and rote learning…not always bad… • Basic inability to assess complex learning outcomes • Guessing is always possible but this can be dealt with through proper construction.

26. Writing a Matching Exercise • Items should be in the left column and options in the right. • Each exercise should consist of homogeneous material. • Keep each exercise relatively short (8-10 items). • Avoid having an equal number of items and options. • Arrange options in some systematic fashion is possible • Avoid specific determiners • Explain clearly in the directions the basis upon which the match is to be made. • Maintain grammatical consistency.

27. Match the famous compositions in Column I with the guys who wrote that kind of stuff in Column II. Each choice in Column II may be used one, or not at all. __ 1. Les Quaaludes A. Offenbach __2. Pictures of an Exhibitionist B. Rimsky-Korsakov __3. Marche Slob C. Chopin __4. Orpheus in His Underwear D. Orff __5. Le Coq Au Vin E. Liszt __6. Carmina Piranha F. Stravinsky __7. Peter and the Moose G. Debussy __8. Capriccio Escargot H. Prokofiev __9. A Midsummer Night on Elm Street I. Beethoven J. Tchaikovsky K. Rachmaninoff

28. Match the famous Operas in Column I with the guys who wrote that kind of stuff in Column II. Each choice in Column II may be used once, more than once, or not at all. __ 1. Rigatoni A. Offenbach __2. La Boingggg B. Rimsky-Korsakov __3. Madam Mothballs C. Chopin __4. Il Fettuccine D. Orff __5. The Cadillac de Saville E. Liszt __6. La Sagna __7. The Marriage of Orthogro __8. Das Fahrvergnugun __9. The Magic Galoot

29. Directions: On the line to the left of each achievement in column A, write the letter from column B that identifies the name of the man noted for that achievement. Each name from column B may be used once, more than once, or not at all. This is POOR PRACTICE Column A Column B __ 1. Invented the telephone A. Alexander Graham Bell __2. Discovered America B. John Glenn __3. First U.S. Astronaut to orbit the earth C. Abraham Lincoln __4. First President of the U.S. D. Ferdinand Magellan E. Christopher Columbus F. George Washington G. Eli Whitney

30. The Multiple Guess Item Form: Typically takes the form of a stem with four or five options only one of which is considered to be correct or the best answer. There are two forms….. Correct answer type: One correct among a number of incorrect ones….. Best answer type: Several correct responses which differ in DEGREE of correctness or completeness.

31. Multiple Guess Items Advantages: • Excellent item sampling • Ease of scoring • Versatile across a range of difficulty • Student writing is minimized • Permits writing tests which can re-used

32. Multiple Guess Items Disadvantages: • Time consuming to write….for each stem that is written a correct response must be created as well as three or four incorrect responses which appear as though they could be correct. • Students complain that more than one correct response

33. Multiple Guess Items Guidelines/Suggestions • Stem should be brief, clear, and concise • Stem should completely set the task • The options should be brief, clear, and concise • Distractors should be plausible • Avoid specific determiners • Correct responses should be randomly placed. • Use “all of the above” or “none of the above” only when appropriate and sparingly.

34. The two types of M/C Items The Correct Answer Type: • Which of the following continents lies totally above the equator? • Africa c. Asia • Austrailia d. South America

35. The two types of M/C Items The Best Answer Type: • If you found yourself on a desert island, which of the following would you need first to find? • Food • Water • Shelter • A “friendly” companion

36. Forms of M/C Stems • Which of the following is the best explanation for why most people wet the bed during earthquakes? Interrogatory • The principle reason for nocturnal enuresis in geologically active regions is: Incomplete sentence • Mark the answer which best describes the reason for bed wetting in volcanic areas. Imperative

37. Question Format Stem • Which one of the following expressions used by the airlines is an OXYMORON? • Complete stop D. Airline food • Direct flight E. Friendly Skies • Nonstop flight

38. A specific determiner… • A “BLEAM” is an: • lizard with 200 eyes, 100 for distance and 100 for reading. • animal with the body of a crab and the head of a certified public accountant. • a singles game played with dice and a large tube of ointment. • savage beast with the head of a lion and the body of a lion, though not the same lion.

39. Avoid specific determiners. • The largest African antelope is an: • Hippo • Dik-Dik • Eland • Warthog

40. Vary the position of the correct response • The largest African antelope is a/an: • Hippo A. Dik-Dik • Dik-Dik B. Eland • Eland C. Hippo • Warthog D. Warthog • By arranging the options in alphabetical order the correct response has in now in the “B” position.

41. Vary the position of the correct response • The Magna Carta was signed in what year? • 1965 A. 1215 • 1812 B. 1776 • 1215 C. 1812 • 1776 D. 1965 • By arranging the options in chronological order the correct response has in now in the “A” position.

42. Avoid Implausible Distrators • About how many calories are recommended daily for a girl, age 14, height 1.57m, weight 50kg, moderately active? • 0 A. 1,500 • 2,000 B. 2,000 • 2,500 C. 2,500 • 30,000 D. 3,000

43. Strive for Parallelism • The cell islets of the pancreas: • Contain ducts • Produce insulin • Disappear as one grows older • Are located around the margins of the pancreas This is poor form…..

44. Strive for Parallelism and let the stem set the task….. • The cell islets of the pancreas secrete the substance called: • adrenalin • insulin • secretin • trypsin This is much better!

45. Strive for Parallelism and let the stem set the task….. • South America: • Is a flat, arid, country • Imports coffee from the U.S. • Has a larger population than the U.S. • Was settled mainly by colonists from Spain

46. Strive for Parallelism and let the stem set the task….. • Most of South America was settled by colonists from: • England • France • Holland • Spain

47. It’s not wise to “chain” questions • Poor: • Scurvy is caused by a lack of vitamin: • A b. B1 c. B12 d. C 2. A good source of this vitamin is: a. Orange juice b. Cod-liver oil c. Whole rice d. liver

48. Avoid “Trick” Questions • Poor: • Which of the following substances is most essential for human life? • a. Protein b. Water c. Vitamins d. Minerals 1. Better: A healthy person is marooned on a deserted island. To survive, the person needs almost immediately to find a source of: a. Protein b. Water c. Vitamins d. Minerals

49. Check the Reading Level! • Poor: The promiscuous use of sprays, oils, and antiseptics in the nose during acute colds is a pernicious practice because it may have a deleterious on: • the sinuses. • red blood cells. • white blood cells. • the olfactory nerve.

50. Check the Reading Level! • Better: Frequent use of sprays, oils, and antiseptics in the nose during bad colds may result in: • Sinuses infections. • mucous membrane congestion. • harm to white blood cells. • olfactory nerve damage.