1 / 47

Question Development

How do I create good multiple choice items?. Question Development. Agenda. Black Box Why we are here. Black Box. Back to the Classroom. Black Box. What happens inside…. Black Box . Does Formative Assessment raise standards?

aleta
Download Presentation

Question Development

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. How do I create good multiple choice items? Question Development

  2. Agenda • Black Box • Why we are here

  3. Black Box

  4. Back to the Classroom

  5. Black Box What happens inside…

  6. Black Box Does Formative Assessment raise standards? (I prefer “achievement” to standards.) 1. enhances communication between student and teacher. 2. actively involves students 3. yields results that are then used to adapt teaching 4. affects the motivation and self-esteem of students

  7. Black Box Is there room for improvement in the way we assess? Issue #1 cultivate understanding, not rote learning; work collaboratively with other teachers; emphasize quality, not quantity. Issue #2 emphasize learning, not grading; encourage each student’s personal improvement; convey your belief that students can learn Issue #3 analyze student work to discern their learning needs; make feedback meaningful

  8. Black Box How can we improve formative assessment? • Feedback to students should be about the particular qualities of their work. • Students should be trained in self-assessment so that they can understand the main purposes of their learning and what they need to do to achieve. • Effective teaching includes opportunities for students to express their understanding.

  9. Black Box So…if I am in the black box, how do I respond? 1. I learn from professional development. 2. I start in low gear and patiently find my own way of incorporating new learning. 3. While remaining aware of external tests, I take leading responsibility for assessing ongoing work to monitor and improve progress. 4. Continually reflect on my work.

  10. Research • Black & WiliamResearch Questions: • Does better formative assessment equal higher test scores? • Does formative assessment need improving? • If so, what improvement is needed?

  11. Research Findings

  12. Why are we here? • What’s coming with Common Core Curriculum? • EOCT vs. GHSGT • Continuous Improvement

  13. Our Element • The teacher uses a variety of summative assessment strategies to evaluate student status relative to mastery of the Georgia Performance Standard. • EQ? How can well designed assessments evaluate student performance on the GPS and provide information about student strengths and weaknesses?

  14. Determine and define vocabulary • Identify key terms within the standard(s)/element(s). Define each term. • In your group: • Underline key terms in the element. • Define key terms The teacher uses a variety of summative assessment strategies to evaluate student status relative to mastery of the Georgia Performance Standard.

  15. Study the standard(s)/element(s) • Generate key questions related to the standard(s) and/or element(s). • Answer each question. The teacher uses a variety of summative assessment strategies to evaluate student status relative to mastery of the Georgia Performance Standard.

  16. Identify strong and weak work • Collect and rate strong and weak samples of work related to the standard(s) and/or element(s). • List examples of strong and weak work • Use information on handout to help The teacher uses a variety of summative assessment strategies to evaluate student status relative to mastery of the Georgia Performance Standard.

  17. Scaffold understanding of the standard(s) and/or element(s) • Paraphrase the standard(s) and/or element(s). The teacher uses a variety of summative assessment strategies to evaluate student status relative to mastery of the Georgia Performance Standard.

  18. Develop “I Can” statements • Describe the standard(s) and/or element(s) as statements of intended learning (e.g., I can use information from what I read to draw conclusions (make inferences), I can use mathematical vocabulary to describe how I solved a problem, etc.). The teacher uses a variety of summative assessment strategies to evaluate student status relative to mastery of the Georgia Performance Standard.

  19. Depth of Knowledge (DOK) • Jigsaw Activity • Count off at your table 1, 2, 3, 4, 1, 2, … • All 1’s meet; All 2’s meet; All 3’s meet; All 4’s meet • In “Expert Groups” read and discuss everything about your DOK Level • Return to your “Home Group” • Experts report out to the “Home Group”

  20. Common Verbs

  21. Rules are made for breaking? • Basic rules for creating multiple choice items • Reduce systematic errors • Increase reliability of test items • Basically – design the questions to determine what the student really knows or does not know… • Not whether or not the student has “test taking skills”

  22. “Misguided” Test Taking Tips • Guess if you are not sure • Look for answer choices that grammatically complete the question • If one answer is much longer than the others – it is most likely the correct answer • If “All of the Above” is provided and you are reasonably sure of two answer choices , “All of the Above” is most likely correct.

  23. Multiple Choice Items • Who was president during World War II? • Franklin Roosevelt • George Washington • William Clinton • Ronald Reagan Key Item Stem Distracters

  24. Rule #1 • Make the problem clear to the student Poor Example: 1. Antigens A. Attack memory cells B. Attack killer T Cells C. Attack pathogens D. Attack helper T cells

  25. Rule #1 Better Example: • Which of these do antigens attack? A. Pathogens B. Helper T cells C. Killer T cells D. Memory cells

  26. Rule #2 • State the item stem in the positive whenever possible. Poor Example: 1. Which of these is not a reason why President Roosevelt decided not to involve the United States in World War II until the bombing of Pearl Harbor? A. The US was upset with Great Britain over the sinking of merchant ships. B. The US was involved in another war at the time. C. The US was already allied with Germany at the time of Pearl Harbor. D. The US had not been attacked directly until Pearl Harbor.

  27. Rule #2 Better Example 1. Which of these best describes why the United States became involved militarily in World War II? A. The US was directly attacked. B. Great Britain asked for assistance against German submarines. C. French resistance needed military arms after Germany invaded. D. The US was obligated to become involved by its membership in NATO

  28. Rule #3 • Make sure the item stem does not give away the correct answer. Poor Example: • What led to the formation of the States' Rights Party? A. The level of federal taxation B. The demand of states for the right to make their own laws C. The industrialization of the South D. The corruption of federal legislators on the issue of state taxation

  29. Rule #3 Better Example: • What led to the formation of the States' Rights Party? A. The level of federal taxation B. The demand by states to make their own laws C. The industrialization of the South D. The corruption of federal legislators on the issue of state taxation

  30. Rule #4 • Emphasize qualifiers such as most likely and best in the stem. Poor Example: 1. Which measurement unit below would be best to use if you wanted to measure the mass of a bumblebee? A. gram B. kilogram C. millimeter D. centimeter

  31. Rule #4 Better Example • Which measurement unit below would be best to use if you wanted to measure the mass of a bumblebee? A. gram B. kilogram C. millimeter D. centimeter

  32. Rule #5 • Make sure the answer choices are plausible. Poor Example: 1. Who was president during World War II? A. Franklin Roosevelt B. George Washington C. George W. Bush D. Jesse James

  33. Rule #5 Better Example • Who was president during World War II? A. Dwight Eisenhower B. Lyndon Johnson C. Franklin Roosevelt D. Woodrow Wilson

  34. Rule #6 • Develop answer choices that are parallel in grammar and in length. Poor Example: • In English class, Cynthia must write a descriptive essay about the person she admires the most. She wants to write about her father. Which of these would be the best way she could begin to write her essay? A. Look through old picture of her and her father B. She could call her best friend to talk about him C. By making a list of activities she has done with her father. D. Asking her father to come to her class

  35. Rule #6 Better Example: 1. In English class, Cynthia must write a descriptive essay about the person she admires the most. She wants to write about her father. Which of these would be the best way she could begin to write her essay? A. By looking at her baby pictures B. By asking her father to come to her class C. By calling her best friend to talk about her father D. By listing all of the things he has done for her and others

  36. Rule #7 • Avoid using “All of the above” and “None of the above.” Poor Example: • Which of these should be used to measure in millimeters? A. A yard stick B. A metric ruler C. A digital scale D. None of the above

  37. Rule #7 Better Example: • Which of these should be used to measure in millimeters? A. A yard stick B. A metric ruler C. A digital scale D. A graduated cylinder

  38. Rule #8 • Place the answer choices in logical order. • Numbers in numerical order • One-word answer choices in alphabetical order • Order sentences from shortest to longest Poor Example: • A teacher is planning a field trip and will need school buses to transport students A school bus holds 36 students. If 1,128 students will be transported, how many buses are needed? A. 32 B. 31.33 C. 36 D. 31

  39. Rule #8 Better Example • A teacher is planning a field trip and will need school buses to transport students A school bus holds 36 students. If 1,128 students will be transported, how many buses are needed? A. 31 B. 31.33 C. 32 D. 32.33

  40. Rule #9 • Avoid clues in the answer choices that give away the correct answer. Poor Example • Which of the following revolutions resulted in increased business by speeding travel? A. industrial B. technology C. textile D. transportation

  41. Rule #9 Better Example • Which of the following best describes a result of the transportation revolution in the 1800s? A. Suburban areas grew B. Demands for slave labor decreased C. Manufacturing and production increased D. People relied less on mass transportation

  42. Rule #10 • Make sure that the correct response is the only correct response. Poor example • Chicago Bears: football:: _________:baseball A. Anaheim Angels B. Los Angeles Galaxy C. Atlanta Braves D. Tampa Bay Buccaneers

  43. Rule #10 Better example • large: big :: triumph : ___________ A. loss B. rule C. small D. success Seinfeld teaches History - Cooperative Learning

  44. The Work • Look at what you will be teaching from Jan. 6th to Feb. 1st • Focus on questions that address the standards • Assign questions: • DOK Tag • Standard/Element Tag • Create a bank of 30-35 questions (80% at DOK 2 or 3)

  45. Question Format 1. Indicate DOK and Standard/Element 2. Leave Column Blank 3. Leave Row Blank

  46. Resources • www.edinformatics.com • Released questions from standardized state tests across the nation • http://vhsprofessionallearning.wikispaces.com/ • PowerPoint • 5 Step Protocol • Template for Question Banks

More Related