Item Writing Training

1. Item Writing Training

2. Two Types of Items Selected response (multiple choice) Item with stem and several possible choices WCSD and NDE offer 4 choices (1 correct and 3 distracters) Wide sampling and broad coverage of content Reliable and efficient to score Constructed response (written response) Requires construction of a written response Allows demonstration of knowledge depth

3. Selected Response Formats Question Which is the best new title for the story? Complete the sentence According to the passage, John was known as Fill in the blank _____ friend did not meet them _____. (their, there) Remind that cloze type item format is only in 3rd grade.Remind that cloze type item format is only in 3rd grade.

5. Selected Response Practice Pick a prioritized standard Read passage Write/choose an item with four possible choices in the testing materials you have Revise and edit the item to best meet the assessment need, using the matrix Discuss

6. Constructed Response Formats Provides an introduction This article tells about the lives of some Korean students. Includes directions for the task A. Describe two ways that the lives of Korean and American students are alike. B. Describe two ways that the lives of Korean and American students are different.

7. Item Guidelines Content Knowledge or skills being tested based on district/state standards Language Actual words used in prompt, stem, and choices Structure How item stem and choices are presented and organized Bias/Sensitivity Fair and inoffensive to all students

8. Item must be aligned to indicator that is being assessed Example: ELA Standard 3.5.2 Make inferences supported by the text about characters� traits and motivations and make predictions about conflicts and resolutions The puppy�s name was Max chose the brown puppy because it was Content Is not aligned to standard 2 is aligned Is not aligned to standard 2 is aligned

9. Content Relevancy Only important skills and knowledge are assessed No trivial items Focus on higher-order thinking skills Example: What is the color of the bus? Which person is the main character? Low level and trivial Better Low level and trivial Better

10. Grade-level Appropriate Content* Items should reflect indicators at appropriate grade level Re-check adjoining grade level indicators to ensure proper grade level item 4.5.5 Identify authors� ideas and purposes in texts, including advertisements and public documents 4.6.5 Evaluate how authors� ideas and purposes shape the content of texts, such as advertisements and public documents

11. Grade-level Appropriate Content* Verb difference in indicator makes a difference. Provide an advertisement for them to write to each in teams of two. Verb difference in indicator makes a difference. Provide an advertisement for them to write to each in teams of two.

12. Clearly Formulated Item Stem Single, clear, complete problem or question All information should be provided to arrive at single, correct response Could the item be answered correctly without seeing the answer choices? If �no� work to make the stem more complete. Is more than one response correct? Eliminate one or ask students to choose �the best answer�

13. Clearly Formulated Item Stem Example: NO: In the story, the grandmother rescued a raccoon. * lived in a very large house. grew crops on a farm. disagreed with her neighbor The item cannot be answered without looking at the choices. The stem is too broad a range for possible answers.The item cannot be answered without looking at the choices. The stem is too broad a range for possible answers.

14. Clearly Formulated Item Stem Example: YES: In the story, the grandmother rescued a farmer. child. raccoon. * chicken. Item can be answered without looking at the answered choices.Item can be answered without looking at the answered choices.

15. Language* Clear, focused wording Basic, grade-level appropriate vocabulary Simple rather than complex sentences Eliminate all non-essential language and details/information Avoid polar opposites Proofread for correct grammar, spelling, and punctuation

16. Language* According to the paragraph that is next to the last in the first section, provide a critique of the reasonibleness of the author�s stance based on circumspectual evidence giving the preeminent and the deficient rationale to support your response Point out errors from previous slide Explain EDL book and appropriate/inappropriate usePoint out errors from previous slide Explain EDL book and appropriate/inappropriate use

17. Structure Concise answers Keep as brief and simple as possible Move common distracter wording to the stem Comparable distracters Similar length Similar complexity and detail Grammatically parallel Grammatically consistent with stem

18. Structure * No unique answer choices Use of proper nouns consistent Starting letters all different, all the same, or 2+2 Verbs and verb tense consistent Positives and negatives Equal use of positive and negative word choice Avoid double negatives Positive frame of reference preferred Provide examples for participants discuss and correct from WestEd bookProvide examples for participants discuss and correct from WestEd book

19. Bias and Sensitivity Issues Equal opportunity and access Fair opportunity to demonstrate what they know regardless of race, ethnicity, gender, religion, disability, SES, or location Accurate portrayal of groups Nothing offensive based on racial, ethnic, religious, cultural, gender, social, or regional groups

20. Bias and Sensitivity Issues* Protection of privacy Not to encroach on values and beliefs Sensitivity issues Those that community may be sensitive to (e.g. violence, suicide, death, religious holidays, divorce, abortion, alcoholism, birth control, etc. Non-offensiveness See handoutsSee handouts

21. Selected Response Formats to Avoid Item stem ending with a colon Item stem ending with a/an Jeopardy format This city is the capital of Nevada Item stem with both question and sentence completion Which statement best supports John�s feelings? John�s feelings are mostly

22. Selected Response Formats to Avoid Absolutes Always, never, all, none, only Mismatches with passage content All distracters should be plausible, logical, and linked to the passage Enough context should be present to identify word meanings Use exact quotes (ellipses are acceptable) Background knowledge from other content areas

23. Ability Levels Reading A-1 � Initial understanding A-2 � Interpretation A-3 � Critical stance

24. Content Clusters Reading C-1 � Word analysis and skills (Standard 1) C-2 � Comprehend literature (Standards 2 & 3) C-3 � Comprehend informational text (Standards 2 & 4)

25. Reading Items

26. A-1 Initial Understanding Locating specific information Main ideas that are explicit Supporting details

27. A-2 Develop an Interpretation Infer Integrate new information with text Generalize Define word meanings with context clues Expand Author�s purpose Contributions of characters and setting to the story (in literary)

28. A-2 Develop an Interpretation Analyze Relate to current understanding Implied main idea Moral of story Cause/effect Reasonable conclusions Reasons or alternative solutions

29. A-3 Critical Stance Evaluate Effective supporting details Adequacy of explanation Critique Compare/contrast Likelihood of event taking place Ability of text to achieve purpose

30. Constructed Response Formats Provides an introduction This article tells about the lives of some Korean students. Includes directions for the task A. Describe two ways that the lives of Korean and American students are alike. B. Describe two ways that the lives of Korean and American students are different.

31. Constructed Response Items C-2 or C-3 clusters from prioritized standards A-2 (Developing an Interpretation) or A-3 (Critical Stance) ability levels Individually worth more than selected response items Can receive partial points

32. Constructed Response Practice Pick a prioritized standard Read passage Write/choose an item from the testing materials you have Write a possible student response that would receive full points Discuss

33. What did you learn about item writing?

34. Next Steps