Assessment: It’s what’s for dinner!

Assessment: It’s what’s for dinner! Laura Henriques California State University, Long Beach www.csulb.edu/~lhenriqu/CSTA2004.ppt Assessment

Some Appetizers True or False? 1. When writing multiple choice items you should always use “none of the above”. 2. Multiple choice items allow you to ask more questions in a given amount of time. 3. Multiple choice tests are less subjective than essay tests.

Multiple Choice 4. When writing multiple choice items the stem, the part of the questions which provides background information and the prompt, A) should include more information than students need. B) should contain multiple parts, enabling you to ask more than one question at a time, thereby resulting in more data about students’ understanding of the content. C) should minimize reading demands. D) should include negatives so that you can catch students who don’t read carefully.

Short Answer 5. _____________ assessment is the type of assessment given at the end of a unit. 6. You should have more ____________ items on a true/false test than ______ items.

Multiple Choice 7. When grading student work it is best to . . . A) grade the best student’s paper first. B) grade the worst student’s paper first. C) grade the same item for every student at the same time before moving on to the next question. D) all of the above.

Overview of Today’s Talk • Understand what makes for “good” test items & how to write them (This is important to understand this because the people who make high stakes tests follow these rules.) • Use assessment results to inform instruction • Understand the relationship between your objectives/goals/standards, instruction, and assessment

Multiple choice True/false Matching Fill in the blank Short/long essay questions Papers Labs Journals Projects Reports Presentations Posters Observations (check lists) How were you assessed? How do you assess your students?

For what purpose do you assess? (or were you assessed?) • Grades • Rank order • To move on to the next level To inform instruction & better help students learn!

As we begin to think about assessment, consider the following two forms: • Formative Assessment- assessments occurring throughout the unit - sometimes referred to as Checking for Understanding • Summative Assessment- the assessment done at the end of a unit or year

Multiple Choice Items • Multiple choice items have a stem or proposition followed by several potential answers. Students must select the most appropriate/best/ideal response. • We call the answer choices foils or distracters.

Writing Multiple Choice Test Items • Formulate useful propositions • Translate the propositions into questions. • Develop a concise answer. • Develop plausible distracters. • Introduce novelty. • When all else fails, consider a true/false alternative.

What does a correctly answered multiple choice question tell you about what students know? • Not necessarily a whole lot!

Types of Multiple Choice(Wiggins & McTighe) • TRADITIONAL- used for testing content standards worth being familiar with and/or important to know and do. • Stem with 4 or 5 choices • ENHANCED- also used for testing content standards worth being familiar with and/or important to know and do. • Stem supported by graph, chart, table or diagram with 4 or 5 choices

Enhanced Multiple ChoiceTaken from CA Dept. of Education (1994) A Sample of Science Assessment (p.100) Leon and Lupe helped to plant a garden for their fifth grade science project on the grounds of their school. • Leon decided to sprout his bean seeds before planting them in the school garden. The picture show several bean seeds starting to grow. The direction of the growth of the root is a response of the root to . . . • light b) heat • c) oxygen d) gravity

Types of Multiple Choice(Wiggins & McTighe) • TRADITIONAL- used for testing content standards worth being familiar with and/or important to know and do. • Stem with 4 or 5 choices • ENHANCED- also used for testing content standards worth being familiar with and/or important to know and do. • Stem supported by graph, chart, table or diagram with 4 or 5 choices • JUSTIFIED- used for a deeper understanding of content knowledge. • A traditional or enhanced question that requires a written justification of the student’s choice.

Justified Multiple ChoiceTaken from CA Dept. of Education (1994) A Sample of Science Assessment (p.104) Directions: Please circle the best answer. Explain why you chose your answer. Judy wanted to find out whether certain materials were good conductors of electricity. She replaced one of the wires in her circuit with a piece of string. What will happen to the bulb now?A. It will blink.B. It will not light.C. It will shine more brightly.D. It will shine less brightly.

Developing the stem: • single idea or question • use of negatives (underlined or capitalized) • minimize reading (if you must have lots of reading consider having several questions linked to the reading) • don't use lots of window dressing/excess baggage that is not needed to answer the question

Developing the options/foils/distracters: • one correct answer is ideal, a "best“ answer is okay • exclusive 3-5 choices is the best -- 2 is not enough, >6 too many (takes too long to read/sort) • use of none of above is okay but it shouldn't be in each item. You don't want it to be seen as filler so be sure that it is the right answer some of the time. • all of the above should be avoided because it doesn’t distinguish between students who know it really is ALL of the above from students who know it’s 2 of the above

Distracters … continued • use sensibly • common misconceptions/misunderstandings make attractive wrong answers • avoid paired opposites (mutually exclusive answers) • common phrases and textbook language are attractive • consistent units and language; watch that grammar 'matches‘ • be sure answers are approximately the same length • be careful about convergence • Use distracters that provide you with meaningful information

Paired Opposites • In a matriarch which group rules? A) Kings B) Men C) Priests D) Women This item was a $8,000 question on the TV show Who Wants to Be a Millionaire?

Consistent Units And Language • A cart with a mass of 10 kg is initially at rest. It accelerates at 4 m/s2. How far does it travel if 5 seconds? A) 100 m B) 20 m/s C) 10 m/s D) 5 s d = ½ a t2 + vo t The only answer that has distance units

Meaningful Distracters • A cart of mass 10 kg is traveling at a speed of 4 m/s. How much kinetic energy does it have? • A) 160 joules • B) 80 joules • C) 40 joules • D) 20 joules Forgot to multiply by 1/2 Correct answer KE = ½ m v2 Used formula for momentum instead of KE Forgot to square velocity

Let’s Review….. 1. When writing multiple choice items you should always use “none of the above”.

4. When writing multiple choice items the stem, the part of the questions which provides background information and the prompt, A) should include more information than students need. B) should contain multiple parts, enabling you to ask more than one question at a time, thereby resulting in more data about students’ understanding of the content. C) should minimize reading demands. D) should include negatives so that you can catch students who don’t read carefully.

True / False Items • A true/false item is a sentence about which students must decide the veracity. • Often times students are instructed to correct (or make true) a false statement.

Developing True/False Items • Write T-F items in pairs. Be sure the content is parallel with one T and one F statement. • Use specific determiners wisely and sparingly* absolutes: never, only, always, every, all * qualifiers: generally, sometimes, usually, some, most, few, often • Internal comparisons help reduce ambiguity. • Use more False than True. • You can ask more T/F items in a given amount of time than you can multiple choice (less reading involved).

Let’s Review….. 2. Multiple choice items allow you to ask more questions in a given amount of time.

Matching Items • Matching items consist of a group of questions, propositions, definitions or problems in one column and a set of answers in another column. Students must match the question with its appropriate answer.

Creating Matching Items • Be sure to group the items around a common theme or topic • Have the longer text be in the left column with shorter texted ‘answers’ on the right column (reduces the reading demands) • Be sure to have the answer column in some logical order (alphabetical, numerical, etc.) • Have more answers than questions • Give instructions that answers can be used more than once

Short Answer (Fill in the Blank) • Short answer items are statements, definitions or propositions which are missing a word or phrase. Students insert the missing word(s) to complete the statement.

Creating Short Answer Items • Put the blank(s) towards the end of the statement. • Be sure that there’s only one answer that works. • Use as many blanks as there are words in the answer. • Don’t give clues by length of blank, make all blanks the same length. • Don’t give grammatical clues i.e., a ___ or an ____ make it a(n) _____

Let’s Review 5. _____________ assessment is the type of assessment given at the end of a unit. 6. You should have more ____________ items on a true/false test than ______ items.

Grading Issues(so far) • Quick and easy! Poor handwriting is less of a concern. • Students view these types of items as being less subjective than essay or short answer tests. • Grading multiple choice, true/false, matching and short answer items is fast and easy. Creating good items, however, is time consuming and difficult.

Essay Questions & Problems • Essay questions are prompts or questions to which students must respond in written format. • Given information, students are required to solve scientific problems. • Both essay questions and problems require students to show their work and often result in the awarding of partial credit.

Essay Questions & Problems • Try to use a larger number of essays that require brief responses as opposed to fewer long essays (able to sample from a larger content area --> more reliable results) • Do not let students select which essay or problems to answer. • It is important to write ideal response before using the item (you will need to do it anyway, it's better to do it as part of item development!) • Specify number of points or the amount of time you expect students to spend on essay.

Essay Questions & Problems (cont.) • Make physical separation between parts of the essay question - prompt/remind students about them (bullets, listed by #, space, etc.) • Pay attention to the amount of space allocated for the answers. • Word choice is fairly important - don't want value questions or philosophical questions, "discuss" is often too vague - focus students on what is wanted.

An Example • Surprised to see a number of students suddenly leave the room only halfway through an exam, the professor followed them into the corridor. He discovered the students completing the balance of the questions while leaning against the walls. When he asked for an explanation, one student pointed to the third question, which asked "Describe Hemingway's The Old Man and the Sea. You may supply a drawing but please leave room to answer." http://www.educationworld.com/a_lesson/lesson275.shtml

Scoring Issues for Essays & Problems • Grade questions in a group (all students’ #1, all students’ #2, etc.) • Consistent use of partial credit • Halo effect • Try not to look at student names when grading • Try not to let sloppy/neat papers influence your assessment of their content

Scoring Rubrics or Grading Criteria • Determine the criteria that will be evaluated • Develop levels of performance/quality for each criteria • Decide if certain areas are to be weighted more heavily than others • Visit http://rubistar.4teachers.org/index.php for help in creating rubrics

Science Fair Experiment : Science Fair 2004

Other Points to Consider • The allocation of points should mirror instructional time (content) • Questions ought to span Bloom’s Taxonomy and be of varying difficulty levels • Types of items/tasks should mirror instructional strategies (process) • Notice the amount of reading - you want to test science knowledge, not reading skills • Develop scoring criteria BEFORE giving test

This should never happen!

Let’s Review….. 3. Multiple choice tests are less subjective than essay tests.

Last one! 7. When grading student work it is best to . . . A) grade the best student’s paper first. B) grade the worst student’s paper first. C) grade the same item for every student at the same time before moving on to the next question. D) all of the above.

Laura Henriques California State University, Long Beach THANK YOU! This presentation is available @ www.csulb.edu/~lhenriqu/CSTA2004.ppt

Assessment: It’s what’s for dinner!