1 / 107

Test Construction and Evaluation

Test Construction and Evaluation. Lizamarie Campoamor-Olegario. Purposes of Classroom Tests. Establish basis for assigning grades Determine how well each student has achieved the course objectives Diagnose student problems for remediation Determine where instruction needs improvement.

alina
Download Presentation

Test Construction and Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Test Construction and Evaluation Lizamarie Campoamor-Olegario Lizamarie Campoamor-Olegario

  2. Purposes of Classroom Tests • Establish basis for assigning grades • Determine how well each student has achieved the course objectives • Diagnose student problems for remediation • Determine where instruction needs improvement Lizamarie Campoamor-Olegario

  3. Test or Scale Development • Step 1. Definition of Construct • Step 2. Table of Test Specifications • Step 3. Item Writing • Step 4. Content Validation including Factor Analysis • Step 5. Pilot Testing to Equivalent Group • Step 6. Item-analysis, reliability analysis • Step 7. Final Administration • Some Reminders Lizamarie Campoamor-Olegario

  4. Step 1. Definition of the construct Lizamarie Campoamor-Olegario

  5. Step 1. Definition of Construct • Literature review • Scientifically defined constructs • Popular definition • Syllabus back Lizamarie Campoamor-Olegario

  6. Step 2. Table of test specifications Lizamarie Campoamor-Olegario

  7. Step 2. Table of Test Specifications • Subject: _______________ • Grade/ Year Level: _____ • Type of Test: Cognitive (Knowledge), Psychomotor (Skills), or Affective (Attitudes)? • Testing Time: _____ • Type of Item/s: Likert? Or rating? • Number of items: ____ Lizamarie Campoamor-Olegario

  8. Knowledge (Cognitive) • (Source: Bloom, B., Taxonomy of Educational Objectives, 1956) Lizamarie Campoamor-Olegario

  9. Levels of Cognitive Domain (Knowledge) • Knowledge • The remembering of previously learned material (recall of facts) • Comprehension • The ability to grasp the meaning of the knowledge being learned • Application • The ability to use learning materials in a new way Lizamarie Campoamor-Olegario

  10. Levels of Cognitive Domain (Knowledge) • Analysis • The ability to break material down into its parts so that its organizational structure may be understood • Synthesis • The ability to combine previous experiences with new material to form a whole new structure • Evaluation • The ability to judge the value of material for a given purpose Lizamarie Campoamor-Olegario

  11. Knowledge Can students RECALL information? Key Questions • Who • What • When • Where • How • How much • Describe • Define • Memorize • Literal questions • Which one • Name • Label • List • Reproduce • Recall Lizamarie Campoamor-Olegario

  12. Knowledge Can students RECALL information? • Example of Knowledge QuestionWhich of the following are raw materials of photosynthesis? • Water, heat, sunlight • Carbon dioxide, sunlight oxygen • Water, carbon dioxide, sunlight • Sunlight, oxygen, carbohydrates • Water, carbon dioxide, carbohydrates Lizamarie Campoamor-Olegario

  13. Knowledge Can students RECALL information? • What position does Mr. Chirac hold in France? • President • Prime Minister • General Secretary • Attorney General Lizamarie Campoamor-Olegario

  14. Comprehension Can students EXPLAIN ideas? Key Questions: • Explain • Describe in your own words • Inferential questions • Summarize • What would go better • Select the definition • Read the graph table • This represents • Condense this paragraph • What part doesn’t fit • What are they saying • Explain what is happening • Give an example • State in 5 words • Explain what is meant • What restriction would you add • Translate • Outline • Locate • Match Lizamarie Campoamor-Olegario

  15. Comprehension Can students EXPLAIN ideas? • Example of Comprehension Question • If living cells similar to those found on earth were found on another planet where there was no molecular oxygen, which cell part would most likely be absent? • Cell membrane • Nucleus • Mitochondria • Ribosome • Chromosomes Lizamarie Campoamor-Olegario

  16. ApplicationCan students USE ideas? Key Questions: • What is this used for? • How would you use • Make a model • Tell what would happen • If…how • Demonstrate how • Construct how • Show how • How much would there be if… • Design a lesson • Choose the statements that don’t apply Lizamarie Campoamor-Olegario

  17. ApplicationCan students USE ideas? • Example of Application Question • Phenylketonuria (PKU) is an autosomal recessive condition. About one in every fifty Individuals is heterozygous for the gene but shows no symptoms of the disorder. If you select a symptom-free male and a symptom –free female at random, what is the probability that they would have a child afflicted with PKU? • (.02)(.02)(.25) = 0.0001 = 0.01%, or about 1/10,000 • (.02)(.02) = 0.0004 = 0.04%, or about 1/2,500 • (1)(50)(0) = 100% = all • (1)(50)(0) = 0 = none • 1/50 = 2%, or 2/100 Lizamarie Campoamor-Olegario

  18. ApplicationCan students USE ideas? • It can be inferred from the passage the Mr. Chirac believes that the greatest impact of strict visa regulations is that they: • help the police to subdue French terrorist organizations • help the police to locate foreign terrorists operating in France • reduce the number of foreign terrorists entering France • give more power to anti-terrorist police Lizamarie Campoamor-Olegario

  19. Analysis Do students SEE relationships? • Whole into parts • Analyze • Research/ Survey • Group/ Categorize • Compare and Contrast • What inconsistencies, fallacies • Arrange • What is the relationship • Chart • What is the function of • Diagram • What conclusions • Reason for… • What does the author believe • Investigate • Make a distinction Lizamarie CampoamorM-Olegario

  20. Analysis Do students SEE relationships? • Cause for • What motive is there • Conclude • State the point of view • Separate • Graph • Differentiate • Dissect • Distinguish fact from fiction • fact and inference • fact from opinion • advantage from disadvantage • good from poor reason • What persuasive technique Lizamarie Campoamor-Olegario

  21. Analysis Do students SEE relationships? • Example of Analysis QuestionMitochondria are called the powerhouses of the cell because they make energy available for cellular metabolism. Which of the following observations is most cogent in supporting this concept of mitochondrial function? • ATP occurs in the mitochondria • Mitochondria have a double membrane • The enzymes of the Krebs cycle, and molecules required for terminal respiration, are found n mitochondria • Mitochondria are found in almost all kinds of plant and animal cells • Mitochondria abound in muscle tissue Lizamarie Campoamor-Olegario

  22. Analysis Do students SEE relationships? • Which of the following best explains why countries of the Eurpoean Community have been exempted from France’s new visa regulations? • few terrorists operate in those countries • France is bound by its international treaties • those countries have signed the European Convention on the Suppression of • Terrorism • France needs the support of those countries in its fight against terrorism Lizamarie Campoamor-Olegario

  23. Synthesis: Can students combine ideas and CREATE a new entity? Key Questions • New ways of doing • Take risks • Consider the unexpected • Pose an alternative • Hypothesize • Create/ Build/ Make • Compose • Solve • Design/ Invent • Blend/ Combine • How else would you • Imagine • Plan • Predict • Link concepts unusual and flexible way • What if Lizamarie Campoamor-Olegario

  24. Synthesis: Can students combine ideas and CREATE a new entity? • Briefly outline an alternative strategy that France could use in order to combat terrorist attacks. Your answer should include • a description of the problem • your proposal for dealing with it • the likely consequences, both negative and positive, of such action. • Your answer should not exceed 1 page. Suggested time for this question is 20 minutes. 10 points are allocated for this question. Lizamarie Campoamor-Olegario

  25. Evaluation – Can students make JUDGMENTS and support them? • Evaluate quality, relevance, reliability, truth, effectiveness • Choose and explain why • Rate/ Grade • Rank/ Order • Defend • Dispute • Criticize • Find the errors • Editorialize • Appraise/ Judge • What fallacies, consistencies, inconsistencies appear • Which is more important, better, moral, appropriate, inappropriate, useful, clearer Lizamarie Campoamor-Olegario

  26. Evaluation – Can students make JUDGMENTS and support them? • Example of Evaluation Question • Disregarding the relative feasibility of the following procedures, which of these lines of research is likely to provide us with the most valid and direct evidence as to revolutionary relations among different species? • Analysis of the chemistry of stored food in female gametes • Analysis of the form of the Krebs cycle • Observation of the form and arrangement of the endoplasmic reticulum • Comparison of details of the molecular structure of DNA • Determination of the total protein in the cell • Sources: • http://www.nwlink.com/~donclark/hrd/bloom.html • http://www.fctel.uncc.edu/index.html Lizamarie Campoamor-Olegario

  27. Evaluation – Can students make JUDGMENTS and support them? • Write a critique of the language used by Mr. Chirac, as reported in this article, in terms of its political flavor and intended audience. • Support your discussion with relevant quotes from the passage in addition to examples drawn from other sources. • Points will be awarded for clarity of thinking, logical argumentation and relevant examples. • Do not exceed 2 pages. • 26 points are allocated for this question. Lizamarie Campoamor-Olegario

  28. Skills (Psychomotor) Lizamarie Campoamor-Olegario

  29. Sample Practice Questions 1. Do you eat breakfast? ____ always ____ sometimes _____never 2. How often do you eat breakfast during the week? ____ 1 – 2 days per week ____ 3 – 4 days per week ____ 5 – 6 days per week ____ everyday ____ never Lizamarie Campoamor-Olegario

  30. Sample Practice Questions 3. What do you usually take to have a good breakfast? Give reason. (Please indicate number below). FOODS REASONS 1. _____________________________________________ 2. ____________________________________________ 3. ____________________________________________ Reason: 1 – nutritious 2 – tastes good 3 – always available 4 – affordable 5 – others, specify ___________________ Lizamarie Campoamor-Olegario

  31. Sample Practice Questions 4. Why do you eat breakfast? (Please check all that apply to you.) ____ I feel better if I eat breakfast ____ I am hungry in the morning ____ my parents told me ____ I enjoy eating ____ I feel it is important to eat breakfast ____ others, specify ___________________ Lizamarie Campoamor-Olegario

  32. Sample Practice Questions 5. If you skip breakfast, how do you feel at mid-morning (9:30-10:00)? Check one that best describes how you feel. _____ Fine _____ Angry/bored _____ Hungry/starving _____ Headache/sick _____ Nervous _____ Easily irritated _____ Weak _____ Others, specify: ____________ Lizamarie Campoamor-Olegario

  33. Attitudes (Affective) Lizamarie Campoamor-Olegario

  34. Sample Attitude Items • I cringe when I have to go to math class. • I am uneasy about going to the board in a math class. • I am afraid to ask questions in math class. • I am always worried about being called on in math class. • I understand math now, but I worry that it's going to get really difficult soon. • I tend to zone out in math class. • I fear math tests more than any other kind. • I don't know how to study for math tests. • It's clear to me in math class, but when I go home it's like I was never there. • I'm afraid I won't be able to keep up with the rest of the class. Lizamarie Campoamor-Olegario

  35. Put a check to the number which best corresponds to your answer: 1 – strongly agree 4 - disagree 2 – agree 5 – strongly disagree 3 – neutral Lizamarie Campoamor-Olegario

  36. Put a check to the number which best corresponds to your answer: 1 – strongly agree 4 - disagree 2 – agree 5 – strongly disagree 3 – neutral Lizamarie Campoamor-Olegario

  37. Table of Specifications Lizamarie Campoamor-Olegario

  38. Step 3. Item writing Lizamarie Campoamor-Olegario

  39. Step 3. Item Writing • 1. Keep the Test Short • Try out a longer test on a small number of subjects • If more than 10% fail to finish the test on time… • 2. Plan to Quantify Your Results. • 3. Good tests must be based on the objectives of a course. • 4. Each test should meet four criteria: RVDN • Reliability • Validity • Discrimination • Non-triviality Lizamarie Campoamor-Olegario

  40. RVDN • R: Does not generate random answers and consistently results in marks that reflect the skill level of each student. • V: Measures what it intends to measure. • D: Shows clearly the differences in the levels of achievement of the students to whom it is administered. • N: Focuses on that which the students should know and not on that which is irrelevant to evaluation. Lizamarie Campoamor-Olegario

  41. Step 3. Item Writing • Questions must be clearly written. • Least complex questions should appear first • Reread all the items from the standpoint of the students • Eliminate any systematic pattern of answers • Guard against cultural, racial, ethnic, and sexual bias • Before administering the test, prepare answer keys and scoring procedures Lizamarie Campoamor-Olegario

  42. Test Formats • essay • true-or-false tests • multiple choice tests • completion-and-short answer tests • matching tests Lizamarie Campoamor-Olegario

  43. Essay Test • Avoid words such as what, who, when, which trigger closed responses • Avoid letting them to answer a choice of questions • Give definitive task to student-compare, analyze, evaluate, etc. • Use checklist point system to score with a model answer • Cover the student’s name and the scored items while scoring the exam • Score one question at a time-all at the same time. • Rearrange Lizamarie Campoamor-Olegario

  44. True-or-false tests • Avoid having more than one idea in a question. • Avoid absolute terms: never, only, all, none, always • Avoid indefinite terms: in most cases, great, sometimes, generally, some, few • Avoid double negatives. • Use exact quantitative language • Don't lift items straight from the book. • Make more false than true (60/40). • Require students to circle or underline a typed “T” or “F”, rather than to fill in. Lizamarie Campoamor-Olegario

  45. Multiple choice tests • In order to measure insight, avoid quoting from the textbook. • Avoid “none of these” or “all of the above” • Stem should be in simple, understood language; delete extraneous words. • Make all distracters plausible • Make all alternatives homogeneous (in subject, content, form, length, explicitness, and grammatical structure) • Use negatively stated items sparingly. When they are used, visually emphasize the negative word. • Present alternatives in logical or numerical order. Lizamarie Campoamor-Olegario

  46. Multiple choice tests • Make each item independent of others on test. • Need more than 3 alternatives, 4 is best. • Ask for the best answer and use terms such as “most” and “primary” if more than one answer is partially correct. • The reading and linguistic difficulty of items should be low. • Use care in the repetition of words or phrases between the stem and the correct answer. • Avoid items that reveal the answer to another item. • Ordinarily, distracters should not overlap, subsume, or be synonymous with one another. Lizamarie Campoamor-Olegario

  47. Completion and short answer tests/ Identification • Limit the number of blanks in each question. • Generally, blanks are not inserted at the beginning of the question. • Make sure there is only one unambiguous response for the blank. • In numerical problems, indicate the type of units in which the answer should appear. • State the item so that only one answer is correct • State the items with words that students understand • Avoid items in which the correct answer is a matter of opinion. Lizamarie Campoamor-Olegario

  48. Matching tests • Organize questions together in logical groups. • In your instructions, indicate whether or not some answers may be used more than once. • Need 15 items or less. • Use homogenous material in each exercise. • Make all responses plausible. • Put all items on a single page. • Put response in some logical order (chronological, alphabetical, etc.). • Avoid grammatical clues • Provide more choices than premises Lizamarie Campoamor-Olegario

  49. Step 4. Content validation Lizamarie Campoamor-Olegario

  50. Step 4. Content Validation including Factor Analysis • Validity denotes the extent to which an instrument is measuring what is supposed to measure. • Forms of Validity • Face Validity • Content Validity • Criterion-related Validity • Construct Validity Lizamarie Campoamor-Olegario

More Related