1 / 15

The Process of Developing Performance Assessments

The Process of Developing Performance Assessments. The story of the drunkard’s search. Why are you looking in the light to find your keys if you know you dropped them out of the light? Because it is brighter over here.

hila
Download Presentation

The Process of Developing Performance Assessments

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Process of Developing Performance Assessments

  2. The story of the drunkard’s search Why are you looking in the light to find your keys if you know you dropped them out of the light? Because it is brighter over here. When we measure, do we measure what we need to know or what is easiest to measure? Abraham Kaplan, The Conduct of Inquiry. San Francisco: Chandler Publishing, 1964, p. 11.

  3. Creating Valid, Reliable Tests Requires • Careful, thoughtful planning about what to measure and how to measure it. • The ability to write quality items/tasks that measure the intended learning constructs and are not confounded by irrelevant constructs. • Time and focused effort spent writing and revising test items or tasks.

  4. Test planning is a decision making process. A good test rarely serves more than one purpose equally well. Therefore, we should explicitly define the: • primary purpose of the test. • trait the test intends to measure. • content, topics, subject matter, domain, cognitive thought processes, skills, attitudes, etc. • format of test items. • difficulty of items. • criteria and procedures to score responses. • decisions or inferences that the test will be used to support or inform. • primary audience of the test. • how the results will be reported. • how the results should be interpreted and used.

  5. What is a performance assessment? • A specific type of test used to determine what an examinee can or cannot DO (in contrast to what an examinee does or does not KNOW). • Measures PROFICIENCY rather than KNOWLEDGE.

  6. Characteristics of a Performance Assessment • Examinees are expected to do or produce something. • Real or simulated exercises are used to elicit actual, observable, performance. • Successful performance requires examinees to apply the knowledge and skills they have acquired. • The application is one in the context of completing a prespecified task according to prespecified criteria. • The “doing” is directly observed and subjectively rated by a trained, qualified, judge.

  7. Advantages and disadvantages of using performance assessments Advantages Disadvantages The process is typically costly in terms of time, effort, equipment, materials, facilities, or funds. The rating process is subjective. The results often provide inadequate basis for generalizing across tasks. • They provide a means of assessing valued skills that can not be directly assessed with traditional tests.

  8. Thinking as an act. • To the extent that we can find a way to observe either the process or the products of student’s thinking, then performance assessment can be used to evaluate their thinking. You have to get their internal thinking out of their head.

  9. What do we want students to DO/perform as a result of our instruction? Short-term Long-term

  10. Desirable Characteristics of Criteria Used in Performance Assessment • Significance • Fidelity • Generalizability • Developmental Appropriateness • Accessibility • Utility (Quellmalz, E.S. (1991). Developing criteria for performance assessments: The missing link. Applied Measurement in Education, 4, 319-331.)

  11. Guidelines for constructing performance assessments • Define instructional targets • Cost/benefit analysis • Key indicators • Create assessment • Develop a feasible scoring and reporting procedure • Pilot test and revise.

  12. Three types of rating scales • Numeric Scale: How honest is this person? 1 2 3 4 5 • Graphic Scale: How honest is this person? • Descriptive Scale: How honest is this person?

  13. Evaluating the assessment Response Authenticity StimulusAuthenticity

  14. Rater Biases and Indiosyncrasies • Generousity Error • Severity Error • Central Tendency Error • Halo Error

  15. PA in regular instruction Often, performance assessment tasks are indistinguishable from good instructional activities. Rather than viewing students as recipients of discrete bits of knowledge, modern learning theory conceives students as active participants in the construction of meaning. According to this view, new information must be actively transformed and integrated with a student’s prior knowledge. High quality PA take student’s background knowledge into account and engage students in the active construction of meaning. Linn and Miller, 257.

More Related