Chapter 7

1. Chapter 7 Selecting, Administering, Scoring, and Interpreting Assessment Results

2. Selecting Assessment Instruments & Strategies • Identify the type of information needed • Identify available information • Determine the methods for obtaining information • Search assessment resources • Evaluate and select an assessment instrument or strategy

3. Identify the Type of Information Needed • Any assessment instrument or strategy is useful only if it provides the information required for the assessment. • Counselors determine what information to collect based on the purpose for assessment.

4. Identify Available Information Pre-existing assessment information about clients can include: • intake questionnaires • biographical data • preliminary diagnoses • school grades • previous assessment results

5. Determine the Methods for Obtaining Information • Counselors can choose from both formal and informal assessment instruments and strategies

6. Search Assessment Resources References sources • Mental Measurement Yearbook • Tests in Print • Test Critiques • Tests

7. Publisher catalogs provide important test information including: Cost of Materials and Scoring Types of Scoring Services While publisher catalogs are a valuable source of information, bias may exist. Catalogs should be used along with other sources of information. Test Publisher Catalogs

8. Test manuals are a primary source of information when selecting tests The Standards for Educational and Psychological Testing (1999) provides a list of key components that should be included in test manuals. Test Manuals

9. Academic journals dedicated to the exploration of testing and assessment can be valuable resources in test selection. Individual journal articles may explore the use of specific tests in various settings. Research Literature

10. Professional Organizations • American Educational Research Association • Association for Assessment in Counseling & Education • National Center for Research on Evaluation, Standards, and Student Testing (CRESST) • National Council on Measurement in Education (NCME) • American Counseling Association • American Psychological Association

11. Online test locator services can be valuable resources for selecting tests. Buros Institute (http://www.unl.edu/buros/) Educational Testing Service (www.ets.org) Internet Resources

12. Directories of Unpublished Instruments • Directory of Unpublished Experimental Measures (2007) • Measures for Psychological Assessment: A Guide to 3,000 Original Sources and Their Applications (1974)

13. Evaluate and Select an Assessment Instrument or Strategy • What is the purpose of the instrument? Who is the intended population? • What sample of the population was used in instrument development, validation, and norming? • Are the results of the instrument reliable? • Do the instrument’s results have evidence of validity? • Does the instrument’s manual provide clear and detailed instructions about administration procedures? • Does the manual provide sufficient information about scoring, interpreting and reporting results? • Is the instrument biased?

14. Evaluate and Select an Assessment Instrument or Strategy cont. 8. What level of competency is needed to use the instrument? • Test User Qualifications A-level: no advanced training required B-level: master’s degree in counseling, psychology, education or related area, or have completed specialized training or expertise C-level: doctorate in psychology, education, or a related field, or licensure.

15. Evaluate and Select an Assessment Instrument or Strategy cont. • What practical issues should be considered for this instrument? • Time required for administering the instrument. • Cost of the instrument • Format • Readability • Ease of Administration • Ease of Scoring • Ease of Interpretation

16. Modes of Administering Instruments: Self-Administered Individually Administered Group Administered Computer Administered Video Administered Audio Administered American Sign Language Nonverbal Administering Assessment Instruments

17. Before Administration • Counselor should be familiar with the instrument. • Review manual, forms, other materials. • Be familiar with the content of the instrument. • Review directions for administering the instrument. • For standardized tests: schedule date, room, facilities; obtain booklets, answer sheets, pencils, other materials • Obtain informed consent from examinees • Provide orientation for examinees

18. During Administration • Final check of materials, lighting, ventilation, seating arrangements, work space, pencils, etc. • For standardized instruments, follow administration procedures and deliver instructions verbatim. • Establish rapport with client. • Be alert to problems and client behaviors during administration.

19. After Administration • Collect materials • Count booklets and answer sheets • Pack materials according to stated procedures • Record any incidents.

20. Scoring Assessment Instruments • Hand-scoring • Computer-scoring • Sent to publisher for scoring • Self-scored by client

21. Scoring Performance Assessment • Performance assessment: portfolios, projects, tasks, open-ended exercises. • No clear-cut right or wrong answers. • Scored using a scoring rubric.

22. Scoring Errors Errors in Hand scoring: • Assigning incorrect values to individual responses. • Incorrectly recoding responses • Incorrectly converting raw scores to standard scores • Calculation errors Errors in Computer scoring: • Inputting data incorrectly • Incomplete erasures or stray marks (when using optical scanners)

23. Interpreting Assessment Results • Determine if score is norm-referenced or criterion-referenced. • Explain the meaning of various standard scores and their implications to the test taker. • Understand factors that can affect assessment interpretation: • Psychometric factors • Test taker factors • Contextual factors

24. Interpreting Assessment Results cont. Computer-generated reports or narratives: • Canned interpretations based on test results. • May be very complex • Not stand-alone interpretations: computers cannot incorporate uniqueness of the client • Counselors ultimately accountable for accuracy of interpretations, even when using computer-generated reports.

25. Consider any major differences between the norms or comparison groups and the actual test takers and any differences in test administration practices or familiarity with the specific questions on the tests. Avoid using a single test score as the sole determinant of decisions about test takers. Avoid using tests for purposes not specifically recommended by the test developer unless evidence is obtained to support an alternate use. Report test results accurately and clearly and in a manner than is understood by the test taker. Routinely review a sample of the test answer sheets to verify the accuracy of the initial scoring. Employ systematic procedures to verify the accuracy and consistency of computer scoring of answer sheets. Standards for Scoring and Interpreting Tests

26. Obtain a separate and independent verification that appropriate scoring rules and normative conversions are used for each person tested. Verify as accurate the computation and conversion of raw scores and the conversion of raw scores to normative or descriptive scales prior to release of such information to the test taker or to users of the test results. Check routinely for accuracy of manual or computer reports of test results. The person performing this task must be qualified to recognize inappropriate or impossible scores. Develop and use systematic and objective procedures for observing and recording the conditions and behaviors of persons being tested and make this part of the scores or test results that are reported. Label clearly the scores that are reported and the date that a particular test was administered.