Effective Assessment Techniques: Standards and Alternatives

1. chapter4 Measurement, Assessment, and Program Evaluation Francis X. Short

2. Measurement, Assessment, and Program Evaluation Common assessment strategies • Standardized approaches • Alternative approaches

3. Standardized Approaches • Usually “store bought” tests • Limited selection of test items • Specific directions for administration provided • Usually known validity and reliability • Generally strong psychometrically but weak authentically • Standards provided to make judgments about student test performance

4. Standards • Norm referenced • Criterion referenced

5. Norm-Referenced Standards • Comparisons are made to others from a specifically defined group (age, sex, disability, etc.). • “Norms” are usually developed by testing large numbers, and results are tabulated. • Percentiles, T scores, z scores are used. • “Above average” and “below average” are the types of judgments made. • IQ tests, older PE tests are examples.

6. Criterion-Referenced Standards • Comparisons are made to predetermined “mastery” scores (minimally acceptable scores for a particular purpose). • Criteria are determined by expert opinion, research data, logic, experience, and so on. • “Competent” and “noncompetent” are the types of judgments made. • FITNESSGRAM is an example.

7. Alternative Approaches • Links assessment to instruction • Has day-to-day applicability • Often teacher constructed • Strong authentically, but weak psychometrically (premium on subjective evaluation) • Rubrics, task analyses, and portfolios are examples

8. Rubrics • A type of rating scale in which a student’s performance is matched to one of multiple levels of a skill via a set of criteria • Student knows where he or she stands and what needs to be done for improvement • Works well with IEPs: if PLP = 2, then STOs = 3 or 4 (or certain aspects of 3 or 4)

9. Task Analysis • Different types exist, but all involve breaking skills down into smaller, perhaps sequentially ordered, steps or “focal points.” • When used as an assessment instrument, “missing” components are identified, and a strategy for teaching is revealed. • It can be expanded for use in ecological assessment (including “functional” and “top-down” approaches).

10. Portfolios • A collection of representative student work • Entries: videos, test results (standardized and alternative), peer evaluations, journals, logs, etc. • Strength: multiple assessments on multiple occasions

11. Standardized vs. Alternative • Generally use both. • Give preference to standardized for “unique need” question. • IDEA requires valid, reliable, objective, and nondiscriminatory testing. • Give preference to alternative when devising instructional strategies. • Assessment is curriculum-embedded.

12. Unique Need • Criteria might include • < 15th percentile • > than 1 SD below the mean (T < 40) • 2-year developmental delay or more • Fails to meet criterion-referenced standards • Fails to meet 70% of the competencies in the PE curriculum • Trial placement recommended for corroboration

13. What to Assess in APE? • Remember IDEA definition of PE • Physical and motor fitness. • Fundamental motor skills and patterns. • Skills in aquatics, dance, and individual and group games and sports (including intramural and lifetime sports). • Minimally test physical fitness (or physical activity) and motor skill (including reflexes, rudimentary, fundamental, and specialized skills, as appropriate). • Affective skills may also be assessed in authentic settings.

14. Milani–Comparetti • Purpose: to assess motor development, including reflexes and reactions, in young children (birth–2 years) and children with developmental disabilities • Description: 27 total test items: 9 test head control, body control, and “active movements;” 5 test for primitive reflexes; 13 test for postural reactions • Scoring: age norms for each of the tests are provided • Comment: often administered by therapists

15. Peabody DMS-2 • Purpose: to assess the fine and gross motor development of children (birth–5 years). • Description: 249 test items (mostly developmental milestones) are arranged across six categories and age levels. • Scoring: gross motor quotient, fine motor quotient, and total motor quotient can be calculated. • Comment: normative data are available.

16. TGMD-2 • Purpose: to test fundamental movement patterns in preschool and early elementary children with emphasis on “process” rather than “product” of performance. • Description: 12 patterns are tested within locomotor and object control subtests. • Scoring: based on “focal points” listed for each of the patterns. • Comment: criterion-based scores can be compared to norm-referenced standards.

17. Sports Skills Program Guides • Purpose: to assist in the assessment and instruction of sport skills for people with disabilities (aged 8 and beyond). • Description: task-analyzed assessments are available for 26 sports. • Scoring: “focal points” are checked off as athletes demonstrate the correct techniques. • Comment: used by Special Olympics; strong “authentically” but no validity or reliability reported.

18. Brockport Physical Fitness Test • Purpose: to assess the health-related fitness of youngsters (aged 10–17) with certain disabilities. • Description: typically 4 to 6 test items are selected from 27 possibilities based on a “personalized” approach. • Scoring: test scores are compared to criterion-referenced standards based on gender, age, and in some cases disability. • Comment: closely related to FITNESSGRAM and supported by computer software (Fitness Challenge).

19. ACTIVITYGRAM • Purpose: to record, analyze, and save student physical activity data and produce reports based on the data. • Description: computer program prompts students to recall previous day’s physical activity in 30-minute time blocks. • Scoring: determines total number of minutes of at least moderate level of activity for previous 3 days. • Comment: has good utility for students with disabilities (but might need help recalling or entering data).

20. Program Evaluation • Increasingly important to demonstrate that instructional program is good, not merely claim it is good. • Requires that program goals be clearly articulated. • Program goal should include a criterion (e.g., 90% of all students will engage in at least 30 minutes of moderate-level physical activity at least 4 days per week). • Student data are aggregated to evaluate the program goal.