1 / 25

Valid, Reliable & Efficient:

Valid, Reliable & Efficient:. A Psychometric Evaluation of “Flash” Word Recognition and “NSSI” Passage Reading Measures. Kathleen J. Brown Matthew K. Fields University of Utah R. Darrell Morris Appalachian State University. Impetus for Current Study.

sera
Download Presentation

Valid, Reliable & Efficient:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Valid, Reliable & Efficient: A Psychometric Evaluation of “Flash” Word Recognition and “NSSI” Passage Reading Measures

  2. Kathleen J. BrownMatthew K. FieldsUniversity of UtahR. Darrell Morris Appalachian State University

  3. Impetus for Current Study • Need for valid, reliable, efficient instruments to determine instructional reading level • Flaws with current instruments • DRA – no rate & time consuming • DIBELS – screen only • IRIs – psychometric evaluation often weak or missing

  4. Impetus for Current Study • Growing use of “Flash” & selected graded passages a.k.a. “NSSI” (Virginia/ASU effect) • Initial psychometric evaluations positive • (Frye, 2004; Frye & Trathen, 2004; Frye Trathen, Olson, & Schlagal, 2002; Palmer, Trathen, Olson, & Schlagal, 2002)

  5. Theoretical Framework (Anastasi, 1988; APA, 1985)

  6. Methods • 4 schools • 2 = Title 1 1 = public, 1 = parochial • 2 = non-Title 1 both = public & mixed SES • 192 students in G2-G5 in March, 2006 • Rank ordered DIBELS or QRI, then sampled 12 students per grade: 4 high, 4 average, 4 poor to achieve a representative distribution for testing

  7. Methods • 135 minutes of assessment in 3 sessions • Presentation order counterbalanced • Flash item selection counterbalanced • 9 on data team; 4 hours protocol training • Manual flash interrater differences = n.s.

  8. Alternate Form Reliability • measure of temporal stability for scores • measure of consistency of response for scores

  9. Alternate Form Reliability • To what extent are NSSI A passage scores equivalent to NSSI B passage scores? • To what extent are computer “Flash” scores equivalent to manual “Flash” scores?

  10. Results: Alternate Form Reliability **p < .01

  11. Content Validity • provides evidence that items on test represent a specific domain • provides evidence that the format and response properties of the test represent the domain

  12. Content Validity • To what extent do the NSSI passages reflect/measure expected grade level benchmarks? • Maybe look at separate means for accuracy, rate, & comp & report those to show • To what extent does the Flash measure reading instructional level?

  13. NSSI Reading Level Criteria

  14. Performance Levelfor NSSI by Grade

  15. Performance Levelfor Flash by Format & Criterion

  16. Concurrent Validity • To what extent are Flash scores and NSSI scores consistent with scores achieved on a “flagship” standardized reading measure (i.e., the GORT)?

  17. Results: Concurrent Validity **p < .01

  18. Average Performance Levelfor NSSI & GORT by Grade Level

  19. Conclusions: For G2-G5 • NSSI A and NSSI B seem to have high validity for identifying students’ instructional reading levels • NSSI A and NSSI B can be considered equivalent forms

  20. Conclusions: For G2-G5 • Manual Flash and Computer Flash seem to have high validity for identifying students’ instructional reading levels when the criterion is set at 85% • Manual Flash and Computer Flash seem to be equivalent forms

  21. Conclusions: For G2-G5 • The GORT does not seem to have high validity for identifying students’ instructional levels—at any grade level. • The GORT over-predicts instructional level—by approx. 2 years. • Note: most GORT comp questions are passage independent (Keenan & Betjemann, 2006)

More Related