Part I : Assessment September 14, 2010 PaTTAN Lana Santoro

1. Promising Practices to Improve Reading Performance of Students who are Deaf or Hard of Hearing: Grades 3-6 Pilot Research Project Part I: Assessment September 14, 2010PaTTANLana Santoro

2. Agenda • Our Research Pilot with Grades 3-6 • Diagnostic Assessment • Reading Progress Monitoring Assessment • Depth of Vocabulary Knowledge • Next Steps

3. Learning Community and Study Group Approach • Collective Participation • Shared Inquiry • Formative Evaluation • Action Research

4. Sample Information

5. Beginning Reading SkillsKindergarten: LNF

6. Beginning Reading SkillsFirst Grade: WIF

7. Reading Connected Text - FluencyFirst-Third: TPRI

8. Reading Connected Text – ComprehensionSecond: MAZE

9. Reading Connected Text – ComprehensionThird: MAZE

10. Reading Connected Text – ComprehensionThird: MAZE

11. Reading Simulation • Find a partner. • Decide who will be the teacher (examiner) and student. • Implement the reading assessment as directed.

12. What do Skilled Readers do? • Skilled readers differ from unskilled readers in “their use of general word knowledge to comprehend text literally as well as to draw valid inferences from texts, in their comprehension of words, and in their use of comprehension monitoring and repair strategies.” (Snow, Burns, & Griffin, 1998, p. 62)

13. What we Know About the Factors that Impact Reading Comprehension • Accurate and fluent word reading skills. • Language skills (receptive and expressive vocabulary, linguistic comprehension) • Extent of conceptual and factual knowledge • Knowledge and skill in use of cognitive strategies to improve comprehension or repair it when it breaks down • Knowledge of text structure and genre • Reasoning and inferential skills • Motivation to understand and interest in task and materials

14. What Reading Does for the Mind • Reading comprehension requires knowledge –of words and the world. • E.D. Hirsch, American Educator (Spring 2003)

15. Selected Statistics for Major Sources of Spoken and Written Language (Sample Means) Rank of Median Word Rare Words per 1000 PRINTED TEXTSAbstracts of scientific articles 4389 128.0Newspapers 1690 68.3Popular magazines 1399 65.7Adult books 1058 52.7Comic books 867 53.5Children’s books 627 30.9Preschool books 578 16.3 TELEVISION TEXTSPopular prime-time adult shows 490 22.7Popular prime-time children’s shows 543 20.2Cartoon shows 598 30.8Mr. Rogers and Sesame Street 413 2.0ADULT SPEECH Expert witness testimony 1008 28.4College graduates to friends, spouses 496 17.3 (Adapted from Hayes and Ahrens, 1988)

16. Exposure to Print • A student in the 20th percentile reads books ______ minutes a day. • This adds up to _________words read per year. • A student in the 80th percentile reads books ______ minutes a day. • This adds up to __________ words read per year. .7 21,000 14.2 1,146,000

17. What “gap” do we want to close?What do we mean by proficient reading? • We want students to close the gap and become proficient in reading comprehension. • “Acquiring meaning from written text” (Gambrell, Block, & Pressley, 2002) • “the process of extracting and constructing meaning through interaction and involvement with written language” (Sweet & Snow, 2002) • “thinking guided by print” (Perfetti, 1985) (Torgesen, 2003)

18. Thinking Guided by Print • In middle and high school, reading can be increasingly defined as “thinking guided by print.”

19. What works for late elementary and intermediate level students? • Learning to Read • Decoding and Word Study (“Phonics”) • Fluency • Reading to Learn • Comprehension and Vocabulary • Content Engagement • Content Enhancement

20. Discussion Guide: • What research questions do you have about grade 3-6 students who are deaf or hard of hearing? • What are your students’ instructional needs and learning challenges with reading? •  What are your instructional challenges when teaching reading? •  What are some things that you do instructionally that seems to work well?

21. Assessment • Assessment tells you what to teach • Assessment tells you where you’re going • Assessment tells you how to teach and what instructional adjustments to make so teaching is more effective • Assessment tells you when you get there • Assessment tells you what to do next • Assessment is instruction

22. Standards-Aligned • Diagnostic Assessment • Ascertain, prior to instruction, each student’s strengths, weaknesses, knowledge, and skills. Using diagnostic assessments enables the instructor to remediate students and adjust the curriculum to meet the student’s unique needs. • Formative Assessment (Progress Monitoring) • Allows teachers to monitor and adjust their instructional practice in order to meet the individual needs of their students. They key is how the results are used. The results should be used to shape teaching and learning.

23. Diagnostic Assessment • Individually administered tests designed to determine specific academic areas of strength and weakness (in some cases a detailed error analysis can be provided) • Instructional levels can be determined • Assist with instructional planning and educational goals • Assist in determining areas for future assessment and progress monitoring

24. Stages of Learning Problem Solving Proficiency Maintenance Generalization 65-80% Advanced Acquisition 0-25% Initial Acquisition (Rivera & Smith, 1997) Entry Level

25. Guidelines for Selecting a Diagnostic Assessment • Aligned with the core content of reading (“Big Ideas” of Reading) • Provides grade-level and instructional-level information • Provides diagnostic/instructional profile, error analysis, helps determine what to teach • Has documented technical adequacy (reliable and valid)

26. Guidelines for Selecting a Diagnostic Assessment • Items are developed from specific performance objectives directly linked to an instructional domain • The score is based on an absolute, not a relative standard • The test measures mastery by using specific standards • The focus is criterion-referenced evaluation: what the student can do and cannot do on specific skills and knowledge tasks.

27. What it looks like. . . Items Missed: On the Word Attack subtest, the long a-e patter in nonsense words -gaked, straced; the long I-e pattern in nonsense word -quiles Deficit Skill:Decoding words with the long vowel-consonant-silent e pattern Probe:Decoding words orally to teacher -cake, make, snake, rate, lake, fake, like, bike, kite Criterion:-Decode 10/10 words for mastery-Decode 8/10 words to 6/10 words for instructional level-Decode 5/10 words or less for failure level; assess prerequisite skill level: discrimination of long/short vowels (vowels: a, i)

28. Test Construction Considerations Must consider. . . (1) Which specific tasks should be included (2) How performance should be judged as mastered or not mastered

29. Setting Standards (1) Judgment of Test Content (2) Judgment of Individual Test Takers (3) Assessing Error Rates

30. Judgment of Test Content Standard setters (judges) analyze test items to determine how many should be passed to reflect minimal proficiency --consider the “borderline” test taker

31. Discussion Guide: • Based on the guidelines just presented, consider what diagnostic assessment you could administer to your students. • Do you currently use a diagnostic assessment? If so, which one? Does it meet the criteria we just discussed? • What are the challenges of using diagnostic assessments with your students?

32. Woodcock Reading Mastery Test-Revised • Purpose: • Norm-referenced assessment that provides diagnostic information for instructional decision making • Content: • Letter identification, word identification, word comprehension (antonyms, synonyms, analogies), general reading vocabulary, science-mathematics vocabulary, social studies vocabulary, humanities vocabulary, passage comprehension • Evaluation: • BOY October/November • EOY March/April

33. Appropriate for: Grades K-16 ages 5 years 0 months through 75 Time – 10-30 minutes for each cluster of individually administered tests

34. Reliability Internal reliability • Tests median = .91 (range .68 to .98) • Clusters median = .95 (range .87 to .98) • Total median = .97 (range .86 to .99)

36. ReadinessForm G Only • Visual Auditory learning • A task to determine if the student can associate symbols with words • Tests memory, attention, grouping of word parts (i.e., ing with verbs) • Letter identification • Alphabet recognition • Different fonts • Print and cursive

37. Basic Skills • Test 3: Word identification • Reading words • Begins with one word on a page and advances to multiple words • 106 items in increasing difficulty • The student does not need to know what any of the words mean • Average score for a kindergarten student is 1 • Average score for a student in 12th grade is 96

38. Basic Skills • Test 4: Word Attack • Reading two types of words • Nonsense words • Words with very low frequency usage • Measures the ability to apply phonic and structural analytic skills • Training is provided so the student will know how to approach the test

39. Comprehension • Test 5: Word Comprehension • 3 subtests • Each begins with sample items • Training continues until competes the item correctly. • Subtest 5A: Antonyms • Measures ability to read a word and respond orally with a word opposite in meaning

40. Comprehension • Subtest 5B: Synonyms • Comprehension of reading vocabulary • Read a word and state another word similar in meaning • Synonyms are “a more difficult cognitive processing task than Antonyms.” p. 7

41. Comprehension • Subtest 5C: Analogies • Read a pair of words; • ascertain the relationship, • read the first word of the second pair, • use the same relationship to supply a word to complete the analogy • Demonstrates content embedded word knowledge

42. Word Comprehension Reading Vocabularies • General reading • Science-mathematics • Social studies • Humanities

43. Comprehension • Test 6: Passage Comprehension • Modified cloze procedure • Short passage with a blank line • Student supplies a word that “fits” in the blank • The first 1/3 of the passage are one sentence long and have a picture related to the text

44. Materials • Examiner protocol • Test record booklet • Stimulus book/easel pages for student • Clipboard • Pencil

45. Administration • Administer: • Test 3: Word Identification • Test 4: Word Attack (if possible) • Test 5A: Word Comprehension (Antonyms subtest) • Test 5B: Word Comprehension (Synonyms subtest) • Test 5C: Word Comprehension (Analogies Subtest) • The test battery will take an experienced tester about 45 minutes • Test by complete pages

46. Basal Rules • Start at the points indicated in the tables in the test easel • If the student is correct on the first 6 items, a basal is established. • If less than 6 are correct, go back a page and administer the whole page. • Continue to test backwards starting with the first item on a page until the first 6 on a page are correctly answered

47. Ceiling Rules • 6 or more consecutively failed items that end with the last item on a test page. • See page 22 for an example of basal and ceiling scoring

48. Word Identification • MUST know how to pronounce the words in the test (p. 28-29) • A table of suggested starting points is provided in the easel • If the student does not respond to the first item, score it 0 and say the word and ask the student to repeat it • NO OTHER WORDS WILL BE READ TO THE STUDENT • WRITE what the student said for incorrect responses • Write comments the student says

49. Word Attack • If the student scores 0 or 1 on the word identification, a score of 0 can be recorded for Word Attack • (For our practice, don’t do this) • Begin with the 2 sample items; then proceed to item 1 • Study the pronunciation guide (p. 28-29) • The student must answer within 5 seconds • The “word” must be read naturally –not sounded out for the final reading • WRITE what the student says

50. Word Comprehension • For all three subtests, the student reads the item aloud and responds orally • Only single word responses are acceptable • Mispronunciations are not errors • WRITE what the student says • Begin with the practice item in each subtest