Context for Change

1. Context for Change Dr. Shannon Hall-Mills, Ph.D., CCC-SLP FLDOE/BEESS & Dr. Carolyn FORD,Ed.D., CCC-SLP USF / Dept. Communication Sciences & Disorders

2. Background – Context for Change • A wave of policy changes has been sweeping through the states to better align with updated federal (IDEA, ’04) and state regulations for students with disabilities, and reflect current knowledge, reflect evidence-based and current best practices. • Initiatives such as Response‐to‐Intervention (RtI) and the role of scientifically research based instruction and evidence based interventions in decision‐making have required states to reconsider methods for evaluation, eligibility determination, and implementation of services for students with disabilities.

3. Background – Context for Change • Changes affect all “program” areas of exceptionality, including speech and language impairments. • Professionals are challenged to embrace shifting roles and responsibilities to meet the diverse needs of students.

4. Background – Context for Change Parents Teachers, etc. SLPs School Administrators Community District Administrators University Faculty FLASHA / ASHA Government Officials • Policy changes in Florida have taken place through collaborativeefforts across avariety of stakeholders throughout the state: parents, teachers, SLPs, school/district administrators, community, university faculty, and professional associations.

5. History of Speech/Language Rule Revision in Florida • 3 year process • 8 rule development workshops with opportunity for public input • Purpose of revision: • reflect current knowledge in the field • update practice in accordance with current best practice • better align with recently revised related SBE rules • Proposed rule organization similar to current rule • Content differs significantly

6. Workgroup’s Topics of Discussion • Included in the discussion: • Federal & state regulations • Rules of eligibility from other states • Rule vs. policy vs. guidance • Implications for implementation • Role of ongoing TA and updated resources • Concurrent SLD and general rule revision • Clarity & flexibility of requirements in rule • Role of clinical/professional judgment

7. Workgroup’s Challenges • Redefining eligibility for Speech Impairment (SI) and Language Impairment (LI) without traditional limitations such as: • Cognitive referencing • Discrepancy formulas • Strict cut-off scores • Determining significance and adverse effect of an impairment (speech or language) • Relation between significance and adverse educational impact • Updating definitions • Outlining minimal evaluation practices • Addressing eligibility (speech and language)

8. Key Changes – General Education Intervention Procedures & Activities • Adherence to general education intervention procedures and activities • outlined in SBE Rule 6A-6.0331, F.A.C., General Education Intervention Procedures, Identification, Evaluation, Reevaluation and the Initial Provision of Exceptional Education Services.

9. Key Changes – Evaluation Scores as snapshots… • “A profile of scores is no more the reality of a person than is a portrait taken in a photo studio.” (p. 30, King, 2000)

10. Key Changes - Evaluation • Office of Special Education Programs (OSEP) letter to ASHA (2007)… • “…public agency must use a variety of assessment tools and strategies to gather relevant functional, developmental, and academic information.”

11. Key Changes - Evaluation • Evaluation components will include: • Review of data indicating results of evidence‐based interventions for identified areas of concern (whether in general or special education setting), • Information gathered parent/guardian, teacher(s), student re: language concerns • Observation(s) of student’s language skills in one or more settings • One or more standardized norm‐referenced instruments designed to measure language (alternative instruments as necessary)

12. Key Changes – Eligibility • Removal of cognitive referencing and discrepancy formulas from eligibility criteria because: • IQ scores reflect current abilities, not potential for language learning (Krassowski & Plante, 1997). • The notion that cognition limits language development so that language cannot exceed cognitive performance levels is unfounded (Kamhi, 1998; Lahey, 1996).

13. Key Changes - Eligibility • Eligibility for language impairment (LI) will occur within an RtI framework (using a problem-solving model) already established in Florida policy. Few Some All

14. Key Changes - Eligibility • Allowance for pragmatic language impairment • Additional observation required • Relation between LI and SLD: • Explicit acknowledgement of similarities between academic manifestation of LI and SLD • Alignment of LI and SLD rules

15. Key Changes - Eligibility • Requirement to document the adverse effect of the speech or language impairment on the student’s ability to perform and/or function in the typical learning environment, thereby demonstrating the need for exceptional student education.

16. A Note Regarding Adverse Effect Office of Special Education Programs (OSEP) letter to ASHA (2007)… • “Education performance is not limited to academic performance.” • Effects determined on a “case‐by‐case basis, depending on unique needs of a child, not based only on discrepancies in age or grade performance in academic subject areas.”

17. Key Changes – In Summary • Increased reliance on: • Significance & educational impact of the communication impairment • Collaborative problem‐solving team and data-based decision making • Review of all available data from multiple sources • Clinical/professional judgment

18. Tell me more… More about the Relation between language and cognition and why cognitive referencing for eligibility decisions is a past practice for LI

19. Language and Cognition Correlational data suggest multidirectional relationshipsbetween language and cognition (Casby, 1992; Kamhi, 1998). The relationship between language and cognition is dynamic and complicated(ASHA, 2002; Casby, 1992; Cole, 1996; Notari, Cole, & Mills, 1992).

20. Language and Cognition • The notion that cognition limits language development so that language cannot exceed cognitive performance levels is unfounded(Kamhi, 1998; Lahey, 1996). • Scores on IQ tests can fluctuate across & within tests over time, so discrepancies can be unstable.

21. Language and Cognition • Some comparisons of cognitive and language scores may yield discrepancies when others do not (Aram, Morris, & Hall, 1992; Cole, Mills, & Kelley, 1994; Nelson, 2000). • Some comparisons vary across different points in development over time(Cole, Dale, & Mills, 1992; Cole, Schwartz, Notari, Dale, & Mills, 1995).

22. Cognitive Referencing • Evidence of children with commensurate language and cognitive abilities benefiting from language intervention at least as much as children with the discrepancy between language and cognition (Cole, Dale, & Mills, 1990; Dale & Cole, 1991; Notari, Cole, & Mills, 1992) and in some cases, more so (Cole, Coggins, & Vanderstoep, 1999; Fey, Long, & Cleave, 1994). • IQ scores reflect current abilities, not potential for language learning (Krassowski & Plante, 1997).

23. The Discrepancy Approach • Shaywitz, Escobar, Shaywitz, Fletcher, and Makuch (1992) defined “reading disability” as a discrepancy between “the level of reading ability predicted on the basis of intelligence (ability) and the actual level of reading achievement” (p. 146). • Also used in defining mental retardation, but challenged in the courts and in the research on the language/dialect factor in IQ tests. IQ tests are never strictly nonverbal.

24. Faulty Logic of the Discrepancy Approach • It depends on a poor showing on one test... but a bad day on any test is not a good basis for a diagnosis. • No one ever proposed judging over-achievement as a discrepancy between low IQ scores, say, and high language proficiency... and it would not make sense to do so because IQ scores, for the most part, are language proficiency scores...

25. Measurement Concerns • Comparisons are made based on norm-referenced language tests, which: • focus on narrow aspects of language (receptive vocabulary) rather than broader aspects (discourse, pragmatics) (ASHA, 1989; 2000); • do not include valid, technically adequate, age-appropriate tools to assess all aspects of language for all language levels (ASHA, 2000); • often lack adequate validity and reliability(ASHA, 1989); • are more likely reflecting factors of cognition, achievement, ethnicity, and motivation (ASHA, 1999, 2000).

26. Other Measurement Concerns • Psychometrically incorrect to compare scores across tests having varied standardization populations and theoretical bases (ASHA, 1989; Whitmire, 2000). • “No pure measures of either verbal or nonverbal abilities; children with language difficulties exhibit problems with nonverbal tasks that could affect their IQ scores (Sattler, 1988), thereby leading to a convergence of test scores.” (ASHA, 1989; 2000) • Cognitive tests likely reflect language difficulties (Francis, Fletcher, Shaywitz, Shaywitz, & Rourke, 1996; Nelson, 2000).

27. Evidence Based Practice Influence of EBP in Evaluation & Eligibility Practices

28. Evidence Based Practice • The goal of EBP is the integration of: (a) clinical expertise, (b) best current evidence, and (c) client values to provide high-quality services reflecting the interests, values, needs, and choices of the individuals we serve. • Conceptually, the trilateral principles forming the bases for EBP can be represented through a simple figure:

29. Relevance of EBP Consider the definition of EBP: • “The American Speech-Language-Hearing Association (ASHA) defines evidence-based practice (EBP) as a clinical decision-making process that fosters the integration of high-quality research evidence with clinician expertise and client preferences” (p. 2, Hall-Mills & Apel, 2007). • EBP is accomplished via an integration of best available evidence for diagnostic and treatment methods with sound clinical expertise and judgment (Hegde & Maul, 2006).

30. Workgroup considered EBP in light of… • Evaluation components for all evaluations of speech and language • Assessment considerations for children from culturally & linguistically diverse backgrounds (CLD populations) • Determination of eligibility • Application of scientifically-based research

31. Evaluation & Determination of Eligibility; Basis in EBP (cont’d) • Identification of language impairment by using an arbitrary low cutoff score is “frequently unsupported by the evidence that is available to clinicians in test manuals”(Spaulding, et al 2006). • “It is inappropriate to use severity cut-off scores (e.g., 1.0, or 1.5 SD below the normative group mean) applied to standardized tests as the sole determinant of eligibility.” (ASHA, 2000). • “…severity is not the sole determinant of whether a condition adversely affects educational performance.”(ASHA, 2000).

32. Evaluation & Determination of Eligibility Based on EBP (cont’d) • Assessment considerations for children from culturally & linguistically diverse (CLD) populations: • Consider cultural and linguistic biases inherent in many standardized assessment tools (Hedge & Maul, 2006; Roseberry-McKibbin, 2002). • Consider advantages of dynamic assessment (ASHA, 2000) and child-specific procedures such as observations and language sampling (Hegde & Maul, 2006).

33. Evaluation & Determination of Eligibility Based on EBP (cont’d) Evaluate data regarding the application of eligibility criteria: • use of multiple sources of evidence in determining eligibility, • “mismatches” between eligibility/dismissal criteria & service delivery model may reflect questionable clinical practice (Apel & Schulman, 2001), • Strong body of evidence across three decades, challenging the use of cognitive referencing in determining eligibility for services.

34. Scientifically based research … • In IDEA [reference to the definition in section 9101(37) of the ESEA] Scientifically based research - • Means research that involves the application of rigorous, systematic, and objective procedures to obtain reliable and valid knowledge relevant to education activities and programs; and • Includes research that – (1) Employs systematic, empirical methods that draw on observation or experiment; (2) Involves rigorous data analyses that are adequate to test the stated hypotheses and justify the general conclusions drawn; (3) Relies on measurements or observational methods that provide reliable and valid data across evaluators and observers, across multiple measurements and observations, and across studies by the same or different investigators; (4) Is evaluated using experimental or quasi-experimental designs in which individuals, entities, programs, or activities are assigned to different conditions and with appropriate controls to evaluate the effects of the condition of interest, with a preference for random-assignment experiments, or other designs to the extent that those designs contain within-condition or across-condition methods; (5) Ensures that experimental studies are presented in sufficient detail and clarity to allow for replication or, at a minimum, offer the opportunity to build systematically on their findings; and (6) Has been accepted by a peer-reviewed journal or approved by a panel of independent experts through a comparably rigorous, objective, and scientific review.

35. Scientifically based research … • In Florida, SBE Rule 6A-6.03411(1)(ff), F.A.C., Definitions, ESE Policies and Procedures, and ESE Administrators. (ff) Scientifically based research. Scientifically based research means research that involves the application of rigorous, systematic, and objective procedures to obtain reliable and valid knowledge relevant to education activities and programs, and includes research that: 1. Employs systematic, empirical methods that draw on observation or experiment; 2. Involves rigorous data analyses that are adequate to test the stated hypotheses and justify the general conclusions drawn; 3. Relies on measurements or observational methods that provide reliable and valid data across evaluators and observers, across multiple measurements and observations, and across studies by the same or different investigators; 4. Is evaluated using experimental or quasi-experimental designs; 5. Ensures that experimental studies are presented in sufficient detail and clarity to allow for replication; and 6. Has been accepted by a peer-reviewed journal or approved by a panel of independent experts through a comparably rigorous, objective, and scientific review.

36. References • Aram, D. M., Morris, R., & Hall, N. E. (1992). The validity of discrepancy criteria for identifying children with developmental language disorders. Journal of Learning Disabilities, 25, 549-554. • American Speech-Language-Hearing Association. Committee on Language Learning Disorders. (1989, March). Issues in determining eligibility for language intervention. Asha, 31, 113–118. • American Speech-Language-Hearing Association. (1999). IDEA and your caseload: A template for eligibility and dismissal criteria for students 3–21. Rockville, MD: Author. • American Speech-Language-Hearing Association. (2000). Special Interest Division 1: Language Learning and Education Newsletter, 7(1), 3–29. • American Speech-Language-Hearing Association. (2004). Admission/Discharge Criteria in Speech-Language Pathology [Guidelines]. Available from www.asha.org/policy. • Casby, M. W. (1992). The cognitive hypothesis and its influence on speech-language services in schools. Language, Speech, and Hearing Services in Schools, 23, 198–202. • Casby, M. W. (1996, April). Cognition and language: Basis, policy, practice, and recommendations. In P. A. Prelock (Ed.), Special interest divisions, language learning and education (Vol. 3, Issue 1, p. 5).

37. References • Cole, K. (1996, April). What is the evidence from research with young children with language disorders? In P. A. Prelock (Ed.), Special interest divisions, language learning and education (Vol. 3, Issue 1, pp. 6–7). • Cole, K.N., Coggins, T.E., & Vanderstoep, C. (1999). The influence of language/cognitive profile on discourse intervention outcome. Language, Speech, and Hearing Services in Schools, 30, 61-67. • Cole, K. N., Dale, P. S., & Mills, P. E. (1990). Defining language delay in young children by cognitive referencing: Are we saying more than we know? Applied Psycholinguistics, 11, 291–302. • Cole, K. N., Dale, P. S., & Mills, P. E. (1992). Stability of the intelligence quotient-language quotient relation: Is discrepancy modeling based on myth? American Journal of Mental Retardation, 97(2), 131–145. • Cole, K. N., & Fey, M. E. (1996). Cognitive referencing in language assessment. In K. N. Cole, P. S. Dale, & D. J. Thal (Eds.), Assessment of communication and language (pp. 143–159). Baltimore: Brookes.

38. References • Cole, K.N., Mills, P.E., & Kelley, D. (1994). Agreement of assessment profiles used in cognitive referencing. Language, Speech, and Hearing Services in Schools, 25, 25-31. • Cole, K., Schwartz, I., Notari, A., Dale, P., & Mills, P. (1995). Examination of the stability of two methods of defining specific language impairment. Applied Psycholinguistics, 16, 103–123. • Dale, P.S., & Cole, K.N. (1991). What’s normal? Specific language impairment in an individual differences perspective. Language, Speech, and Hearing Services in Schools, 22, 80-83. • Fey, M. E., Long, S. H., & Cleave, P. L. (1994). Reconsideration of IQ criteria in the definition of specific language impairment. In R. V. Watkins & M. L. Rice (Eds.), Specific language impairments in children (pp. 161–178). Baltimore, MD: Paul Brookes. • Fletcher, J. M., Francis, D. J., Rourke, B. P., Shaywitz, S. E., & Shaywitz, B. A. (1992). The validity of discrepancy-based definitions of reading disabilities. Journal of Learning Disabilities, 25, 555–561. • Francis, D.J., Fletcher, J.M., Shaywitz, B.A., Shaywitz, S.E., & Rourke, B.P. (1996). Defining learning and language disabilities: Conceptual and psychometric issues with the use of IQ tests. Language, Speech, and Hearing Services in Schools, 27, 132-143.

39. References • Hall-Mills, S., & Apel, K. (2007). A hybrid model for teaching and practicing evidence based practice. Perspectives on Language Learning and Education, 14, 1, 20-22. • Hegde, M. N., & Maul, C. A. (2006).Language disorders in children: An evidence-based approach to assessment and treatment. Boston: Allyn & Bacon. • Justice, L. (2006). Evidence-based practice, response-to-intervention, and the prevention of reading difficulties. Language, Speech, and Hearing Services in Schools, 37, 284-297. • Kamhi, A.G. (1998). Trying to make sense of developmental language disorders. Language, Speech, and Hearing Services in Schools, 29, 35-44. • Krassowski, E., & Plante, E. (1997). IQ variability of children with SLI: Implications for use of cognitive referencing in determining SLI. Journal of Communication Disorders, 30, 1– 9. • Lahey, M. (1996, April). Who shall be called language disordered? An update. In P. A. Prelock (Ed.), Special interest divisions, language learning and education (Vol. 3, Issue 1, pp. 5–6). • Lahey, P. (1990). Who shall be called language impaired? Some reflections and one perspective. Journal of Speech and Hearing Disorders, 55, 612–620. • Nelson, N. W. (1996, April). Discrepancy models and the discrepancy between policy and evidence. Opening remarks: Are we asking the wrong questions? In P. A. Prelock (Ed.), Special interest divisions, language learning and education (Vol. 3, Issue 1, pp. 3–5).

40. References • Nelson, N. W. (2000, July). Basing eligibility on discrepancy criteria: A bad idea whose time has passed. Perspectives on Language Learning and Education, 8-12. • Notari, A. R., Cole, K. N., & Mills, P. W. (1992). Cognitive referencing: The (non) relationship between theory and application. Topics in Early Childhood Special Education, 11(4), 22–38. • Sattler, J.M. (1988). Assessment of Children (3rd edition). San Diego: Jerome M. Sattler. • Shaywitz, S., Escobar, M., Shaywitz, B., Fletcher, J., & Makuch, R. (1992). Evidence that dyslexia may represent the lower tail of a normal distribution of reading ability. The New England Journal of Medicine, 326(3), 145–150. • Smith, M. W., & DickinsonSpaulding, T.J., Plante, E., & Farinella, K.A. (2006). Eligibility criteria for language impairment: Is the low end of normal always appropriate? Language, Speech, and Hearing Services in Schools, 37, 61-72. • Whitmire, K. A. (2000, July). Cognitive referencing and discrepancy formulae: Comments from ASHA resources. Perspectives on Language Learning and Education, 13-16.

41. Resources • State Rulemaking Handbook: http://www.flrules.org/rmhb.pdf • ASHA EBP general link (ASHA members): http://www.asha.org/members/ebp/ • EBP in Schools: http://www.asha.org/slp/schools/prof-consult/EvdncBsdSchls.htm