1 / 85

Markers, models, and measurement error:

Markers, models, and measurement error: . Exploring the Links Between Attention Deficits and Language Impairments . Acknowledgements.

king
Download Presentation

Markers, models, and measurement error:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Markers, models, and measurement error: Exploring the Links Between Attention Deficits and Language Impairments sean.redmond@health.utah.edu SRCLD 2014

  2. Acknowledgements • Funding provided by: NIDCD grants R03DC008382 “Psycholinguistic and Socioemotional Profiling of SLI and ADHD” and R01DC011023 “Co-occurrence of Language and Attention Difficulties in Children”. • Project Manager: Andrea Ash, PhD • Consultants: Sam Goldstein, PhD Neurology, Learning and Behavior Center, SLC UT; Tiffany Hogan, PhD MGH Institute of Health Professions, Boston MA. • Community contacts: Lisa Holmstead (SLC School District), Rebecca Garda, Deb Luker (Jordan School District), Linda Smith (CHADD), Carrie Francis (Boys and Girls Club) • Research assistants: David Aamodt, Chelsea Ash, Lyndi Ballard, Peter Behnke, Hannah Caron, Kimber Campbell, Jessica Carrizo, Jamie Dressler, Olivia Erickson, Micah Foster, Kristin Hatch, Nathan Lily, Amy Ludlow, Kristi Moon, ElieMunyankindi, Britta Rajamaki, Michelle Stettler, Jennifer Thinnes Whittaker, McKenzie Rohde, Heather Thompson, and Melissa Whitchurch.

  3. Disclosure Statement I have no relevant relationships to disclose (financial or non-financial).

  4. SLI ADHD “Household term” “developmentally inappropriate levels of inattention, hyperactivity, and impulsivity Prevalence: 3-6% (e.g. Willcutt, 2012) The most common pediatric psychiatric disorder DxRate: most recent CDC (2013) report = 11%; varies considerably by region M:F ratio = 4.0:1.0 “Researcher argot” “diminished language proficiencies in the absence of significant limitations in hearing acuity, cognitive development, or social development” Prevalence: 5-7% (e.g. Tomblin et al., 1997) The most common pediatric communication disorder Dx Rate: Unknown. Not tracked by CDC, USDOE, or ASHA - but probably considerably less than prevalence (Johnson et al., 1999; Zhang & Tomblin, 2000; Jones et al., 2014) M:F ratio = 1.6:1.0

  5. SLI cont’d ADHD cont’d Standardized informant rating scales, psychiatric interviews, EF/CPT tasks Highly comorbid condition (co-occurring anxiety, depression, externalizing disorders, reading disability) Heterogeneity has encouraged pursuit of various subtyping schemes that have yet to document developmental stability (e.g. Lahey et al., 2005) Research focus on establishing markers and procedures for differential diagnosis “Well-resourced” disability relative to prevalence rate (Bishop, 2010) Standardized language tests, language samples, verbal memory tasks Frequently co-occurs with reading disability; associations with socioemotional disorders mixed Heterogeneity has encouraged pursuit of various subtyping schemes that have yet to document developmental stability (e.g. Tomblin et al., 2003) Research focus on expanding phenotype to include non-linguistic symptoms associated with other disorders “Under-resourced” disability relative to prevalence rate (Bishop, 2010)

  6. ADHD and LI ADHD has also been one of the most frequently reported co-occurring neurodevelopmental disorders in study samples of children with language impairments (LI) (e.g. Beitchman, Hood, & Inglis, 1990; Benasich, Curtiss, & Tallal, 1993; Lindsay, Dockrell, & Strand, 2007; St. Clair, Pickles, Durkin, & Conti-Ramsden, 2011; Willinger et al., 2003)…. …althoughthe literature provides some discrepant findings with the extent to which co-occurrence rates have exceeded expectations based on general population estimates (cf. Lindsay & Dockrell, 2008; Redmond & Rice, 2002; Rescorla, Ross, & McClure, 2007; Whitehouse, Robinson, & Zubrick, 2011).

  7. ADHD prevalence rate: APA (2013); Willcutt et al. (2012) 0 5 10 1520 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100

  8. ADHD prevalence rate: APA (2013); Willcuttet al.(2012) ADHD diagnosis rate: both sexes CDC (2013) 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100

  9. ADHD prevalence rate: APA (2013); Willcutt et al. (2012) ADHD Dx rate: both sexes CDC (2013) ADHD Dx rate: males CDC (2013) 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100

  10. ADHD Dx rate also varies as a function of locale: western states below 6%; southern/eastern states above 10% ADHD prevalence rate: APA (2013); Willcutt et al. (2012) ADHD Dx rate: both sexes CDC (2013) ADHD Dx rate: males CDC (2013) 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100

  11. LI/SLI prevalence rate: Johnson et al., (1999); Tomblin et al., (1997) ADHD prevalence rate: APA (2013); Willcutt et al. (2012) ADHD Dx rate: both sexes CDC (2013) ADHD Dx rate: males CDC (2013) 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100

  12. LI/SLI Dx rate: ? LI/SLI prevalence rate: Johnson et al., (1999); Tomblin et al., (1997) Ironically, ASHA has tracked service provision for cases of ADHD but hasn’t collected a census on the cases of primary LI/SLI served by SLPs ADHD prevalence rate: APA (2013); Willcuttet al. (2012) ADHD Dx rate: both sexes CDC (2013) ADHD Dx rate: males CDC (2013) 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100

  13. ADHD+LI CO-OCCURRENCE Lindsay & Dockrell (2008); Snowling et al. (2006) Gualiteri et al. (1983) LI/SLI prevalence rate: Johnson et al., (1999); Tomblin et al., (1997) ADHD prevalence rate: APA (2013); Willcutt et al. (2012) ADHD Dx rate: both sexes CDC (2013) ADHD Dx rate: males CDC (2013) 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100

  14. ADHD+LI CO-OCCURRENCE Lindsay & Dockrell (2008); Snowling et al. (2006) LI/SLI prevalence rate: Johnson et al., (1999); Tomblin et al., (1997) Redmond & Rice (2002); Willinger et al. (2003) Gualiteri et al.(1983) Tomblin et al. (2000) Baker & Cantwell (1987); St. Clair et al. (2011) Warr-Leeper et al. (1994); Walsh et al. (2014) ADHD prevalence rate: APA (2013); Willcuttet al. (2012) Beitchman et al. (1989) Trautman et al. (1990) Tirosh & Cohen (1998); Love & Thompson (1988) Cohen et al. (1998) ADHD Dx rate: both sexes CDC (2013) ADHD Dx rate: males CDC (2013) 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100

  15. POTENTIAL CONTRIBUTERS TO CROSS SIGNALS • “Berkson’s bias” • Clinical: 3-90%; Epidemiological: 18-59% • Primary recruitment: • LI cases: 3-59%; ADHD cases: 45-90% • Older/newer reports: • Before 2000: 17-90%; After 2000: 3-73% • Low nonverbal IQ: • Not controlled: 3-90%; Controlled: 3-33%

  16. POTENTIAL CONTRIBUTERS TO CROSS SIGNALS, cont’d • Measurement? • Co-occurrence estimates are meaningless if indices cannot be trusted to differentiate between disorders. • Some popular measures are incapable of differentiating typical from atypical status. • Some measures are good with the typical vs. atypical distinction but are poor when used to differentiate among atypical designations.

  17. Psycholinguistic Markers and Differential Diagnosis • Vocabulary, verbal IQ, and pragmatic indices represent poor choices • Many language tests cannot reliably differentiate cases of LI from non-LI; vocabulary metrics particularly weak (Spaulding, Plante, & Farinella, 2006). • ADHD symptoms align with pragmatic deficits (Camarata, Hughes, & Ruhl, 1988). • Fine (2006) “Language in Psychiatry”: semiotic/pragmatic framing of clinical features associated with ADHD, psychotic disorders, mood disorders, personality disorders, etc. • Pragmatic symptoms load on to a common factor with psychiatric symptom scales and not on to a factor with other language measures (Ash & Redmond, 2014).

  18. Psycholinguistic Markers and Differential Diagnosis, cont’d • Verbal memory (NWR, SR) and tense-marking indices represent good choices (Archibald & Joanisse 2009; Conti-Ramsden, Botting, & Faragher, 2001; SLI Consortium, 2002). • Projects: 2 study samples • Clinically sourced • Community sourced

  19. Psycholinguistic Markers and Differential Diagnosis, cont’d I. Clinically sourced sample (n = 60, 7- 8-years) (Redmond, Thompson, & Goldstein, 2011) • Ascertainment procedure: • SLP caseloads, clinical psychologist caseloads/CHADD, school flyers, community bulletins • Eligibility Protocol: • Exclusionary = hearing screening, speech screening, nonverbal IQ < 80, autism, monolingual status • 20 cases of SLI = Dx of LI, receipt of services, below cutoff on CELF4 screening test (CELF4-ST) • 20 cases of ADHD = Dx of combined-type ADHD, receipt of services, above cutoff on CBCL DSM-IV ADHD • 20 cases of TD = no services, above cutoff on CELF4-ST, below cutoff on CBCL DSM-IV ADHD • ADHD+LI (8 supplemental cases)= met criteria for SLI and ADHD

  20. Psycholinguistic Markers and Differential Diagnosis, cont’d • Measures: • TEGI screening (past, present probes) (Rice & Wexler, 2001),Dollaghan and Campbell’s (1998) NWR, Redmond’s (2005) sentence recall, and the Test of Narrative Language (Gillam & Pearson, 2004). • Naglieri Nonverbal Achievement Test (Naglieri, 2003),My Life in School Checklist (Sharp et al., 1994),Feelings about School Survey (Valeski & Stipek, 2001),Test of Variables of Attention (Dupuy& Greenberg, 1993). • Cases of ADHD tested “off medication”

  21. Redmond, Thompson, & Goldstein (2011)

  22. Redmond, Thompson, & Goldstein (2011)

  23. Redmond, Thompson, & Goldstein (2011)

  24. Narratives: Redmond, Thompson, & Goldstein (2011)

  25. Psycholinguistic Markers and Differential Diagnosis, cont’d • Parigger (2012) “Language and Executive Functioning in Children with ADHD” • Replication and extension of Redmond, Thompson, & Goldstein (2011)in a sample of Dutch-speaking children (SLI < ADHD =TD). • n.s.associations between NWR, SR, Tense-Marking, Narratives (Frog Stories) and ADHD symptoms or EF measures within any of the groups (SLI, ADHD, TD). • n.s. associations between CCC-2 and EF measures (contrary to predictions based on Tannock & Schachar, 1996).

  26. Psycholinguistic Markers and Differential Diagnosis, cont’d II. “Clinically enriched” community sourced sample (n = 122, grades 3 and 2 available). • Ascertainment procedure • On site school screenings via 1,000+ recruitment flyers sent home. • Targeting students in regular ed, CD, LD, EBD, resource/Tier 2 services (n = 420), using Redmond’s (2005) SR (cf. Archibald & Joanisse, 2009) and the Past Probe from the TEGI. • Screener cutoffs determined using only regular ed students (cf. Pena, Spaulding, & Plante, 2006). • All positive screening cases (n = 71) and a comparison group of negative cases (n = 57) invited to participate in blinded confirmatory testing using CELF4 (67% retention). • Supplemental cases of known LI (n = 23) added from another school district.

  27. Psycholinguistic Markers and Differential Diagnosis, cont’d • Eligibility Measures: • Exclusionary: ELL, “gifted/enriched learning programs”, speech screening, hearing screening • 50 Confirmed LI = below -1.0 SD on CELF4 • SLI (70%), NLI, ADHD+LI, autism+LI, EBD+LI • 72 Confirmed Non-LI = above -1.0 on CELF4 • TD (75%), Low-Nonverbal, ADHD, autism, EBD, LD/RD, SWMI • Measures: • CELF4, TEGI • My Life in School Checklist, Automated Working Memory Assessment (Alloway, 2007),Children’s Communication Checklist-2 (Bishop, 2006), NWR • Cases of ADHD tested “off medication”

  28. Zero order Pearson product correlations among screening (SR, PT) and confirmatory measures (CELF-4, TEGI) N = 122

  29. Psycholinguistic Markers and Differential Diagnosis, cont’d • Temporal stability associated with SR and PT screeners • 2-5 month gap between screenings and confirmatory testing • Data available for 102 participants • Different examiners (blinded) • SR: r = .884, p <.001 • PT: r = .820, p <.001

  30. Psycholinguistic Markers and Differential Diagnosis, cont’d • A distinct communication profile for ADHD? • ADHD status associated with elevated levels of vocal hyper-function in young males (loudness, speaking rate, excessive talking) and poor voice quality (hoarseness) relative to TD controls (Garcia-Real et al., 2013; Hamdan et al., 2009). • ADHD > SLI mazes/utterance formulation problems in conversational samples (Redmond, 2004).

  31. Socioemotional Behavioral Markers and Differential Diagnosis • Behavioral/Neuropsychological indices: EF tasks represent poor choices • Stroop, Go/No-Go Tower of London/Hanoi, Wisconsin card sorting, Porteus mazes, etc. • Planning, response inhibition, attentional shift, vigilance, working memory, etc. • Meta-analyses: moderate effect sizes and lack of universality of EF deficits among ADHD (~50%) compromise both positive and negative predictive powers (Willcutt et al., 2005; Nigg et al., 2005). • Many “home-grown” versions of EF tasks in the literature with unknown levels of reliability and validity • Test-retest reliability associated with standardized EF tasks often <.60 • EF tasks may be particularly problematic with LI cases: • 81% false positive rate for GDS EF task against standard of parent/teacher rated ADHD [LI vs. ADHD+LI] (Rielly et al., 1999).

  32. Socioemotional Behavioral Markers and Differential Diagnosis, cont’d • Behavioral/Neuropsychological indices: CPTs represent poor choices • X-type, AX-type, not-X type, XX-type, etc. • Omission errors, commission errors, sensitivity (d’), response bias (Beta), response time, etc. • Riccio, Reynolds, & Lowe (2001) “Clinical Applications of Continuous Performance Tests”: • “….virtually any disorder of childhood that disrupts or compromises CNS integrity or function is a strong candidate to produce decrements in CPT performance (p. 229)” • Eg: mental retardation, TBI, affective disorders, sleep disorders, phenylketonuria, autism, schizophrenia, learning disabilities, Tourette’s syndrome, fetal toxic exposures, maltreatment, neurofibromatosis, low birth weight, conduct disorder, congenital heart defects, seizure disorders, hearing impairment, and general medical referrals….. • ….and SLI (e.g. Finneran Francis, & Leonard, 2009; Spaulding, Plante, & Vance, 2008) • “Reliance on CPTs as a primary diagnostic tool will result in an unacceptably high number of false positive errors (i.e. over-diagnosis of ADHD)” (pp. 232).

  33. Some unpublishable data from the clinically sourced sample…

  34. Some unpublishable data from the clinically sourced sample…

  35. Socioemotional Behavioral Markers and Differential Diagnosis, cont’d • Standardized parent ratings preferred over teacher ratings • Parent ratings are sufficient – 90%agreement between parent and teacher assignment of ADHD status if parent rating positive (Biederman et al, 1990). • …..However, levels of overall agreement between parent and teacher ADHD ratings have consistently been modest (r < .50) (Barkley, 2006). • Teacher ratings don’t agree with observational measures of ADHD symptoms (e.g. Schachar, Sandberg & Rutter, 1986) or with each other (Barkley, 2006; Redmond & Rice, 2001) • Heritability estimates of ADHD higher for parent ratings than teacher ratings (Merwood et al., 2013). • Teacher ratings of ADHD symptoms may be particularly biased against students with LI relative to other clinical groups when compared against blinded psychiatric interviews -twice as many false positives as true positives: 20% vs. 10% (Charach et al., 2009).

  36. Socioemotional Behavioral Markers and Differential Diagnosis, cont’d • DSM III: Presence of ADHD symptoms in either academic or non-academic settings- “situational ADHD”. • DSM IV: “…at last some ADHD symptoms need to be present in multiple settings” • DSM 5: “Children with specific learning disorder may appear inattentive because of frustration, lack of interest, or limited ability. However, inattention in individuals with a specific learning disorder who do not have ADHD is not impairing outside of academic work”

  37. Socioemotional Behavioral Markers and Differential Diagnosis, cont’d • Parent ratings work best if adjusted for presence of language/academic items • Redmond (2001): Language and academic items have often featured on socioemotional behavioral rating scales (e.g. “won’t talk” ,”speech problems”, “poor school work”, “difficulty doing or completing homework”, “does not seem to listen to what is being said to him/her”, “cannot grasp arithmetic”, “has sloppy handwriting”, and “spelling is poor”). • Consequences of removing items on instrument validity? • Removing overlapping symptoms from scales or requiring higher cutoffs for the purposes of differential diagnosis represents a common suggestion (e.g. Brock, Jimmerson, & Hansen, 2009; Millberger et al., 1995).

  38. Socioemotional Behavioral Markers and Differential Diagnosis, cont’d • Redmond & Ash (2014) • Language and academic items appear on several of the CBCL and the Conners’ syndrome scales • CBCL: Withdrawn, Social Problems, Attention Problems, Internalizing • Conners’: Cognitive Problems/Inattention, DSM-Inattentive, DSM-Total • Removing items from these scales improved accuracy of differentiation between SLI and ADHD without compromising differentiation between ADHD and TD (clinical sourced sample) • ADHD vs. SLI differentiation improved with slightly higher cutoffs • Best ADHD vs. SLI differentiation achieved on Conners’ DSM-hyperactive and DSM-impulsive scales (ROC areas: .929 -.971).

  39. The Impact of Comorbidity on LI • Multiple co-occurring disorders might produce additive/interactive effects on clinical symptoms (cf. Wachs, 2000)…. • NLI worse than SLI across various language metrics (Wetherell et al., 2007; Fey et al., 2004; Nippold et al., 2008; Pearce et al., 2010 Rice et al., 2004) • NLI associated with greater risk for behavior problems (Beitchman et al., 1989; Elbro et al., 2011; Law et al., 2009; Snowlinget al., 2006). • …. Or not • SLI = SLI+RD across various language metrics (Bishop et al., 2009; Catts et al., 2005). • Co-occurring disorders can also be associated with subtractive effects if protective factors associated with one of the disorders offsets risk factors associated with the other

  40. Combined clinical and community samples (n = 57, 7- 9-years): Redmond, Ash, & Hogan (under review)

More Related