Salomeh Keyhani MD MPH Lawrence C. Kleinman MD MPH Funding: Agency for Health Care Research

1. The Persistent Disconnect Between Practice and Guidelines in the Management of Children with Otitis Media Salomeh Keyhani MD MPH Lawrence C. Kleinman MD MPH Funding: Agency for Health Care Research and Quality, VA HSR&D Career Development Award

2. Background • Otitis Media takes two forms: Acute Otitis Media (AOM) and Otitis Media with Effusion (OME) • Otitis Media is the most common illness for which children present to the doctor • Tympanostomy tube insertion for otitis media is the most common procedure requiring general anesthesia for children in the US • Main Rationale: hearing loss may impact the cognitive, psychosocial and language development of children

3. Objective To describe the persistent discrepancy between OM treatment guidelines and clinical practice for children in two different cohorts spanning 15 years

4. Methods-Study Design • We compare two sets of expert standards on tympanostomy tube insertion for otitis media to data collected from two separate cohorts • We report on the percent of care that is discordant with each standard

5. Expert Standards 1) Explicit criteria developed by a diverse group of 9 physicians through the RAND Appropriatness Method 2) National guidelines developed jointly by the American Academy of Pediatrics, American Academy of Family Physicians and the American Academy of Otolaryngology-Health and Neck Surgery

6. 1990-91 Cohort of Children who Received Tubes • Data from private utilization review firm • Data collected from physicians through telephone contact • 6429 Children—national sample • Average age 3.1 years, majority male (61%), majority white, private insurance (100%) Kleinman, LC et al, JAMA, The Medical Appropriateness of Tympanostomy Tubes For Children, 1994

7. 1990 Standards • RAND Appropriateness Method • Two-round modified Delphi process to integrate literature with expert opinion into explicit criteria • Independent panel of 5 pediatricians and 4 otolaryngologists • Exhaustive and mutually exclusive list of potential clinical scenarios • The panelists rate each scenario on a scale of one to nine with one meaning very inappropriate and nine very appropriate • Scores of 1, 2, and 3 are interpreted as inappropriate, 7, 8, and 9 as appropriate, and 4, 5, and 6 as of uncertain appropriateness • Two or more panelists rating a scenario 7 through 9 and two or more rating the scenario 1 through 3 to represent significant disagreement—uncertain appropriateness

8. Expert Panel Ratings For Tympanostomy Tube Insertion 80 clinical scenarios identified and rated by Panel Examples: • Appropriate: 2 y/o child with >90 days of bilateral effusion, abnormal hearing test and receipt of one or more courses of antibiotics • Uncertain: 2 y/o child with 60 days of bilateral effusion, abnormal hearing test and no treatment with antibiotics • Inappropriate: 2 y/o child with normal hearing, less than 30 days of bilateral effusions and no treatment with antibiotics Kleinman, LC et al, JAMA, The Medical Appropriateness of Tympanostomy Tubes For Children, 1994.

9. 1990-91 Cohort Comparison of cases to RAND Panel explicit criteria: • 41% of the cases were proposed for appropriate indications • 32% for uncertain indications • 27% for inappropriate indications • Results were controversial (private utilization review firm data, standards were criticized)

10. National Guidelines? 1994 Guidelines (AHRQ-AAFP, AAP, AAOHNS) Scope: healthy child age 1 through 3 years • Antibiotic therapy or bilateral myringotomy with insertion of tympanostomy tubes to manage bilateral otitis media with effusion that has lasted a total of 3 months in a child who has a bilateral hearing deficit. 2) Insertion of tympanostomy tubes to manage bilateral otitis media with effusion that has lasted a total of 4 to 6 months in a child who has bilateral hearing deficit.

11. 1994 National Guidelines Eligible Sample: Among the 1990-91 cohort we found 1982 children healthy and age 1 to 3 For these children Tympanostomy: • Recommended for 5% • Not Recommended for 95%

12. Moving ahead to 2000 • AHRQ funded study to examine the appropriateness of tympanostomy tube insertion • Convened RAND Panel • Developed updated criteria • Nine Physicians: 4 pediatricians, 1 family practitioner and 4 otolaryngologists • 2268 clinical scenarios developed and rated by panel

13. Ratings For Tympanostomy Tube Insertion—2000 Examples: • Appropriate: 2 y/o child with >120 days of bilateral effusion, speech delay, bilateral abnormal hearing test and, past history of ear disease, receipt of one or more courses of antibiotics • Uncertain: 2 y/o child with 60 days of effusion, unilateral abnormal hearing test, no speech delay, no past history of ear disease, and treatment with antibiotics • Inappropriate: 2 y/o child with normal hearing, less than 60 days of effusions and no treatment with antibiotics

14. 2002 NY Metropolitan Area Cohort • Data abstracted from the medical records of 5 diverse hospitals for 682 children who had received tympanostomy in 2002 • All clinically relevant data from community pediatricians, otolaryngologists, and hospitals for 1 year prior to surgery • Sample: average age 3.8 yrs, 57% male, 61% white, majority privately insured (74%) Keyhani et al, “Clinical Characteristics of NYC Children who received Tympanostomy Tubes in 2002”, Pediatrics, 2008

15. 2002 NY Metropolitan Area Cohort • Data compared to explicit criteria developed by the panel Cohort (N=682) Appropriate: 7% Uncertain: 23.3% Inappropriate: 69.6%

16. 1994 Tympanostomy Guidelines? • 1994 Guideline was in force at time of data collection Eligible Sample: 172 healthy children from 1 through 3 years old For these children Tympanostomy: • Recommended for 7.5% • Not Recommended for 92.5%

17. New National Guidelines? 2004 Guidelines (AAFP, AAP, AAOHNS) Scope: 2 months to 12 years of age Insertion of Tympanostomy Tube recommended Children when: • OME lasts 4 months or longer with persistent hearing loss or other signs and symptoms • Recurrent OME lasting 4 months or longer in children “at risk” regardless of hearing status or duration • Recurrent OME lasting 4 months or longer Structural damage to the tympanic membrane or middle ear. “At risk” children are defined to include those with hearing loss independent of OME, language or speech disorder, autism and other developmental symptoms, Down syndrome or other craniofacial syndromes that include cognitive, speech or language delay, visual impairment, cleft palate and developmental delay.

18. 2002 Cohort and 2004 Tympanostomy Guidelines Data Compared to 2004 Academy Guidelines Eligible Sample: 412 children between 2 months & 12 y/o Tympanostomy tube insertions • 3.8% of were recommended • 96.2% were not recommended

19. Limitations • Collection of the data for every cohort had numerous limitations • Limitations generally associated with method of data collection • Survey data from physicians (1990-91 cohort) • Data from medical records (2002 cohort) • Imputed duration of effusion • No data on quality of ear exams in the community

20. Conclusions • A substantial amount of practice departs from expert recommendations. • The publication of Guidelines did not appear to impact practice. • The finding that practice is discordant from recommendations appears to be robust to time, to method of data collection, and to choice of expert standard. • Either current practice represents a substantial overuse of surgeries or the guidelines are overly restrictive.

21. Implications • The persistent disconnect between guidelines and clinical practice cannot be good either for children or for those interested in improving their health care. • Substantial overuse would expose children to risk and consume resources that could be better applied to otherwise improving the health of children. • Erroneous guidelines could lead clinicians, policy makers, and researchers to ill advised interventions.

22. Implications Future research needs to explore both the optimal course of treatment and why clinical practice so frequently deviates from accepted guidelines.

24. Five Hospitals: Two academic medical centers, one tertiary care teaching hospital, one private not for profit community hospital, and one teaching public hospital

25. Medical Treatment of AOM/OME

26. Antibiotic Guidelines-OME • 2004 Guidelines (AAFP, AAP, AAOHNS) • Antimicrobials do not have long-term efficacy and are not recommended for routine management

27. Antibiotics Guidelines-AOM 2004 Guidelines (AAFP, AAP in partnership with AHRQ ) Scope: uncomplicated AOM in children from 2 months through 12 years of age without signs or symptoms of systemic illness unrelated to the middle ear. Antibiotics can be withheld in • otherwise healthy children 6 months to 2 years of age with nonsevere illness at presentation and an uncertain diagnosis • to children 2 years of age and older without severe symptoms at presentation or with an uncertain diagnosis.

28. 2004 Cohort • National sample of children treated for AOM or OME in physician offices • Data Source: National Ambulatory Medical Care Survey 2005. • The children in sample were on average 6 years of age, 56% female, 83% white and 61% privately insured

29. 2004 Cohort and Medical Treatment Guidelines