How to choose between diagnostic tools? Discussion: a plea for efficiency

1. How to choose between diagnostic tools?Discussion: a plea for efficiency Dorothy V. M. Bishop University of Oxford http://psyweb.psy.ox.ac.uk/oscci/

2. Purpose of diagnosis • In clinical contexts: differential diagnosis, establish intervention targets, identify possible causal factors, assess suitability for educational placement, etc. • In research: • a) to confirm that participant meets standard criteria for comparability with other studies and/or • b) to quantify areas of impairment

3. Clinical context • Requirement for comprehensive coverage of symptoms • ADI, DISCO, 3Di all seem well-suited • Need for direct observation • ADOS only contender. • Inherent problems: • Process of assessment may affect child’s behaviour • Limited assessment setting may give atypical picture • Self-report for adults • Possible problems with literacy/insight • But AQ and EQ from AAA seem to work well

4. Research context • NIH national database for autism research regards ADI-R and ADOS-G as essential tools • Goal: “to help accelerate scientific discovery” • But are these the best tools to meet that goal?

5. Postgrad project on autism • Comparison of 30 children with autism and 30 IQ-matched controls • Experimental tasks of executive function • Also assessment of IQ and language level • Time per child for travel/testing/scoring: 4 hr • Total hours = 4 x 60 = 240 hr • Achievable in 12 weeks (20 hr per week)

6. Additional time for ADI-R • Travel to see parents, 1 hr (low estimate!) • ADI-R administration, 3 hr • ADI-R scoring, 20 mins • Extra hours = 4.33 x 60 = 260 • Equates to additional 13 weeks • Likely underestimate because • a) Harder to schedule seeing parents than children • b) Longer scoring time for inexperienced person

7. Additional time for ADOS-G • ADOS-G administration, 45 mins • ADOS-G scoring, 15 mins • Extra hours = 1 x 60 = 60 hr • Equates to additional 3 weeks

8. Acceleration of scientific discovery? • Duration of study goes from 12 weeks to 28 weeks – more than 2-fold increase • But this is without including time for training on ADI-R and ADOS-G! • Estimated time to find a course: 6 months • Estimated time to complete each course, 1 week • Estimated time from course to validation: 6 months • Study that was achievable in 3 months now estimated to take minimum of 1 yr 6 mo. • This excludes time for consensus coding

9. Matson et al, 2007 • “Some measures emphasize the fact that they are very detailed. We would argue that detail equals time. From a pragmatic perspective, our view is that a major priority should be to develop the balance between obtaining relevant information to make a diagnosis, while parsing out items that do not enhance that goal”. (p. 49)

10. Is a 2-stage procedure viable? • Many cases of autism are clearcut, and should be readily identifiable on a brief assessment • Evidence that Social Communication Questionnaire (SCQ) as effective as ADI-R in categorising such cases • STEP ONE: use brief instrument to select cases all would agree on • Should be sufficient for many studies

11. Is a 2-stage procedure viable? • For more marginal cases, will we ever reach agreement? • No evidence from studies to date; may be because autism is inherently dimensional (on multiple dimensions) – categorical divide won’t work • Suggests STEP TWO should be measurement of the dimensions of interest quantitatively as in DISCO/3di

12. Recommendation for evaluation of new research instruments EFFICIENCY IS IMPORTANT! • As well as reporting data on reliability and validity, authors should be required to identify the minimum set of items that can be used without losing sensitivity and specificity • Don’t assume interview is better than questionnaire – do an empirical test!

13. Questions for discussion • Why does autism diagnosis take so much longer than diagnosis of ADHD, dyslexia or SLI? • Advantages/disadvantages of direct assessment of child vs. parental report vs. self-report • What are positive reasons and/or problems for retaining current focus on ADI-R and ADOS-G? • When is a dimensional approach preferable? • Gold standard: is “expert clinical judgement” legitimate? What about statistically defined categories? • Do we need more normative data?