SDQ added value score

1. SDQ added value score Why do we need it, where it came from and what it can and cannot do

2. Why do we need it? • The chronic and fluctuating nature of childhood psychiatric symptoms • Attenuation • Regression to the mean

5. 3 year follow-up of 1999 survey a) Symptoms are persistent at a group level Total symptoms (SDQ)

6. 3 year follow-up of 1999 survey b) Impact is persistent at a group level Total impact (SDQ)

9. Where did it come from? Children in the DoH British child and adolescent mental health survey 2004 and its six-month follow up who:- • Were rated as having a psychiatric disorder • Or children with parents who had approached primary health care or teachers in relation to this child’s mental health within the previous year. • (n=604)

10. How did we calculate it? • Calculated empirically using linear regression after exploring the factors that were known or suspected to increase the persistence of psychopathology. • Added value (in SDQ points) = 2.3 + 0.8*baseline total difficulties score + 0.2*baseline impact score – 0.3* baseline emotional difficulties subscale score – follow up total difficulties score.

11. y= a+bx

12. y= a+b1x1+b2x2+b3x3…. • Added value (in SDQ points) = 2.3 + 0.8*baseline total difficulties score + 0.2*baseline impact score – 0.3* baseline emotional difficulties subscale score – follow up total difficulties score. • y = added value score • a = 2.3 • b1 = 0.8 and x1= baseline total difficulties score • b2 = 0.2 and x2 = baseline impact score • b3 = 0.3 and x3 = baseline emotional difficulties subscale score

13. Added value score = expected score– observed score Added value (in SDQ points) = 2.3 + 0.8*baseline total difficulties score + 0.2*baseline impact score – 0.3* baseline emotional difficulties subscale score – follow up total difficulties score. Where Expected score given baseline parameters Observed score • Expected > observed: added value = positive; follow up score lower than expected; better than expected change • Expected < observed; added value = negative; follow up score higher than expected; worse than expected change. • Expected – observed; added value =0 => same change as in the community

15. We looked at:- Type or severity of diagnosis Age and gender Poor physical health Maternal educational level Maternal anxiety and depression Family (type, function and size) Housing tenure Neighbourhood characteristics Using stepwise linear regression, these factors explained:- 0.6% of variance of the added value score 35.9% of baseline SDQ scores 24.2% of follow up SDQ scores Ie. Very small influence of these factors on the SDQ added value score Complexity factors

17. Added value score applied to children with psychiatric disorder from 1999 survey

18. Added value score applied to 39 clinic cases; mean =2.7 Effect size =0.56 95% confidence interval 0.12-0.99

19. Does it work? • Tested with data from a community based trial of the IY parenting program that had used to the SDQ at two time points 4-8 months apart, including the impact scale and detected a difference. • If the SDQ value added scale worked it should accurately predict the change measured by the trial for the intervention group and while the control group should show no change • We tested the added value score against simple change scores (T1 total difficulties score – T2 total difficulties)

21. The SDQ added value score can:- • Accurately estimate change among groups of children with significant levels of impairing psychopathology, such as high risk groups or those attending services. • We estimate that teams should be able to produce effect sizes of 0.1 or greater, but this needs testing with clinical data • The test trial was on a Sure Start intervention in a community setting using sure start staff

22. Caveats • The added value score is only calibrated for use with therapeutic or targeted interventions and will overestimate change in groups with low levels of psychopathology, so it should not be applied to universal interventions. • The added value score is a tool for evaluating the impact of interventions on groups of children, but the confidence intervals around the scores of individual children will be too wide to interpret in most instances. • The added value score requires follow up to occur between 4 and 8 months after the initial measure. Follow up after a fixed interval is preferable to administration at discharge because of the risk that discharge may follow soon after a spontaneous improvement, and thereby capitalize on chance remission.

23. Lee et al (2005)

24. Caveats • The added value score is based on the SDQ, which is a “wide angle” measure. Clinicians may want supplement the SDQ with more specific outcome measures relating to each child’s individual problems. • The use of multiple measures (clinician, parent, child, process, satisfaction) will provide services with richer data for improving services. • Services need to aim for high response rates from parents in order to obtain representative data. This requires resources. • The added value score cannot provide all the answers to outcome monitoring; it is one of an array of tools • There is a need for further replication with clinical and research data