Sheryl L. Bishop, PhD

1. Screening for PTSD in a community sample after Hurricane Ike: evaluating the psychometric utility of a brief screening instrument to assess coping, worry, and depression Sheryl L. Bishop, PhD

2. Parent Study: The Impact of Stress and Coping on Physical and Mental Health after Hurricane Ike SL Bishop, University of Texas Medical Branch RJ Ruiz, University of Texas at Austin R Stowe, Microgen Labs, Inc CR Brewin, University College, London CC Benight, University of Colorado at Colorado Springs Phase I: At 4-6 months post-hurricane, online baseline assessment of measures of hurricane impact, perceived stress, depression, PTSD, resource loss, resilience, positive growth, physical symptoms, coping, social support, worry, displacement, damage, injury, financial data, health utilization data, employment & work disruption & four biological measures of stress via salivary samples were collected.

3. Parent Study: The Impact of Stress and Coping on Physical and Mental Health after Hurricane Ike Phase II: Repeated assessment of measures of all variables were collected at the 12 month and 24 month anniversary of Hurricane Ike.

4. Specific Aims • Among many foci of interest was the opportunity to validate a short screening instrument (10 items) for post-traumatic stress symptomology, the Trauma Screening Questionnaire (TSQ) by Brewing et al.1 • Developed from the PSS-SR2 a 17 item instrument • We used a modified PSS-SR, the MPSS-SR3, that included intensity as well as frequency for each of the 17 items as our ‘gold standard’. • Chris R. Brewin, Suzanna Rose, Bernice Andrews, John Green, Philip Tata, Chris McEvedy, Stuart Turner and Edna B. Foa, Brief Screening Instrument for post-traumatic stress disorder, British Journal of Psychiatry (2002), 181, 158-162 • Foa, E. B., Riggs, D. S., Dancu,C.V., et al (1993) Reliability and validity of a brief instrument for assessing post-traumatic stress disorder. Journal of Traumatic Stress, 6, 459-473. • Falsetti, S. A., Resnick, H. S., Resick, P. A., & Kilpatrick, D. (1993). The Modified PTSD Symptom Scale: A brief self-report measure of posttraumatic stress disorder. The Behavioral Therapist, 16, 161-162.

5. Sample • UTMB community: Faculty, staff and students • 7th largest employer in the region (13,000 employees statewide) • 6 hospitals • State’s oldest schools of Medicine, Nursing, Health Professions and Biomedical Sciences • 2,338 students(2,094 from 122 TX counties), over 700 interns, residents, and fellows, and 1,116 faculty. • 391 respondents with complete data for this analysis

6. N=477 Respondents

7. A great deal None

8. Instrument Characteristics Criterion for affirmative PTSD symptomology for TSQ is a score of 6 and 46 for the MPSS-SR.

9. Frequency 8.2%

10. N=401 21.2%

11. N=393 N=393 48.8% 34.6%

12. Validating • Correlation TSQ x MPSS-SR =.767, p<.001 However correlation only indicates that a positive relationship exists. It cannot indicate whether evaluations AGREE. Therefore, it is NOT the appropriate approach.

13. Indicators of Agreement • Interclass correlation coefficient – ICC • Must transform both instruments into the same scale metric…z scores computed for this purpose. • Indicates moderately high agreement however, Bland and Altman (1990) provide compelling arguments against the use of the ICC to evaluate whether two methods are equivalent. Intraclass Correlation Coefficient: Two-way random effects model ICC 95% Confidence Interval F Test Lower Bound Upper Bound Value Sig Single Measures .767 .723 .805 7.581 .000 Average Measures .868 .839 .892 7.581 .000

14. Crosstabulation Analyses • Kappa – compares rating agreement when variables are dichotomized into DX = Yes/No. • Clearly the criteria used to determine cut-offs is critical • And when evaluating a new instrument against a parent instrument, the assumption that the parent instrument is ‘true’ may be unsupported. Thus conclusions about concurrence may be biased in favor of the reference instrument.

15. Kappa =.414, p<.001 is moderate agreement • Sensitivity (True Positives)=78.95% • Specificity (True Negatives)=84.7% • False Positives (Type I error)=15.3% • False Negatives (Type II error)= 21.1% • PV+=35.7% • PV-=97.4%

16. Distribution of Difference Scores d=-.0102 sd=.68153 Limits of Agreement: d-2s = -.0102-(1.36306)= -1.37326 d+2s=-.0102+(1.36306)= 1.35286 N=13 N=7

17. Convergent Validity • Forward stepwise regressions were conducted with the Depression the as outcome variable • The predictor set was hurricane exposure scores, worry, 14 coping styles and either the TSQ or the MPSS-SR PTSD scores • Convergent validity would be demonstrated if both instruments comparably predicted depression.

18. Predicting Depression

19. Summary • All ‘traditional’ assessments of equivalency show a high degree of ‘agreement’ over response patterns or classifications. • Newer procedures also support a moderate to high degree of equivalency between the two instruments. • Limits of agreement indicate a fairly narrow range of variability between assessments between the two instruments.

20. Summary Continued TSQ does a fairly good job of identifying both true positives and true negatives. False positive and false negative rates indicate that TSQ missed more people WITH PTSD than incorrectly diagnosing those as having PTSD who did not suffer High Negative Predictive value = 97.4% = accurate in identifying those that are NOT likely to be suffering from PTSD, Low Moderate Positive Predictive value = 35.7% = fair at identifying those that definitely have symptoms of PTSD.

21. Summary continued • Regression analyses further demonstrated convergent validity by highly similar and reflective patterns of prediction of depression. • There is some indication of support for more complex modeling with the use of the TSQ to allow for finer distinctions in coping behaviors.

22. Conclusion • The TSQ appears to be a good screening tool for use in early triage situations where false positives are more preferable to false negatives. It has high negative predictive value and can reliably rule out those individuals that do not exhibit PTSD symptoms but may incorrectly classify some as having symptoms that will later be ruled out on further testing.

23. QUESTIONS?