Establishing the psychometric properties of measures of suicidal ideation and attempts

1. Establishing the psychometric properties of measures of suicidal ideation and attempts Jill Harkavy-Friedman, PhD

2. Reasons to Assess Suicidal ideation and attempts in Clinical Trials • Adverse Event • Did the event occur during the trial • Focus of treatment • Did the behavior change during the trial • Define change relative to baseline levels • There is a need for different assessment tools

3. Dilemma: How to standardize across studies • Have well defined universal criterion that represents goal of assessment • Final universal criterion must have established psychometric properties and clinical utility • Study Measures • Permit classification with universal criterion • Measures must have established psychometric properties • An independent rater can make categorization to universal criterion using results of the specific measure

4. How to chose an instrument • Multiple factors that lead to choice of measure • Who: Patients, controls • What: Ideation, attempts, suicides • Where: Office, home, phone/computer • When: Baseline, Weekly, Discharge, Follow-Up • Why: Adverse event; Reduction of behavior • How: Interview (clinical or lay), questionnaire • Feasibility: Cost, time, personnel

5. Choosing the variable of interest? • Based on goal of study & review of literature • Impact/Importance of variable • Thoughts of death, Suicidal ideation, Suicide attempt, Suicide • Absolute value or change score • Did suicidal ideation or attempt occur during trial? • Did suicidal ideation or number of attempts change during trial? • Emergent suicidal ideation and attempts behavior in non-suicidal sample • In high risk samples requires pre-post assessment using prospective design to demonstrate change • Multiple measures vs. single measure • Suicidal ideation and attempts in context of risk factors vs. suicidal behavior alone • Depression, substance use, stress, psychosis

6. Psychometrics • Evaluate the psychometric properties • Reliability • Validity • Sensitivity • Specificity • Variability • Ceiling and Floor effects

7. Reliability=Reproducability

8. Inter-rater Reliability • If 2 or more of us collect and/or see the same information will we make the same rating. • Kappa: Categorical • (attempt/no attempt) • Intra-class correlation: • Numerical (Total score)

9. Test-Retest: Over time Correlate time 1 and time 2 If we measure today and measure tomorrow will we get the same answer.

10. Parallel Forms: Across Measurements Correlate forms Avoid learning effects due to repeated assessments

11. Internal Consistency: Within a test Are the items of a measure assessing a single content or construct? • Split-Half • Spearman Brown • Cronbach’s Alpha

12. What does the number mean? • Range: 0-1.0 • reliability2= percent of variance accounted for by measure (the rest is random error or due to other factors) • 0-.30 - weak • .31-.69 - moderate • .70-1.0 - strong (accounts for 50% of variance or more)

13. Validity • Face Validity: Does it look like it measures what it is supposed to ? • Content Validity: Is the content representative? • Criterion Validity: Predictive, Concurrent • Construct Validity: Accrual of meaning through convergent and discriminant validity

14. Content Validity: Is the content representative? • What forms of suicidal behavior are of interest • Ideation • Frequency • Planfulness • Persistence • Intent • Preparatory Behaviors • Aborted Attempt • Interrupted Attempt • Attempt • Self-Injurious Behavior

15. Criterion Validity: Predictive, Concurrent • Predictive Validity • Does the measure predict future criterion • Requires prospective design • Sensitivity and Specificity for future suicidal behavior • Concurrent Validity • Does the measure relate to variables it ought to relate to at the time of assessment • 90+% of people who suicide have a diagnosable psychiatric condition- are you sure behavior in study is due to the medication being tested • Depression, Substance Use, Aggression, Impulsiveness, Psychosis

16. Construct Validity: Accrual of meaning through convergent and discriminant validity • Validity is accrued • Convergent Validity • Does the measure related to what it ought to • Depression, substance use, stress, health, psychosis • Discriminant Validity • Is the measure independent of variables not related to suicidal behavior (e.g. positive life events)

17. Reliability is the upper limit of validity

18. Can you find an effect? • Sensitivity and Specificity • Variability • Ceiling and Floor effects

19. Sensitivity and Specificity • sensitivity = probability of a positive test among patients with disease • specificity = probability of a negative test among patients without disease

20. Example study establishing psychometric properties • Goal: Establish the psychometric properties of a self-report measure of suicidal ideation and attempts • Sample: • Psychiatric clinic sample without controlling for diagnosis • Measures: • Universal Criterion: measure requiring trained interviewer to determine no suicidal behavior/suicidal ideation/suicide attempt in past week • Self-report questionnaire of suicidal ideation and attempts over past week • SxList: assesses psychopathology associated with behavior • SubUse: assesses substance use • LifeList: assesses positive and negative stress

21. Procedures: • Patient completes all self-report measures • Patient is interviewed by clinical interviewer who categorizes suicidal behavior • Patient returns 2-3 days later and completes suicidal behavior questionnaire • Initial Data Analysis: • Frequency distribution of items, total score and Universal Criterion from suicidal ideation and attempts questionnaire • Internal consistency of suicidal ideation and attempts questionnaire • Correlate suicidal ideation and attempts questionnaire from time 1 and time 2 • Compare categorization by questionnaire vs. clinical rater • Examine relationships of questionnaire and interviewer ratings with additional measures using multi-method multi-trait matrix • Examine sensitivity and specificity of questionnaire relative to interviewer categorization

22. Considerations for assessing suicidal behavior in clinical trials • Suicidal ideation and attempts are multiply determined, intermittent and often recurrent • Measures in clinical trials can include categorization into universal criterion that can be used across studies • Goal of the clinical trial plays an integral role in determining: • Should suicidal ideation and attempts be measured? • How will suicidal ideation and attempts be measured? • When should suicidal ideation and attempts be measured? • How will Adverse Events be measured? • When suicidal ideation and attempts are assessed should associated risk factors also be assessed (at least at baseline)? • Can evaluate possible causal link to Adverse Event?

23. Validity is accrued across studies