“Creating Composite Measures Using Factor Analysis: The Total Illness Burden Index

1. “Creating Composite Measures Using Factor Analysis:The Total Illness Burden Index Sherrie H. Kaplan, PhD, MPH Professor of Medicine UC Irvine School of Medicine Academy Health ARM June 8-10, 2008

2. Some Background…

3. Role of Purpose of Measurement • Changes content of aggregate measure • Changes tolerance of error • Changes psychometric requirements of aggregate • Changes ‘level of confidence’, dissemination strategy

4. How to create composites:Lessons from psychometrics… • Choose measures that broadly represent underlying (latent) construct (sampling from domain of observables); each item adds unique information • Hypothesize structure of items in composites before analysis (what measures what?)

5. How to create composites:Lessons from psychometrics… • Conduct confirmatory cluster, latent variable analyses (construct validity) • Decide on scoring methods (simple algebraic sum, weighting, conjunctive or compensatory); test scoring methods • Test reliability, predictive validity of derived composite

6. Models for Composite Scoring • Conjunctive scoring (‘ands’): highest, lowest levels achieved define score • Rheumatoid arthritis trials: patient responded if: • at least a 20% improvement in tender joint count and • 20% improvement in swollen joint count and • at least 20% improvement in 3 out of 5 of the following: pain assessment, global assessment, physician assessment, etc. • Compensatory scoring (‘ors’): high scores on one component make up for low scores on another

7. Models for weighting • Expert defined • Conditioned by ‘expert’ representation • Regression-based • Conditioned by database (provider, patient sample, sample size) • Factor analysis-based • Conditioned by variables included in factor analysis • Reliability-based • Conditioned by database (sample size)

8. Classic Measurement Theory: Using Factor Analysis to Create Composites • Each factor represents ‘latent’ construct • Correlations of items with factors (factor ‘loadings’) represent statistical structure of set of variables • Factor analysis does not require items have difficulty structure

9. Cronbach’s alpha • Measure of internal consistency reliability • Given by formula: • Where: • N = number of tests • σYi2 = variance of item i • σx2= total test variance                                                                                           ,

10. Cronbach’s alpha • Alpha is unbiased reliability estimator if items have equal covariances (means and item variances may differ); i.e. have common factor in factor analysis                                                                                           ,

11. Total Illness Burden: The Latent Construct • Patient-reported composite measure of severity of multiple diseases • Taken together represent increasing risk for substantial declines in health and increased risk for mortality (1-5 years post initial observation)

12. Purposes of Measurement • Post hoc case-mix adjustment • A priori risk stratification of clinical trials • Improve clinical decision making for ‘tailoring’ treatment

13. Subdimensions… • Pulmonary disease • Heart disease • Stroke and neurologic disease • Gastrointestinal conditions • Other cancers (excluding prostate) • Arthritis • Foot and leg conditions

14. Subdimensions (cont’) • Eye and vision conditions • Hearing problems • Hypertension • Diabetes

15. Sample Questions: COPD 1. During the past 6 months, how often did you have wheezing? a. Never b. Once or twice  c. About once a week d. Several times a week e. Several times a day 

16. Sample Questions: COPD 4. During the past 6 months, did you use extra pillows in order to sleep at night because of problems with your breathing? • No  • Yes, 1 pillow  • Yes, 2 pillows  • Yes, 3 or more pillows

17. Steps in Constructing Subdimensions • Transformed variables to uniform metric by clinical definition of severity • Tested reliability of clinically defined scale (Cronbach’s alpha > .70) • Created composite of each subdimension using simple algebraic sum, mean • Items in each subdimension varied • Validated each subdimension as scale using SF-36, etc.

18. Steps in Constructing Composite • Conducted principal components analysis, higher order factor analysis using scales as entries • First factor explained 67% of variance • Other factors had Eigen values, scree indicating single factor solution • Factor loadings ranged from .40 - .70 • Used factor loadings to create composite • Validated derived composite

19. Understanding and Reducing Disparities in Diabetes Care:Coached Care for Diabetes Sherrie H. Kaplan, PhD, MPH Sheldon Greenfield, MD NovoNordisk Lund, Sweden May 28-20, 2008

20. Characteristics of Patient Sample

21. Principal Components Analysis: First Factor

22. Cronbach’s alpha (.799)

23. Correlation of TIBI with Patient-Reported Health Status Measures by Ethnic Group

24. Other TIBI Validation Studies…

29. Preventing Cardiovascular Disease: Identification of Co-Morbidity Subgroups who may not Benefit from Aggressive Diabetes Management Greenfield S, Nicolucci A, Pellegrini F, Kaplan SH

30. QuED Study • Prospective cohort study of consecutively enrolled patients with diabetes who completed TIBI at enrollment in Italian Quality of Care and Outcomes in Type 2 Diabetes Study Group (n=2,613)

31. QuED Study

32. QuED: Total 5-yr CV Events by TIBI

33. QuED: % 5-yr Survival by TIBI

34. “Complex diabetes patients, those with the greatest burden from competing co-morbidities (highest TIBI scores) may benefit less from aggressive glycemic control due to their increased risk of mortality from other causes before those benefits could be realized.”

35. Conclusions • Using factor analysis, possible to derive latent construct that reflects patients’“total illness burden” • Potentially useful in case-mix adjustment, clinical trials design, clinical decision making • Future research aimed at improving sensitivity, specificity, particularly at ‘intermediate ranges’