370 likes | 493 Views
Secondary Data Analysis:. Opportunities and Pitfalls. Who am I and why am I here?. Laurel A. Copeland, BS MPH PhD – VA Health Services Research & Development investigator; UTHSCSA Department of Psychiatry Assistant Professor - Research copelandl@uthscsa.edu http://czresearch.com/dropbox.
E N D
Secondary Data Analysis: Opportunities and Pitfalls
Who am I and why am I here? • Laurel A. Copeland, BS MPH PhD • – VA Health Services Research & Development investigator; UTHSCSA Department of Psychiatry Assistant Professor - Research • copelandl@uthscsa.edu • http://czresearch.com/dropbox
What Can Secondary Data Analysis Do for You? • Provide preliminary data for grant proposals • Facilitate publication while you apply for grants • Explore new areas
Shope, Jean T; Copeland, Laurel A; Maharg, Ruth; Dielman, TE; Butchart, Amy T (1993). Health Education Quarterly 20(3): 373-390. Shope, Jean T; Copeland, Laurel A; Dielman, TE (1994). Alcoholism: Clinical and Experimental Research 18(3): 726-733. Shope, Jean T; Copeland, Laurel A; Marcoux, Beth C; Kamp, Mary E (1996). Journal of Drug Education, 26(4): 323-337. Shope, Jean T; Copeland, Laurel A; Maharg, Ruth; Dielman, TE (1996). Alcoholism: Clinical & Experimental Research, 20(5): 791-798. Copeland, Laurel A; Shope, Jean T; Waller, Patricia F (1996). Journal of School Health, 66(7): 254-260. Barry, Kristen Lawton; Fleming, Michael F; Manwell, Linda; Copeland, Laurel A (1997). Journal of Family Practice, 45(2): 151-158. Zimmerman, Marc A; Copeland, Laurel A; Shope, Jean T; Dielman, TE (1997). Journal of Youth and Adolescence, 26(2): 117-141. Barry, Kristen Lawton; Fleming, Michael F; Manwell, Linda; Copeland, Laurel A; Appel, Scott (1998). Family Medicine, 30(5):366-371. Blow, Frederic C; Barry, Kristen Lawton; BootsMiller, Bonnie J; Copeland, Laurel A; McCormick, Richard; Visnic, Stephanie (1998). Journal of Psychiatric Research, 32: 311-319. Valenstein, Marcia; Barry, Kristen Lawton; Blow, Frederic C; Copeland, Laurel A; Ullman, Esther (1998). Psychiatric Services, 49(8): 1043-1048. Shope, Jean T; Copeland, Laurel A; Kamp, Mary E; Lang, Sylvia W (1998). Journal of Drug Education, 28(3): 185-197. Kales, Helen C; Blow, Frederic C; Copeland, Laurel A; Bingham, Raymond C; Kammerer, Ericka E; Mellow, Alan F (1999). American Journal of Psychiatry, 156(4): 550-556. Blow, Frederic C; Barry, Kristen Lawton; Copeland, Laurel A; McCormick, Richard; Lehmann, Laurent; Ullman, Esther (1999). Psychiatric Services, 50(3): 390-394. Maio, Ronald F; Shope, Jean T; Blow, Frederic C; Copeland, Laurel A; Gregor, MaryAnn; Brockmann, Laurie M; Weber, Janet E; Metrou, Mary E (2000). Annals of Emergency Medicine, 35(3):252-257. Kales, Helen C; Blow, Frederic C; Bingham, Raymond C; Copeland, Laurel A; Mellow, Alan M (2000). Psychiatric Services, 51(6): 795-800. Roberts, J Scott; Blow, Frederic C; Copeland, Laurel A; Barry, Kristen Lawton; Van Stone, William (2000). Journal of Geriatric Psychiatry and Neurology, 13 Summer: 78-86. Blow, Frederic C; Walton, Maureen A; Barry, Kristen Lawton; Coyne, James C; Mudd, Sharon A; Copeland, Laurel A (2000). Journal of the American Geriatrics Society, 48(7): 769-774. Kales, Helen C; Blow, Frederic C; Bingham, Raymond C; Roberts, J Scott; Copeland, Laurel A; Mellow, Alan M (2000). American Journal of Geriatric Psychiatry, 8:301-309. Blow, Frederic C; Ullman, Esther; Barry, Kristen Lawton; Bingham, C Raymond; Copeland, Laurel A; McCormick, Richard; Van Stone, William (2000). American Journal of Orthopsychiatry, 70(3): 389-400. Valenstein, Marcia; Copeland, Laurel A; Owen, Richard; Blow, Frederic C; Visnic, Stephanie (2001). Journal of Clinical Psychiatry 62(7): 545-551. Valenstein, Marcia; Copeland, Laurel A; Owen, Richard; Blow, Frederic C; Visnic, Stephanie (2001). Psychiatric Services, 52(9): 1242-1244. Duffy, Sonia A; Terrell, Jeff; Valenstein, Marcia; Ronis, David; Copeland, Laurel A; Connors, Mary (2002). General Hospital Psychiatry, 24: 140-147. Valenstein, Marcia; Copeland, Laurel A; Blow, Frederic C; McCarthy, John F; Zeber, John E; Gillon, Leah; Bingham, C Raymond; Stavenger, Thomas (2002). Medical Care 40(8): 630-639. Hegedus, Andrea M; Copeland, Laurel A; Barry, Kristen Lawton; Blow, Frederic C. (2002). American Journal of Orthopsychiatry 72(3): 331-340. Copeland, Laurel A; Blow, Frederic C; Barry, Kristen Lawton (2003). Health Education & Behavior 30(3):305-321. Copeland, Laurel A; Zeber, John E; Valenstein, Marcia; Blow, Frederic C (2003). American Journal of Psychiatry, 160:1817-1822. Valenstein, Marcia; Blow, Frederic C; Copeland, Laurel A; McCarthy, John F; Zeber, John E; Gillon, Leah; Bingham, C Raymond; Stavenger, Thomas (2004). Schizophrenia Bulletin 30(2): 255-64. Janz, Nancy K.; Wren, Patricia A.; Copeland, Laurel A.; Lowery, Julie C.; Goldfarb, Sherry L.; Wilkins, Edwin G. (2004). Journal of Clinical Oncology 22(15):3091-3098. Lambert, Michael T.; Terrell, Jeffrey E.; Copeland, Laurel A.; Ronis, David L.; Duffy, Sonia A. (2005). Nicotine & Tobacco Research 7(2):233-241. Copeland, Laurel A; Fletcher, Carol E; Patterson, Judith (2005). Military Medicine 170(7):602-606. Copeland, Laurel A; Elshaikh, Mohamed A; Jackson, James; Penner, Lou A; Underwood III, Willie (2005). Cancer 104(7):1372-1380. O'Neill, Jessica L; Flanagan, Petra S; Zaleon, Carolyn R; Copeland, Laurel A (2005). Pharmacotherapy 25(11):1560-1565. Copeland, Laurel A; Zeber, John E; Rosenheck, Robert A; Miller, Alexander L (in press). Medical Care. Pugh, Mary Jo V; Copeland, Laurel A; Zeber, John E; Cramer, Joyce A; Amuan, Megan T; Cavazos, Jose E; Kazis, Lewis E (in press). Epilepsia. Silveira, Maria J; Copeland, Laurel A; Feudtner, Chris (2006). American Journal of Public Health v96:1243-1248. Hopp, Faith P; Woodbridge, Peter; Subramanian, Usha; Copeland, Laurel A; Smith, David; Lowery, Julie C (2006). Telemedicine and eHealth. Zeber, John E; Copeland, Laurel A; Grazier, Kyle L (2006). Military Medicine v171(7):619-626. Alexander JA, Copeland LA, Metzger ME (in press). Clinical Toxicology. Wilkins EG, Lowery JC, Copeland LA, Goldfarb SL, Wren PA, Janz NA (in press). Medical Decision Making. My CV
My CV without 2° Data • Shope, Jean T; Copeland, Laurel A; Maharg, Ruth; Dielman, TE; Butchart, Amy T (1993). Health Education Quarterly 20(3): 373-390. • Shope, Jean T; Copeland, Laurel A; Dielman, TE (1994). Alcoholism: Clinical and Experimental Research 18(3): 726-733. • Shope, Jean T; Copeland, Laurel A; Marcoux, Beth C; Kamp, Mary E (1996). Journal of Drug Education, 26(4): 323-337. • Shope, Jean T; Copeland, Laurel A; Maharg, Ruth; Dielman, TE (1996). Alcoholism: Clinical & Experimental Research, 20(5): 791-798. • Copeland, Laurel A; Shope, Jean T; Waller, Patricia F (1996). Journal of School Health, 66(7): 254-260. • Barry, Kristen Lawton; Fleming, Michael F; Manwell, Linda; Copeland, Laurel A (1997). Journal of Family Practice, 45(2): 151-158. • Zimmerman, Marc A; Copeland, Laurel A; Shope, Jean T; Dielman, TE (1997). Journal of Youth and Adolescence, 26(2): 117-141. • Barry, Kristen Lawton; Fleming, Michael F; Manwell, Linda; Copeland, Laurel A; Appel, Scott (1998). Family Medicine, 30(5):366-371. • Blow, Frederic C; Barry, Kristen Lawton; BootsMiller, Bonnie J; Copeland, Laurel A; McCormick, Richard; Visnic, Stephanie (1998). Journal of Psychiatric Research, 32: 311-319. • Valenstein, Marcia; Barry, Kristen Lawton; Blow, Frederic C; Copeland, Laurel A; Ullman, Esther (1998). Psychiatric Services, 49(8): 1043-1048. • Shope, Jean T; Copeland, Laurel A; Kamp, Mary E; Lang, Sylvia W (1998). Journal of Drug Education, 28(3): 185-197. • Kales, Helen C; Blow, Frederic C; Copeland, Laurel A; Bingham, Raymond C; Kammerer, Ericka E; Mellow, Alan F (1999). American Journal of Psychiatry, 156(4): 550-556. • Blow, Frederic C; Barry, Kristen Lawton; Copeland, Laurel A; McCormick, Richard; Lehmann, Laurent; Ullman, Esther (1999). Psychiatric Services, 50(3): 390-394. • Maio, Ronald F; Shope, Jean T; Blow, Frederic C; Copeland, Laurel A; Gregor, MaryAnn; Brockmann, Laurie M; Weber, Janet E; Metrou, Mary E (2000). Annals of Emergency Medicine, 35(3):252-257. • Kales, Helen C; Blow, Frederic C; Bingham, Raymond C; Copeland, Laurel A; Mellow, Alan M (2000). Psychiatric Services, 51(6): 795-800. • Roberts, J Scott; Blow, Frederic C; Copeland, Laurel A; Barry, Kristen Lawton; Van Stone, William (2000). Journal of Geriatric Psychiatry and Neurology, 13 Summer: 78-86. • Blow, Frederic C; Walton, Maureen A; Barry, Kristen Lawton; Coyne, James C; Mudd, Sharon A; Copeland, Laurel A (2000). Journal of the American Geriatrics Society, 48(7): 769-774. • Kales, Helen C; Blow, Frederic C; Bingham, Raymond C; Roberts, J Scott; Copeland, Laurel A; Mellow, Alan M (2000). American Journal of Geriatric Psychiatry, 8:301-309. • Blow, Frederic C; Ullman, Esther; Barry, Kristen Lawton; Bingham, C Raymond; Copeland, Laurel A; McCormick, Richard; Van Stone, William (2000). American Journal of Orthopsychiatry, 70(3): 389-400. • Valenstein, Marcia; Copeland, Laurel A; Owen, Richard; Blow, Frederic C; Visnic, Stephanie (2001). Journal of Clinical Psychiatry 62(7): 545-551. • Valenstein, Marcia; Copeland, Laurel A; Owen, Richard; Blow, Frederic C; Visnic, Stephanie (2001). Psychiatric Services, 52(9): 1242-1244. • Duffy, Sonia A; Terrell, Jeff; Valenstein, Marcia; Ronis, David; Copeland, Laurel A; Connors, Mary (2002). General Hospital Psychiatry, 24: 140-147. • Valenstein, Marcia; Copeland, Laurel A; Blow, Frederic C; McCarthy, John F; Zeber, John E; Gillon, Leah; Bingham, C Raymond; Stavenger, Thomas (2002). Medical Care 40(8): 630-639. • Hegedus, Andrea M; Copeland, Laurel A; Barry, Kristen Lawton; Blow, Frederic C. (2002). American Journal of Orthopsychiatry 72(3): 331-340. • Copeland, Laurel A; Blow, Frederic C; Barry, Kristen Lawton (2003). Health Education & Behavior 30(3):305-321. • Copeland, Laurel A; Zeber, John E; Valenstein, Marcia; Blow, Frederic C (2003). American Journal of Psychiatry, 160:1817-1822. • Valenstein, Marcia; Blow, Frederic C; Copeland, Laurel A; McCarthy, John F; Zeber, John E; Gillon, Leah; Bingham, C Raymond; Stavenger, Thomas (2004). Schizophrenia Bulletin 30(2): 255-64. • Janz, Nancy K.; Wren, Patricia A.; Copeland, Laurel A.; Lowery, Julie C.; Goldfarb, Sherry L.; Wilkins, Edwin G. (2004). Journal of Clinical Oncology 22(15):3091-3098. • Lambert, Michael T.; Terrell, Jeffrey E.; Copeland, Laurel A.; Ronis, David L.; Duffy, Sonia A. (2005). Nicotine & Tobacco Research 7(2):233-241. • Copeland, Laurel A; Fletcher, Carol E; Patterson, Judith (2005). Military Medicine 170(7):602-606. • Copeland, Laurel A; Elshaikh, Mohamed A; Jackson, James; Penner, Lou A; Underwood III, Willie (2005). Cancer 104(7):1372-1380. • O'Neill, Jessica L; Flanagan, Petra S; Zaleon, Carolyn R; Copeland, Laurel A (2005). Pharmacotherapy 25(11):1560-1565. • Copeland, Laurel A; Zeber, John E; Rosenheck, Robert A; Miller, Alexander L (in press). Medical Care. • Pugh, Mary Jo V; Copeland, Laurel A; Zeber, John E; Cramer, Joyce A; Amuan, Megan T; Cavazos, Jose E; Kazis, Lewis E (in press). Epilepsia. • Silveira, Maria J; Copeland, Laurel A; Feudtner, Chris (in press). American Journal of Public Health. • Hopp, Faith P; Woodbridge, Peter; Subramanian, Usha; Copeland, Laurel A; Smith, David; Lowery, Julie C (in press). Telemedicine and eHealth. • Zeber, John E; Copeland, Laurel A; Grazier, Kyle L (in press). Military Medicine. • Alexander, Jeffrey A; Copeland, Laurel A; Metzger, Maureen E (in press). Clinical Toxicology. • Wilkins EG, Lowery JC, Copeland LA, Goldfarb SL, Wren PA, Janz NK (in press). Medical Decision Making.
Is it scientifically valid to use data for purposes for which it was not originally collected? • Yes, because…. • No, because… • Maybe…please use caution
Today and next week • Uses of secondary data • Common biases encountered with secondary data • Methods of adjustment • Sources of secondary data • Possible effects of HIPAA on this type of research
Different Uses • Health Care Delivery • Quality assessment • Geographic variation; under-/over-utilization • Adverse events • Outcomes of a particular treatment • Cost • Natural history • incidence, prevalence • prognosis • Association not causation • Other • Regulatory
Some Examples To show you the range of uses To show you the work is publishable
Regulatory enforcement • TENET HEALTHCARE - 2002 • Medicare Report Jan-Oct 2004 included these findings: • – At Alvarado Hospital, prosecutors accused Tenet of running covert kickback arrangements • – Document requests were received from federal prosecutors in L.A. • – False claims were uncovered and the company agreed to pay $22.5M to settle allegations • – Redding Medical Center was determined to be performing unnecessary cardiac surgery • – Tenet sold some holdings to Hospital Partners of America to satisfy part of the 2003 settlement with federal officials • – Whistleblowers were to receive over $8M
Whose data set is it anyway? Sharing raw data from randomized trials Andrew J Vickers Trials. 2006 May 16;7:15 http://www.trialsjournal.com/content/pdf/1745-6215-7-15.pdf
Risk of death in elderly users of conventional vs. atypical antipsychotic medications Wang PS, Schneeweiss S, Avorn J, Fischer MA, Mogun H, Solomon DH, Brookhart MA. N Engl J Med. 2005 Dec 1;353(22):2335-41 http://content.nejm.org/cgi/reprint/353/22/2335.pdf
Cancer incidence in Kentucky, Pennsylvania, and West Virginia: disparities in Appalachia Lengerich EJ, Tucker TC, Powell RK, Colsher P, Lehman E, Ward AJ, Siedlecki JC, Wyatt SW (2005). J Rural Health 21(1):39-47. http://www.blackwell-synergy.com/toc/jrh/21/1
Does Medicare coverage of colonoscopy reduce racial/ethnic disparities in cancer screening among the elderly? Shih YC, Zhao L, Elting LS (2006). Health Aff (Millwood). Jul-Aug; 25(4):1153-62.http://content.healthaffairs.org/cgi/content/full/25/4/1153
Can you think of examples of secondary data analysis that you have read in the medical literature? • Any that have changed your practice or your research direction?
Methods of Learning • Lecture • In class epidemiologic exercise • Journal article evaluation – readings are online (http://czresearch.com/dropbox/) • Optional brief proposal for your own secondary data analysis
Exercise #1 Policy Decision on FundingOf Regional Trauma Center (handout)
Secondary Data Exercise #1 A Regional Trauma Center (which encourages the surrounding hospitals to refer patients with serious injuries to it for expert care) is seeking additional funds from next year’s health budget for more equipment and staff. A local politician (who would rather spend such money on a new hospital named after his father) criticizes this request for additional resources by claming that Regional Trauma Centers do not, in fact, save lives, and he submits the following data to back up his claim. You are asked for your opinion on these data. How should you respond?
Secondary Data Exercise #1 A Regional Trauma Center (which encourages the surrounding hospitals to refer patients with serious injuries to it for expert care) is seeking additional funds from next year’s health budget for more equipment and staff. A local politician (who would rather spend such money on a new hospital named after his father) criticizes this request for additional resources by claming that Regional Trauma Centers do not, in fact, save lives, and he submits the following data to back up his claim. By examining the data stratified by severity, the value of the regional trauma center becomes clear.
Bias • Systematic error in measurement or a systematic difference (other than the one of interest) between groups • Selection • For cohorts, assembly, migration, contamination, and referral bias • Measurement • Confounding
Bias: Anticipate and Control • Restriction (may lose generalizability) • Matching • Stratification • Standardization • Multivariate Adjustment • Assuming the worst (sensitivity analyses) • Discussing possible effects on your results
Example: Hospital Mortality Report Cards • Originally unadjusted • Hospitals without trauma centers, doing primarily elective surgery, etc., looked really good • Made hospitals who took care of the sickest of the sick look bad
Quality Assessment • Data Quality: Garbage in, garbage out • Risk Adjustment: To remove the confounding effect of different institutions providing care to patients with dissimilar severity of illness and case complexity
Data Quality • If no reliable (accurate and adequate) data are available, questions about risk adjustment are moot • Inconsistent practices in assigning standard ICD9 codes & names to diseases exist—but this has improved over time (Do you know who assigns the codes?) http://www.eicd.com/ • Lack of specificity of ICD9 system in some diagnostic areas, especially with regard to severity
Administrative Data Limitations • Need patient-specific identifiers to link episodes • Need multi-year data when outcomes are infrequent • Limited generalizability if restricted by type of institution or hospital, by type of payor, or by location/region • May lack important clinical variables known to be related to outcomes (especially clinical tests or qualitative evaluations of severity)
Risk Adjustment • Controls for those patient characteristics that are related to the outcomes of interest • Removes the confounding effect of different institutions providing care to patients with dissimilar severity of illness and case complexity • Addresses regional variations • Inadequate case-mix adjustment can lead to misclassification of outlier status
Risk Adjustment • Primary data collection vs. administrative data • Disease-specific vs. generic • Commercial vs. developed for your study • Predictors vary by outcomes being predicted
Essential Elements of Risk Adjustment • Outcome-specific • Contains specification of the principal diagnosis • Contains demographics as proxies for preexisting physiological reserve • Measure count of comorbidities and all the most important comorbidities to assume their own empirically derived coefficients
Classification of Disease States • ICD-9: too many specific codes (n~10,000) • Clinical Classifications for Health Policy Research (CCHPR): good for chronic illness and longitudinal care [http://www.ahrq.gov/data/hcup/his.htm] • Primary diagnosis: good for studies that focus on a single episode of care
Risk Adjustment: Charlson • Advantages: • Commonly used case-mix classification system in the health care industry • System with which most clinicians and reviewers are familiar
Risk Adjustment: Charlson • Disadvantages • Principal diagnosis not differentiated • Original work did not specify ICD-9 codes that went into the disease categories • Developed on inpatients predicting mortality; may not be well suited to outpatients at low risk of death • Not good for longitudinal care / chronic illness
Demographic Factors to Consider in Risk Adjustment • Age (e.g., age-adjusted Charlson) • Proxies of Social Support • Marital status • Race • Gender • SES (occupation, employment status, education) • Proxies of Socioeconomic Status • Health insurance status • Home address zip code average income
Race and Gender • Don’t adjust for automatically • Ideally adjust for variation in the patients’ physiological reserve and disease burden but not for variation in care rendered to patients
Propensity Scores • Useful when dataset is small, to conserve power • Need a good proxy to develop a propensity score • Ask: propensity for what? (Tx) • Published schema may include predictors you want to study separately • Best for non-randomized studies of treatment effect where you want to adjust for the factors that may have influenced the treatment choices
Your Assignment • Read other two articles • If you want extra feedback, e-mail me a 1-page proposal (abstract) for a database analysis related to your area of interest any time in next few years copelandl@uthscsa.edu