Improving Faculty Research: Useful Tips from a Statistician

1. Improving Faculty Research: Useful Tips from a Statistician Alok Dwivedi, Ph.D. Associate Professor Biostatistics & Epidemiology Texas Tech University Health Sciences Center El Paso alok.dwivedi@ttuhsc.edu

2. Outline • Guidance for planning clinical research • Formulating a research question (PICOT and FINER) • Selecting an optimal study design (4 standard designs) • Essential methodological steps required in conducting and publishing a research study • Proposal writing, data collection sheet, data management, analysis, presentation and publication

3. Research

4. Why do we need to do clinical research? EBP

5. Potential hurdles and solutions in research • Potential Solutions • Research is an essential component in academic medical schools • Heavy clinical services (time) • 1. Clear public policies by institute to monitor and improve clinical research • 2. Financial incentive, mentoring programs and protected time for clinical faculty • 1. Institute must have short courses/training opportunities in research methodology • 2. Research training and projects should be required components of training programs • Lack of basic knowledge of clinical research methodology • 1. Increase funding for clinical research fellowships and mentors • 2. Enlist established clinical or research mentors as collaborators • Lack of leaders/mentors • 1. Reach out to researchers in other disciplines such as basic scientists, epidemiologists, behavioral scientists, public health researchers • Lack of collaborators • 1. Join an ongoing project • 2. Look for gaps in the literature, recommendations, and limitations • 3. Attend lectures and conferences • Lack of research ideas • 1. Develop and improve efficiency of central research support (biostat & research design, mentoring, data management, coordinator, editorial review, seed funding etc.) • Lack of research infrastructure

6. Identify interests and area of research • Qualitative improvement studies • SQUIRE (Standards for QUality Improvement Reporting Excellence, Ogrinc et al, 2008) • Screening/prevention studies (cross-sectional, cohort, interventional studies) • A reader's guide to the evaluation of screening studies (Earle and Hebert, 1996) • Diagnostic studies (cross-sectional, cohort) • STARD (Statement for reporting studies of diagnostic accuracy, Bossuytet al, ‎2003) • Descriptive, prognostic or etiology studies (observational study) • STROBE (Strengthening the reporting of observational studies in epidemiology, Elm et al, ‎2008 ) • Therapy/ harm (clinical trial) • CONSORT (Consolidated Standards of Reporting Trials, Schulz et al, ‎2010) • Meta analysis and systematic review • PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses, Moher et al, 2009) • Animal studies (excremental designs) • Animal Research: Reporting of In Vivo Experiments (Kilkenny et al, 2010) • Economic evaluations • CHEERS (Consolidated Health Economic Evaluation Reporting Standards, Husereau et al, 2013)

7. Identify your role and form a team Team Science Identify researchers with different skills

8. Work-out on major resources • Primary data collection (survey development, data collection cost, time, IRB, data storage, statistical analysis cost, etc.) • Retrospective data collection (data extraction and statistical analysis cost) • Using own electronic health records (EMR) • Using publically available databases • Using published studies

9. Clinical research using databases • Many research questions can be answered quickly and efficiently by performing secondary data analysis of existing databases • Rapid and inexpensive (mostly no IRB requirement and data collection), population-based, more generalizability • Metabolic syndrome, cancer, infant & child health, health disparity etc. • Available sources • NHANES (National Health and Nutrition Examination Survey), CORI (Clinical Outcomes Research Initiative), NCHS (National Center for Health Statistics), PUDF (Texas Inpatient Public Use Data File-PUDF), SEER (Surveillance, Epidemiology, and End Results) • Association of metabolic syndrome with human papillomavirus (HPV) (published in CEBP) • Association of metabolic syndrome with hepatitis C virus (HCV) (Preetha Rajkumar, manuscript under-preparation) • Effect of metabolic syndrome on infant mortality (Ivette Gonzalez-Diaz, work-in-progress) • Association of immigration status with allostatic load (Alex Quesada , manuscript under-preparation) • Association of unhealthy behavior with allostatic load and depression (Luis Jauregui, work-in-progress) • Metabolic syndrome and depression using NHANES (Sherly, work-in-progress)

10. Clinical research using published studies • Many contradictory findings in published studies may be solved by analyzing published studies: • No IRB requirement, rapid and inexpensive, more generalizability • Systematic reviews and meta-analyses • Well designed systematic reviews and meta-analysis produce high level of evidence. • Breast Parenchymal Enhancement (BPE) at MRI and breast cancer (Chris Thompson, under review) • Effect of metformin on metabolic syndrome (Bernardo Tamis, under preparation) • Cut offs for mammographic breast densities in screening breast cancer (Diego, manuscript under preparation) • Behavioral outcomes of patients with or without treatment for unruptured cerebral aneurysm (Kabeer Masih,work-in-progress) • Effect of exercise on left ventricular mass index (Aaron, published)

11. Research Study Proposal writing Develop research question Decide most appropriate study design Compute sample size Finalize research question, study design and sample size Conduct study/CRF/ Questionnaire Analyze data Present data Data Management Literature Review

12. Formulating a Research Question Step 1

13. Development of a research question • Goal is to find an important research question that can be developed into a feasible and valid study plan. • Sources of research questions: • Literature review • Being alert to new ideas or techniques (conference, imaging, techniques for genetic analysis) • Keeping the imagination roaming (careful observations of patients, teaching, imagining new answers to old questions/previous research)

14. Elements of a research question • Obesity or metabolic abnormalities with maternal and fetal health (broad topic of interest) • PICOT (T is optional) is now a widely recommended strategy for framing research questions. POSE Population Exposure Outcome Study design S Study design Procedure or protocol under which study would be carried out

15. PICOT (S)/POSE • Population (P) • Age • Parity • Comorbidity (T2DM, HTN) • Ethnicity (Hispanics or Mexican American, etc.) • Setting (in specific country population) • Intervention(I/E) & control (C) • Obesity, cholesterol, diabetes, triglyceride, hypertension • Underweight • Overweight • Actual pre-pregnancy or during pregnancy BMI • Normal weight • Metabolic syndrome • Outcome (O) and type of outcome • Fetal death • Miscarriage • Stillbirth (antepartum, intrapartum) • Infertility • Neonatal death (early or perinatal or post) • Infant death • Cesarean delivery • Fetal/neonatal complications (macrosomia, dystocia, obesity) • Operative morbidity • Time (T) optional • Within a year of delivery • Within 28 days of delivery • Within 7 days of delivery • Study design (S) • Randomized trials • Observational studies (retrospective or prospective cohort studies, case-control study, cross-sectional) • Meta-analysis

16. Carrying out the literature review • Process of searching information related to the research topic: • Finding knowledge gaps • Generating a good question • Accurately defining a problem • Identifying proper methodologies to be used

17. Types of review studies

18. Choosing best question F I N E R Ethical Relevant Novel Interesting Feasible • Follow ethical guidelines • Regulatory approval from IRB • Through literature search • Confirm new finding or extend previous findings • Guidance from mentor or experts • Appropriate design • Number of subjects • Technical expertise • Time, staff, funding • Manageable scope • Scientific knowledge • Influence on clinical practice • Furthering research and health policy • Interesting as a researcher or collaborator • Motivation to make it interesting F I N E R

19. Finalize research question • Population All live-born singleton infants ≥20 weeks’ gestation born to women • p • I • Intervention/Exposure Metabolic Syndrome(MetS) • C • Comparative group No MetS • Outcome Infant mortality (death of a live- born infant before age 1 year) • O • T • Time In a year of follow up • Study design Meta-analysis of cohort studies and a secondary analysis of NCHS database • S

20. Choosing a best study design Step 2

21. Why to know about the study design? • Level of evidence (high, moderate, low) • Kind of data (Prevalence/ Incidence) • Sample size • Measures of association • RD, RR, OR, HR, RRR • Statistical analysis (linear, logistic, Cox) • Making tables (row-wise, column-wise) • Interpretation (association, causation)

22. Step 2: Choose study design Study Design Analytical Descriptive Experimental /Interventional Observational Case report, case-series, correlation study Cross-sectional Case-control Cohort Clinical trial Community trial Field trial Chart review is not a study design

23. Level of evidence Meta-Analysis of clinical trial Highest level of evidence Clinical trial Highest level of evidence Cohort Moderate level of evidence Case-control Moderate level of evidence Cross-sectional Weakest level of evidence Case report, case-series, correlation No evidence

24. Summary E:exposure; O: Outcome

25. Computing sample size Step 3

26. Requirements • Sample size is the statistical calculation of the number of subjects that are required to meet the objectives • Required in any studies for ethical, scientific, and economic reasons • Fewer than needed sample size would lead to an underpowered study • Fewer than needed sample size would lead wasting resources • Sample size estimation requires inputs from PIs of the study: • What would be the expected mean & standard deviation / proportion of each outcome in control group? • What would be the anticipated minimum clinically relevant (increased or decreased) value in exposed group for each outcome ?

27. Important formula • Common outcomes • Proportion (p) • Mean (p) • Estimation of proportion • Estimation of mean • Comparison of two proportions • Comparison of two means • SD: average standard deviation • d= difference in means • p=average proportion of two groups • q=1-p • d= absolute difference

28. Proposal/Protocol writing • Purpose is to obtain ethical approval, commitment, funding. • Includes everything except for results and discussion • No universal format (depends on the institution) • Structured outline for each step in the research process • Significance (how important topic is?) • Impact/Scope (how much generalizability- broad, narrow, local) • Objectives (too broad to reach conclusive results) • Robust & feasible approach (study design, sample size, statistical analysis, data collection & management) • Innovation (in terms of research question, approach, methodology, population) • Environment & team (expertise & collaboration) • Anticipated difficulties and proposed alternatives • Optimum budget (allocation for statistician as collaborator or consultant, required expertise) • Clear justification of each step

29. Recruitment strategies, Data collection and management Step 4

30. Recruitment strategies • Setting: Description about the location or area where the study will be carried out: • Type of institution (tertiary care, specialized unit etc.) • Load of patients • Staff availability • Community served • Services provided • Time period • Inclusion and exclusion criteria

31. Eligibility criteria (inclusion and exclusion) R: Opposite of inclusion should not be in exclusion criteria

32. Inclusion and exclusion • To assess the effect of neoadjuvant chemotherapy on overall survival in breast cancer patients • Inclusion criteria: Characteristics of subjects that are essential for their selection to participate in the study • Criteria that will ensure the homogeneity of the sample, feasibility, lower its costs, and reduce confounding • Age >25 year, Hispanic , Stage I-III Exclusion criteria: Characteristics of subjects that may threat internal and external validity of the study (bias the study results) • Criteria that will minimize ethical concerns, increase safety of patients, effect of intervention difficult to interpret • Impossible to have outcome of interest • Impossible to measure the outcome/exposure of interest • Unethical to include a subject • Individuals do not have reliable information • Recurrent cancer, pregnant women, inoperable advanced, evidence of metastases, patients already received some kind of treatments

33. Bad data entry Variables are not in one row No unique IDs No group variable to separate groups Inconsistent entries (different units, text and numbers) Inconsistent entries for missing Two variables in same column Inconsistent formats of date Aggregated data are entered

34. Appropriate data entry

35. Problems with entries in excel sheet • Allows you to enter measurement unit in numeric variable • Allows you to enter the same data in inconsistent ways • Allows you to enter same patient’s record more than one time • More than one observation per cell • Does not prevent typos • Missing values are indicated in inconsistent ways • Allows you to enter data in different units • Allows you to enter data in text and inconsistent ways • You can accidently sort only one column • It’s easy to accidentally change the data on the wrong row.

36. Summary • Pick the tool that is appropriate for the data needs of the project • Use REDCap • For a large survey study • For a longitudinal study with many time points • For complex data entry forms • Multisite study or multiple user access • Clinical trial • Use Excel • For a small prospective study • For a retrospective study • For easy data entry forms

37. Steps in data entry

38. Data Analysis Step 5

39. Common mistakes in data analysis and interpretation • Inappropriate use of statistical tests and reporting. • Use robust methods, each test has some assumptions, adjusting for confounders • Report sufficient information to interpret or apply results to your patients • If p-value is not significant then authors conclude that there is no difference between groups (WRONG). • Absence of evidence is not evidence of absence • We can only conclude that we failed to demonstrate the difference between two groups. • Limited studies provide correct interpretation of 95% confidence interval or report it. • 95% confidence interval provides the range where the true difference between groups can lie. • A difference could be statistically significant but has no clinical relevance. Often the interpretation of significant difference roams around p-value. • Very rarely discussion is made that how much the result is conclusive and useful and add value to the body of evidence. • Absence of significant correlation is often interpreted as no correlation between two or more variables. • Many studies fail to explore effect modifiers and mediators.

40. Steps in data analysis

41. Reporting Step 6

42. Manuscript writing • Title • Should be catchy, include PICOT/POSE, include key aspects of study design • Abstract • Concise summary of the manuscript • Easy to understand and broadly appealing, informative but not too detailed • Read two-three times to get overall theme of the paper • Introduction • The context, relevance, or nature of the problem, question, or purpose (hypothesis) • Read to assess purpose and hypothesis of the study (judge novelty and relevance) • Methods • All the necessary details to replicate the study • Evaluate given the materials, can you replicate the study? • Evaluate whether used methods are appropriate and up to date, if not acknowledge in the limitations

43. Manuscript writing • Results: • Write results which are consistent with materials and methods section, do not over-interpret or claim what you have not done, no references, no discussion • Discussion: • Findings are consistent with the study objectives elucidated in the last paragraph of the introduction • Describe how the results are consistent or not with similar studies and discuss any confounding factors and their impact, do not repeat the study findings/results • Discuss strengths and limitations of the study • Discuss generalizability of the study findings in view of • Validity: closeness to the truth • Results (impact) : size of the effect • Relevance (applicability): usefulness in our clinical practice • Conclusion: • Present a concise and clear “take home” • Report conclusions that are consistent with the results

44. Summary

45. Best Practice

46. Level of problems in statistical consulting Many mistakes and completely discouraged Contact several days before proposal/analysis is due Not fixable More common Statistical adjustments not always work Fixable but time consuming task Wastage of time and effort Statistician needs to understand your area and needs input Fixable but do not like Fixable but discouraged Less common Best practice

47. References • Bandholm T, Christensen R, Thorborg K, Treweek S, Henriksen M. Preparing for what the reporting checklists will not tell you: the PREPARE Trial guide for planning clinical research to avoid research waste. Br J Sports Med 2017 Oct;51(20):1494-1501. • Brannan GD, Dumsha JZ, Yens DP. A research primer: basic guidelines for the novice researcher. J Am Osteopath Assoc. 2013 Jul;113(7):556-63. • TamimH, Armenian H, Onazi M, Shanafey S. Introduction to clinical research for residents. Saudi Commission for Health Specialties, 2014 • http://www.icmje.org/ethical_lauthorship.html • Thabane L, Thomas T, Ye C, Paul J. Posing the research question: not so simple. Can J Anesth/J Can Anesth (2009) 56:71–79. • Riva JJ, Malik KM, Burnie SJ, Endicott AR, Busse JW. Endicott, Jason W. Busse. What is your research question? An introduction to the PICOT for clinicians. J Can Chiropr Assoc. 2012 Sep; 56(3): 167–171. • Data management. Adopted from Colorado Clinical and Translational Sciences Institute. • Bad data entry, adopted from Scott (VandyBiostat). • Schulz K. F, Altman D. G, Moher D, the CONSORT Group (2010) CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ 340: c332. • Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, et al. (2003) The STARD statement for reporting studies of diagnostic accuracy: Explanation and elaboration. ClinChem 49: 7–18. • Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2008) Preferred reporting items for systematic reviews and meta-analyses: The PRISMA Statement. PLoS Med 6: e1000097. 10.1371/journal.pmed.1000097. • Von Elm E, Altman DG, Egger M, et al. The Strengthening the Reporting of Observational Studies in Epidemiology [STROBE] statement: guidelines for reporting observational studies. J ClinEpidemiol. 2008;61:344–349. • Giancarlo Maria Liumbruno, Claudio Velati, Patrizio Pasqualetti, Massimo Franchini. How to write a scitific manuscript for pulication. Blood Transfus. 2013, 11(2): 217–226.