Conducting flawless research – the right question, the right design

1. Conducting flawless research – the right question, the right design Dr. Dick Menzies July 10th, 2015

2. The starting point in Research – a Problem • An unusual clinical observation • An unusual cluster (Outbreak of XDR in S Africa) • An unusual complication (Acquired Rifamycin resistance) • An observation from reported data • Temporal trend (Increase in TB in Foreign-born) • Geographic clustering (TB in hospitals) • Problems in diagnosis • Delayed diagnosis (cultures, DST) • Missed diagnosis (AFB smears in HIV infected) • Problems in treatment: • Treatment too long (Standard “short-course” therapy) • Treatment too toxic (INH) • Treatment too long AND too toxic (MDR, 9INH)

3. An unusual clinical observation – exampleExtensively drug-resistant tuberculosis as a cause of death in patients with TB and HIV in rural South AfricaNeel R Gandhi, Anthony Moll, A Willem Sturm, Robert Pawinski, Thiloshini Govender, Umesh Lalloo, Kimberly Zeller, Jason Andrews,Gerald Friedland • Among 475 patients with culture-confirmed TB: • 39% (185 patients) MDR-TB • 6% (30 patients) XDR-TB. • Of XDR-TB: • 45% had been previously treated for TB ; • 67% had a recent hospital admission. • 100% of tested for HIV were co-infected. • 98% died, with median survival of 16 days

4. From clinical problem (XDR outbreak) to Research question • Questions arising: • How (and why) did the patients acquire XDR? • Why the association with HIV co-infection? • Why the association with hospitalization? • Why was mortality so high? • Why was mortality so rapid? • Each of these questions needs a separate study, and unique design - to be answered. • WHICH question should be answered? • IS this the right problem?

5. Choosing the right study question Step 1: Definition of the problem • Who gets the problem, • How important is it - does it affect death? Costs? Quality of life? Step 2: Understanding the biology? • Manifestations - what organs are affected • Pathogenesis - probable or know • Transmission/hosts/reservoirs Step 3: make the question as precise as possible • (PICO or pseudo-PICO format)

6. Choosing the right study question“That year in the lab saved me a day in the library” Step 4. Review the literature (again) (and again) • This is often under-utilized - can avoid wasted time • Avoid repeating errors others have made • Do not study a problem that has been well described. • RCT of FDCs (have been 15), • Comparing positive TST vs IGRA (have been hundreds) • Know what has been described • Effect of standard treatment • Accuracy of standard diagnostics Step 5. Get the opinions of others • Speak to experts (or pseudo-experts) • Speak to providers • Speak to patients, community members, others affected

7. From Research qst to study design The biggest determinants of study design: 1) Time – investigators’ and patients’ 2) Money (“The biggest determinant of sample size is the budget”) 3) Resources – what do you have to work with? (Diagnostic Laboratory, Molecular Epi, Molecular Biology, Genetics) 4) Collaborators – who do you have to work with? What can they do? These 4 factors determine the kinds of studies you can do – which determines the question you can answer.

8. Hierarchy of study designs • Descriptive studies • Case reports, and case series • Reported data, • Prevalence surveys • Analytic studies • Observational • Cross-sectional • Ecologic studies • Case control studies • Cohort studies • Experimental studies • Uncontrolled trials • Randomized controlled trials • Individual level • Field trials = group level Increasing costs and complexity

9. Reported Data • Useful to: • Define incidence/prevalence • Identify geographic or temporal differences • Describe clinical characteristics • Describe outcomes. • Implicit comparison with general population • Risk factors can be identified. • Useful if data is COMPLETE and ACCURATE

10. Temporal trends may indicate clues to causal exposure Coal use and TB in USA: 1953 – 2003 Coal use ( ) and TB incidence ( ) .

11. Temporal trends may indicate clues to causal exposure Coal use and TB in China: 1978 - 2004 Total coal combustion ( ) Notified cases of TB ( ).

12. Ecologic Studies Advantages • Usually very easy and quick studies • Take advantage of already gathered data • Exposures • Diseases Disadvantages • Relationship may be due to completely unmeasured factors • VERY substantial potential for confounding

13. Directionality in research • Retrospective: Start with persons with disease and ‘look backward’ in time to ascertain exposures. • Advantages: Biggest is convenience – do not have to wait a long time, because disease HAS happened. Makes these studies much quicker to complete, and much cheaper. • Prospective: Start with a population and observe them ‘going forward’ in time. Exposures are measured first, and health events are measured afterward, as they occur. • Advantages: Biggest is accuracy of exposure, and certainty that exposure precedes disease.

14. Cross-sectional or Prevalence Studies General approach: Measure disease(s) and exposure(s) all at once in a population. Estimate effect: Prevalence odds ratio Advantages • Good for common/chronic diseases • Good for common exposures • Allows one to measure multiple disease or conditions and multiple exposures Disadvantages • Measurement of exposure may be difficult • Recall problems if long latency • May change over time (Alcohol, smoking, blood pressure) • Can not distinguish cause and effect • (Tobacco Industry defense)

15. Tuberculin (or IGRA) SurveysA special type of cross-sectional survey • Once TST or IGRA convert to positive with TB infection – they remain positive lifelong (some exceptions) • Cross-sectional survey – detects all with positive tests – from recent or remote • From prevalence of positive test at a given age – can calculate average annual risk of TB infection. • Can compare prevalence in different populations • If different ages, or different exposure periods can estimate trends in infection

16. University students in Brazil(All BCG vaccinated in infancy) Silva et.al. IJTLD2000; 4:420-426 Average ARI: Preclinical 0.2% Clinical 2.9% Incidence of TB in Brazil: 75/100 000

17. Example of Kaplan-Meier analysis: General Hospital Ventilation and time to TST conversion

18. Case Control Studies General Approach: • Measure exposures retrospectively in persons with disease (cases) and without disease (controls) • Estimate odds of exposure for disease (Odds ratio) Advantages • Relatively cheap and quick • Particularly useful for studying rare conditions • Or conditions with long latency Disadvantages • Controls, Controls, Controls • Very difficult to select proper controls • This is the source of most problems in case control studies • And is why they are generally considered weak evidence. • Difficulties of retrospective exposure assessment • particularly if long latency

19. Fitness costs of drug-resistance mutations in MDRTB: a household-based case-control study. Salvatore…. and Cohen T, JID 2015 • Cases: MDR index patients with at least 1 other HH member with same strain • Controls: MDR but no-one in HH with same strain • Matching: • Same number of contacts in House-Hold • Same extent of Drug Resistance • Same time to detection • Not matched: Extent of disease, Cavitation, smear, duration of symptoms, prior episodes, Size of house • Findings: katG Ser315Thr mutation associated (OR: 2.39; 95% CI: 1.21- 4.70). BUT combination of katG Ser315Thr & rpsL-Lys43Arg mutations was protective.

20. Nested Case Control study – exampleTB outbreak in Inuit villageBehr, Fox, Khan, Lee, Menzies & many others • Major ‘outbreak’ of TB in small village in N Quebec • 695/940 residents investigated as contacts during outbreak • Newly infected • 50 Confirmed disease, 19 probable disease • Case control study of housing, nutrition, lifestyle determinants • “Nested” within all 695 investigated as contacts • Case-control1: New Infection • Cases=Newly infected, Controls = Not infected • Case-control2: Disease • Cases=Confirmed or Probable, Controls = Infected but no disease

21. Cohort Studies General approach: • Start with a population free of disease. Measure exposures, and follow them all. • Estimate of effect: Risk ratio (Risk disease in Exposed/unexposed) Advantages: • Can measure many exposures, and many diseases • Temporal relationship clearer (cause before disease) Disadvantages • Long and expensive (often very $$$) • Good for common diseases (some cancers, cardiovascular). • But not for rare diseases or long latency • Also what if you fail to measure key determinants • (Solution = freezer)

22. Average Annual Incidence of Tuberculosis Among Navy Recruits By History of Household Contact Comstock, Edwards, and Livesay; Am Rev Respir Dis 1974; 110:576

23. Incidence of active TB after a positive IGRA The 5 largest studies from high burden countries(Rangaka, Pai, et al. Lancet Inf Dis, 2012)

24. Randomized Trials Advantages • Best method to evaluate an intervention • Best control of bias and confounding Disadvantages • Not easy or feasible for all interventions • Very Expensive and long • Not for studies of risk factors or natural history • Substantial refusal or drop-out rates can restrict generalizability • Population selected may not be representative • Younger adults with only one condition • Often exclude pregnant woman, kids, elderly! Solution – Biomarker? • 2 month culture conversion instead of fail/relapse

25. Five Phase 2 trials: 2 month culture conversion with FQN(all patients received 4 months HR in continuation)

26. Three Phase 3 trials of 4 month regimens with FQN

27. Experimental Community or Field Trials General Design • Pick an intervention to be applied at a community level • Fluoride in water, public education, vaccination • Find several communities or population groups • Apply intervention to some and not others • Randomly again • Measure outcomes at population or group level

28. Cluster randomized Trials – exampleEffect of improved tuberculosis screening and isoniazidpreventive therapy on incidence of tuberculosis and death inpatients with HIV in clinics in Rio de Janeiro, Brazil: a steppedwedge, cluster-randomised trialBetina Durovni, Valeria Saraceni, Lawrence H Moulton, Antonio G Pacheco, Solange C Cavalcante, Bonnie S King, Silvia Cohn, Anne Efron,Richard E Chaisson, Jonathan E Golub • 29 HIV clinics in Rio de Janeiro. • Staff trained in TB screening, TST and INH. • Clinics randomly allocated to date of starting the intervention. 2 clinics started every 2 months starting from Sept 2005, until Aug 2009 • Outcome: TB incidence +/- death

29. A randomized trial to compare 4 months Rifampin vs 9 months INH for the treatment of LTBI Phase 1: Compliance and completion Completed in 2003 Phase 2 – Adverse events and costs Completed in 2007 Phase 3: Efficacy and effectiveness Enrolment completed in 2014: 840 children, 6020 adults Follow-up will finish in 2016 Publication in ???

30. Thanks