Operational research: methods and examples

1. 1 Udo Buchholz, WHO/Stop TB/TME Operational research: methods and examples

2. 2

3. 3 In the beginning there was ... a question

4. 4 Description of defaulters in Russia1 Profession: unemployed: 26%, labourers 21%, students of vocational schools 19%, disabled 7% Education: incomplete secondary education: 70% Residence: homeless 5%, >5km away from treatment site 26% Behavioural risk factors: alcoholism 44%

5. 5 Are these variables risk factors for default? � use of patient cohort for cohort study

6. 6 Social support system Examples from different oblasts: Food incentives Hygienic kits Free transportation Psychological support ....

7. 7 Adherence with social support

8. 8 More examples "Defaulting from anti-tuberculous treatment in a teaching hospital in Rio de Janeiro, Brazil" (IJTLD 2004) "A concurrent comparison of home and sanatorium treatment of PTB in South India" (BWHO 1959) " 'Lost' smear positive PTB cases: where are they and why did we lose them?" (IJTLD 2005)

9. 9 Determinants of a study Problem or question Data available Funding and staff available Political or hierarchical support ? Type of study

10. 10 Which scientific methods can we use? - Type of studies Descriptive studies Analysis of surveillance data Ecological study (correlational) Cross-sectional survey Analytical studies Observational (case-control study, cohort study) Experimental Other E.g. capture-recapture study

11. 11 Example: Surveillance data reveal large provincial differences of ss+ TB/all PTB

12. 12 No. of slides/patient is correlated with proportion of ss+/PTB

13. 13 Ecological comparison (correlational) Correlation of aggregated or group data Association on the individual level is unknown and may be different Many relationships on global level are strictly speaking of ecological nature

14. 14 Example of an "ecological" comparison: The prevalence of HIV in TB patients (y-axis) against the prevalence of HIV in adults (x-axis).

15. 15 Cross-sectional survey Collection of representative data Based on sampling size calculations, sampling frame and sampling scheme Simple random sample Systematic sampling Cluster sample (design effect!)

16. 16 Surveys are frequently used in TB epidemiology Sampling universe is the population: Prevalence surveys Tuberculin skin test surveys Sampling universe is "all TB patients" Proportion of diagnosed new TB patients with HIV test Sampling universe is the number of culture positive TB patients Drug resistance surveys

17. 17 Analytical studies Are used to identify risk factors or other forms of "exposure" and their association with an outcome, e.g. death, default, etc. Make use of a comparison group Hypotheses are tested Null hypothesis: "There is no association of exposure and outcome" or: "Exposure and outcome are independent" ? We then calculate the probability that this is true based on the data

18. 18 Case control study Starts with a group of cases, i.e. with a certain outcome, that is consistent with a case definition The case definition must be specific in regards to time, place and person E.g. "a person with smear positive TB diagnosed in Geneva city in 2004" Then select a group of persons without the outcome from the same population, here for example the general population From the case definition it follows: "a person without TB living in Geneva in 2004"

19. 19 Case control study: ascertainment of exposure status After identification of cases and controls the exposure status preceding the outcome is investigated E.g.: income (high versus low) Thus, the directionality is usually retrospective

20. 20 Selection of controls Imagine the cohort from which the cases would have arisen Or: Would the control have been a case if he/she had had the outcome in question? Example: cases of rare kidney disease in the Mayo clinic

21. 21 Typical control options Friend controls Neighbourhood controls Physician controls Hospital controls Population-based controls Consider: Selection bias Feasibility

22. 22 2 x 2 table (CCS (1)) 50/1000 ss+ TB cases (5%) were poor, but only 5 of 2000 (0.25%) among the non-TB persons ? Ss+ TB patients were 20 times more likely than the general population to be poor, however ... ee

23. 23 2 x 2 table (CCS (2)) The chances of ss+TB patients to be poor is expressed as the odds = probability of poverty / prob of rich = 50/1000 / 950/1000 = 0.053 The odds of non TB persons for poverty is therefore:5/2000 / 1995/2000 = 0.00251 The ratio of the two odds (the odds ratio (OR)) is: 0.053/0.00251 = 21 ee

24. 24 Use of case control studies When type of outcome is rare We can examine >1 exposure Usually relatively quick and inexpensive Disadvantages: Not useful for rare exposures Because exposure is in the past: watch out for recall bias Selection of cases and controls often not straightforward (selection bias)

25. 25 Cohort study Starts with a group of people or a population that can be divided in two groups based on a defined exposure which some have and some don't The groups are then followed-up and an outcome is counted A case definition is still important The directionality is usually forward, but can also be backwards (retrospective cohort study)

26. 26 2 x 2 table (cohort study) We follow 100 low income TB patients and 200 high income TB patients up for adverse outcomes It turns out that 20 of 100 (20%) poor have a bad outcome versus 10 of 200 (5%) of the rich. Thus, the poor are 4 times more likely to have an adverse treatment outcome. Measure of association is the risk ratio (RR) = 0.2/0.05 = 4

27. 27 Use of cohort studies When exposure is rare We can examine >1 outcome The outcome measure for the strata is an incidence rate or (cumulative) risk and the overall point estimate the rate ratio or risk ratio (RR) Disadvantages: Not suitable for rare outcomes Not ideal for outcomes in the far future (unless you have much time or lots of scientific altruism) Watch out for loss to follow-up (they may represent a certain category of patients)

28. 28 The TB quarterly "cohort" Pro- or retrospective cohort study  (Nested) case-control study

29. 29 2 x 2 table

30. 30 Cohort study

31. 31 Case control study

32. 32 Analytical study: experimental / intervention study Prospective Use of a cohort Exposure is usually an intervention, a drug or vaccine Patients are ideally randomized which guarantees minimisation of bias Example: IPT intervention study in South African gold miners; recruitment in random sequence; comparison before / after IPT phase

33. 33 Steps for a OR protocol (1) Starts with a problem or question: e.g. "Why is there no decline in urban TB in Japan?" Gathering of information: Analyse exhaustively routinely collected (surveillance) data and disaggregate also by province etc Talk with stakeholders Investigation of the literature Contact other countries Develop a hypothesis Depending on money and staff available: generate a protocol; but this can also be used to generate money and staff

34. 34 Steps for a OR protocol (2) Writing of the protocol: You can structure it similar to a scientific paper Introduction/rationale Objective Methods (study type, sample size, case definitions used, inclusion/exclusion criteria, training, data collection, data entry (double entry?, data validation), quality control, lab methods, method of analysis) Ethical considerations Results: shell tables, expected figures Timeline Budget Appendices (questionnaire, maps, consent form...) Good idea to do a pilot: feasibility, cost, first crude data ?verify sample size assumptions

35. 35 Now it is up to you