The good, the bad and the ugly (evaluating empirical climate and health studies)

1. The good, the bad and the ugly(evaluating empirical climate and health studies) 18 July 2006 Sari Kovats Lecturer, Public and Environmental Health Research Unit, LSHTM

2. Outline • Basic environmental epidemiology • Study designs • Data issues (exposure and outcome measures) • Systematic reviews • Discuss abstracts • Climate and health studies • Time series (again) • Inter-annual variability • Trends: early effects of climate change?

3. Environmental epidemiology • Disease driven approach • Identification of disease endpoints, followed by the examination of potential hazards in effort to establish causation • Exposure-driven approach • Identifying potential hazards and then examining their effects on human health

4. Exposures and outcomes • In an epidemiological study there are: • (a) the outcome of interest • (b) the primary exposure (or risk factor) • of interest • (c) other exposures that may influence • the outcome (potential confounders)

5. INTERVENTION (EXPERIMENTAL) We observe only We allocate exposure EPIDEMIOLOGICAL STUDIES OBSERVATIONAL (NON-EXPERIMENTAL)

6. EPIDEMIOLOGICAL STUDIES INTERVENTION (EXPERIMENTAL) OBSERVATIONAL (NON-EXPERIMENTAL) DATA FROM GROUPS DATA FROM INDIVIDUALS DATA FROM GROUPS DATA FROM INDIVIDUALS COMMUNITY TRIAL (f) CLINICAL TRIAL, INDIVIDUAL FIELD TRIAL (g) DESCRIPTIVE (a) ANALYTIC DESCRIPTIVE ANALYTIC ECOLOGICAL STUDY (b) CROSS-SECTIONAL STUDY (c) COHORT STUDY (d) CASE-CONTROL STUDY (e)

7. Ecological studies use.. • Average exposure for a group • E.g. temperature, rainfall • A population measure of outcome – • Risk or Rate • Counts of events

8. Ecological studies • Strengths • Quick and relatively inexpensive • Simple to conduct • Availability of data from surveillance programs and disease registries • Limitations • Difficulties in linking exposure with disease • Limitations in controlling for potential confounding factors • time series avoids some confounding issues…. • “Ecological fallacy” – making a causal inference about an individual phenomenon or process on the basis of group observations

9. Situations where group level variables may be better • Exposures without much within group variability (salt consumption in U.S.) • Exposures which can only be measured at population level • Herd immunity in studying infectious disease (vaccination levels may be more informative than individual behavior) • Social capital • Climate

10. Cross-sectional studies • also called survey or prevalence study • measures exposure and outcome at the same point in time • involves disease prevalence • usually involves random sampling and questionnaire measurement • cannot distinguish whether hypothesized cause preceded the outcome • Spatial/geographical studies: links environmental data with survey data

11. Case control studies • Example. Chicago heat wave 1999 Naughton et al. • Cases: 63 deaths from heat stroke during heat wave • Control – 77 alive controls, matched on age and neighbourhood. Cases - • Range of social, environmental risk factors for heat wave deaths • “Working air conditioner at home” Odd Ratio 0.2 (95% CI 1.0, 0.7) • Must consider selection of controls • Cannot calculate rates or attributable risks

12. Bias • Selection bias • how were subjects selected for investigation • how representative were they of the target population with regard to the study question? • Information bias (recall bias) • what was the response rate, and might responders and non-responders have differed in important ways? • how accurately were exposure and outcome variables measured? • Random vs. systematic errors – have different implications for final estimate

13. Chance • Hypothesis testing • p-value • Precision of estimate • Confidence intervals • Assumes estimates/data are unbiased • Beware of multiple testing!

14. Smoking during pregnancy potential confounding factor Confounding Question: Is alcohol consumption during pregnancy associated with increased risk of low birthweight Low birth weight outcome Alcohol during pregnancy exposure

15. Time series- consider time varying confounders High temperature exposure Daily mortality outcome Air pollution potential confounding factor

16. Epidemiological data • Routine sources of health data • Vital Registration (births, deaths) • Hospital statistics (admissions, clinic attendance) • Primary care • Laboratory data (notifiable diseases) • Health Surveys • Epidemiological Studies (cohort or longitudinal studies, cross-sectional surveys) • Demographic and Health Surveys (low and middle income countries)

17. Applications of different observational and analytical study designs 1 Unless the sampling fraction is known for both cases and controls; i.e. unless the proportion of cases and proportion of controls sampled from the population is known.

18. Strengths and weaknesses of different observational • analytic study designs 1. But high if you are not aware of, or do not measure, confounding factors

19. Reviewing the literature • Develop a clear written Search strategy • Clear research question • Inclusion/exclusion criteria • Search >1 database, plus hand searching, snowballing.. • Some assessment of quality of studies • Limit to peer review published articles only. • Beware publication bias • Language bias • Climate change bias! – editors like novel or hot topics

20. Reviews- you need a “search strategy” Ahern et al. 2005

21. Quality control: flooding and health studies • Clearly stated hypothesis • Individuals included in the study and how they were selected (i.e. using some form of randomisation or probability sampling procedure) • Sample to include those who were affected by the flood event, and those who were not. The latter are often referred to as the ‘control’ or ‘comparison’ group • Data collection in both the pre- and post-flood period. Prospective data collection is given higher weighting than retrospective data collection, as the latter is particularly susceptible to recall bias • Results should include p-values or confidence intervals, and limitations of the study should also be highlighted • Clinical (e.g. mental health outcomes) or laboratory (e.g. leptospirosis) diagnosis is given greater credence than self-reported diagnosis. Ahern et al. 2006 Flood Hazards and Health. EarthScan Book.

22. Abstracts • Identify • Exposure measure • Outcome measure • Study design • Measure of uncertainty? • Confounders?

23. Climate and health studies

24. Three research tasks Empirical studies [epidemiology] Risk Assessment Scenario Sensitivity Mechanisms Responses Causality? Early effects? detection attribution 1970s=? future present

25. IPCC: different types of evidence for health effects • Health impacts of individual extreme events (heat waves, floods, storms, droughts); • Spatial studies, where climate is an explanatory variable in the distribution of the disease or the disease vector • Temporal studies (time series), • inter-annual climate variability, • short term (daily, weekly) changes (weather) • longer term (decadal) changes in the context of detecting early effects of climate change. • Experimental laboratory and field studies of vector, pathogen, or plant (allergenic) biology.

26. Exposures: climate/weather parameterization • Long-term changes in mean temperatures, and other climate "norms" • climate change requires changes over decades or longer. • Interannual climate variability • including indicators of recurring climate phenomena – [El Niño years or SOI] • Short term variability [weather] • including monthly, weekly or daily meteorological variables. • Isolated extreme events • simple extremes, e.g. of temperature/precipitation extremes. • complex events such as tropical cyclones, floods or droughts.

27. Time series analysis: weekly Salmonellosis and Temp Sporadic cases only Outbreaks removed Kovats et al. 2004

28. Results by age: Relative risks for 5 countries, same threshold, by age group

29. Time lags/time windows • Acute events • Cause before effect (temporality) • Use literature to hypothesise the time lags (days) • Need to address incubation period for infectious diseases • 1-2 days salmonellosis, 7-14 days typhoid fever • Delays in reporting process • Critical time windows • Aetiological relevant exposure windows • E.g. childhood exposures to UV, in utero exposures • Need to address latency periods (?years) between exposure and outcome.

30. ENSO and health • Large scale climate phenomenon • Irregular occurrence • Climate variability can be important driver of year to year variation in disease. • ?driven by precipitation • Insight into effects not evident at local scales • rainfall, predator balance (Venezuela) • Applications • Epidemic prediction using seasonal forecasts • Effects of increased frequency of ENSO events under climate change • But cannot directly assess effects of progressive warming from direct extrapolation of ENSO-health relationships

31. Systematic review – ENSO and health • Criteria for inclusion. • Published in peer reviewed journal • Original research article using epidemiological data. • Quantified association with an ENSO parameter (e.g. El Niño year, SST, SOI or other index). • The outcome was an infectious disease in humans. • The time series included more than one El Niño event.

32. Systematic review – ENSO and health

33. Evaluating ENSO-health studies • Need to identify correct climate “driver” • Biological mechanisms • Alternative explanations, • e.g. cyclical changes in immunity • Hay et al. Inter-epidemic periods in mosquito-borne diseases • Dengue – new serotypes on population • Limited data series - • need more than 1 event.. • Most appropriate geographical aggregation • Disease data is of uncertain quality (and may not be disease-specific)

34. Tick-borne Encephalitis, Sweden: 1990s vs 1980s: winter warming trend Early1980s Mid-1990s White dots indicate locations where ticks were reported. Black line indicates study region. (Lindgren et al., 2000)

35. Evaluating early effects: Criteria.. • What constitutes evidence of early effects? • To detect changes in distribution or phenology/seasonality, sample sizes should be maximised by studying multiple species/diseases/populations. • To detect polewards or altitudinal shifts in vector or disease distributions, studies should extend across the full range (Parmesan 1996), or at least the extremes of the range. (Parmesan et al. 2000), so as to exclude simple expansions or contractions. • Given the natural variability in both climate and biological responses, long data series are needed (i.e. > over 20 years). • Variability in the climate series (e.g. year to year) should correspond to variability in the health time series. • Analyses should take into account, as far as possible, otherchanges that have occurred over the same time period which could plausibly account for any observed association with climate. Kovats et al. 2001

36. Surveys up to 1940 Surveys up to 1960 Surveys up to 1980 Surveys up to 2000

37. Summary I: Get the study right • 1. Correct design • 2. As accurate a measure of exposure and outcome as possible • 3. Control confounding

38. Summary II: Evaluating • Reviews must be systematic and thorough • Epidemiological literature must be evaluated • Climate and health studies should have.. • clear hypotheses • plausible biological mechanisms • reported validity and precision

39. Summary III: Criteria • Good studies……………. • measure and control confounders; • describe the geographical area from which the health data are derived; • use appropriate observed meteorological data for population of interest (the use of reanalysis data may give spurious results for studies of local effects); • have plausible biological explanation for association between weather parameters and disease outcome; • remove any trend and seasonal patterns when using time-series data prior to assessing relationships; • report associations both with and without adjustments for spatial or temporal autocorrelation.

40. Thank you!