231 likes | 344 Views
Interpreting numbers ScotPHO training course March 2011 Dr Gerry McCartney Head of Public Health Observatory Division NHS Health Scotland gmccartney@nhs.net. Approaching numbers: some questions to ask. 68% of Doctors don’t listen to their patients. What is being counted? Definitions
E N D
Interpreting numbers ScotPHO training course March 2011 Dr Gerry McCartney Head of Public Health Observatory Division NHS Health Scotland gmccartney@nhs.net
Approaching numbers: some questions to ask 68% of Doctors don’t listen to their patients • What is being counted? • Definitions • Type of numbers - counts, means, etc. • Who/where (population): what is the denominator? • When (time): what time period do they cover? • How (source): where did they come from? • Why were they produced: is there an agenda? SMOKERS ON PILL DOUBLE STROKE RISK
Incidence and Prevalence • Dealt with in more detail later in course • Incidence describes the number of new cases in the population over a period of time • Prevalence describes the number of cases present in a population at any one point in time
Framework for interpreting numbers Could your interpretation be affected by either: • Error • Chance • Confounding (the mixing of two effects) • Bias (systematic departure from truth – either deliberate or unintentional)
Example: COPD (lung disease) variation • Is COPD more common in Board A or Board B? • Errors (e.g. different definitions used in each Board)? • Chance (e.g. no confidence intervals used)? • Bias (e.g. are there systematic differences in how disease is recorded)? • Confounding (e.g. are there mixed effects – such as age structure)?
Sources of error: • Mistakes in data collection, data recording, data storage, data transmission • Coding errors, transcription errors • Can be random or systematic • Do the numbers add up? • Are the number plausible?
Chance and interpreting numbers • Most figures report data for a sample from a larger population • A different sample would give a different result • Year to year fluctuation can be due to chance • The size of the sample dictates the degree to which a difference is likely to be due to chance • Confidence intervals and p-values give estimates of the precision of a value • E.g. Relative risk of heart disease amongst diabetics is 7.4 (95% CI 6.5-8.6) means that there is a less than 1 in 20 chance of the true value lying outwith the range 6.5 to 8.6
Bias – identification and interpretation • Bias is a systematic alteration of figures away from the true value Examples • Selection bias – critically appraise sampling strategy, loss to follow-up, response rate • Information bias – completeness of data, calibration, participant self-report, recall time • Publication bias – think about a funnel plot
Confounding: when separate effects are mixed together • In this example, the effect of location is mixed with (confounded by) the effect of age • The population of Western Isles is older so has higher rates of CHD admission • Is CHD more common in Western Isles after taking age into account?
Methods for dealing with confounding Design • Randomisation (only for experimental studies) • Restriction (e.g. narrow the comparison groups by age, sex, ethnicity, socioeconomic status) • Matching Analysis • Stratification (i.e. compare sub-groups, but has dangers) • Standardisation* • Multivariate analysis* *dealt with in more detail elsewhere
Standardisation: brief interpretation • A method of “removing” the effect of other factors to allow a “fair” comparison • The other factors are most commonly age and sex, but standardisation can be used for other factors • Standardisation shows the rates you would get if the population had a “standard” age and sex structure
Standardised Mortality Ratios (SMRs): • This is a comparison of mortality in a population with a ‘standard’ population taking account of age structure • The standard population is allocated a value of 100 for whatever the mortality rate is • The age and sex standardised mortality of the population of interest is then divided by that in the standard population to give a figure for comparison with the 100 • An SMR of 150 indicates that mortality is 50% higher after accounting for age and sex differences
Interpreting associations: does A cause B? Causal relationship A B A B A C Confounding B Chance A ? B
Some quick notes on interpreting graphs: • Beware of: ambiguity, distortion and distraction
Data distortion (1) 10.1 10.2 10.3 10.4 0 1 2 3 4 5 6 7
Data distortion (4) General acute inpatient discharges with an alcohol-related diagnosis in any position, by gender, Scotland, 1982/3 - 2009/10
Data distortion (5) General acute inpatient discharges with an alcohol-related diagnosis in any position, by gender, Scotland, 1982/3 - 2009/10
Summary • Always ask the questions: what, who, where, when, how and why • Think about possible problems with data: errors, chance, bias and confounding • Even when things are associated they may not be cause and effect • Beware of the possibility of graphs creating distortions, distractions or ambiguity
Questions gmccartney@nhs.net