280 likes | 539 Views
Systematic reviews of animal studies. Malcolm Macleod. Why do systematic reviews of animal studies?. To summarise existing data To help design clinical trials To understand where evidence is lacking To understand the limitations of animal models. Basic requirements.
E N D
Systematic reviews of animal studies Malcolm Macleod
Why do systematic reviews of animal studies? • To summarise existing data • To help design clinical trials • To understand where evidence is lacking • To understand the limitations of animal models
Basic requirements • Understanding the model – or knowing someone who does • Clear a priori hypotheses • A clear search strategy with inclusion and exclusion criteria • Information and data management
The importance of hypotheses • Observational research • Susceptible to identification of statistically significant but biologically meaningless spurious associations • If there isn’t a question for which you wish to know the answer, why are you doing it?
Examples • “Hypothermia improves outcome in animal models of stroke” • “The efficacy of hypothermia in animal models of stroke depends on the degree of cooling” • “Evidence for the efficacy of NXY-059 is confounded by poor study quality” • “The evidence for efficacy of stroke dugs in animals is confounded by publication bias”
Search strategy • What are you looking for … • Controlled studies testing the effect of hypothermia in an animal model of focal cerebral ischaemia brought about by occlusion of a cerebral artery, where outcome was measured as infarct size or neurobehavioural score. • Exclusion criteria • hypothermia was accomplished with use of a pharmacological agent that may also have an intrinsic neuroprotective property; • cooling was used to counteract (spontaneous) hyperthermia after MCAO; • brain cooling lasted <10 min, for example to counteract heating in models of photochemically induced cerebral infarction; • data were presented in a way not suitable for use in a meta-analysis (e.g. no information on group size, mean or variance); or if • mortality was the only outcome measure.
Example search strategy Studies of hypothermia in animal models of acute ischaemic stroke were identified from … • PubMed, EMBASE and BIOSIS up till December 31, 200> with the search strategy [[[<cerebral> OR <brain> OR <neuron> OR <neuronal> OR <nervous>] AND [<ischemia> OR <ischaemia>]] OR <stroke>] AND [<hypothermia> OR <temperature>] (limit: animals) • hand searching of abstracts of scientific meetings of the International Society of Cerebral Blood Flow and Metabolism, the International Stroke Conference [‘Joint (International) Conference on Stroke and Cerebral Circulation’ before 2000] and the European Stroke Conference during the same time period; • reference lists of identified publications; and • requests to authors of identified publications
Source selection • Download search results to reference management system (eg RefMan) • 2 investigators independently select sources against inclusion/ exclusion criteria • May have to retrieve full text • Discrepancies resolved by negotiation or in discussion with third investigator
Data extraction • Publication meta-data
Data extraction • Publication meta-data • Outcome data
Data analysis • Standardised mean difference analysis • Weighted mean difference analysis • Normalised mean difference analysis • Fixed Effects • Random Effects • Meta-regression
SMD s.d. e.s. = difference/s.d. “sd units” difference e.s. = n1-n2 “real” units WMD 0 mm3 n1 n2 250 mm3 NMD e.s. = d1/100% “percent improvement in outcome” d1 0% unlesioned 100% lesioned
Testing significance • Partitioning of heterogeneity: • Observed heterogeneity = within group heterogeneity + between group heterogeneity • Test against chi squared distribution with n-1 degrees of freedom • Observational, so set high statistical bar
So, what can it do …? • Describe a literature ….
Estimate efficacy … • Hypothermia • 101 papers • 277 experiments • 3353 animals
Show potential sources of bias … NXY 059 9 publications 29 experiments 408 animals Improved outcome by 44% (35-53%)
Illustrate publication bias … 991 publications
Provide evidence to change practice … • Animals • Sample size calculation • Inclusion and exclusion criteria • Randomization • Allocation concealment • Reporting of animals excluded from analysis • Blinded assessment of outcome • Reporting potential conflicts of interest and study funding
Further resources • http://www.camarades.info • http://www.camarades.info/index_files/papers.htm • malcolm.macleod@ed.ac.uk • The CAMARADES podcast … • http://www.camarades.info/index_files/podcasts/podcast.xml