The Basics of Reading A Research Article

1. The Basics of Reading A Research Article Amy Gutman MD ~ EMS Medical Director prehospitalmd@gmail.com / www.TEAEMS.com

2. EMS Research History Definitions Breaking down research articles Evidence-based medicine Research resources Summary OBJECTIVESInformation Presented Meets or Exceeds NREMT Educational Standards

3. OVERVIEW • Much “sensationalized” but bad research • Cellphones cause brain cancer • Vaccines cause autism • How do you tell “good” from “bad”? • Is “good” research “important”? • Review basics of reading a research paper so you can keep up with current trends in prehospital care

4. WHAT IS RESEARCH? • Science is systematized knowledge derived from observation& experimentation • Research is a systematic investigation to discover or establish facts by utilizing the “scientific method” • The scientific method is how researchers construct an accurate, reliable & repeatable answer to a question

5. SCIENTIFIC METHOD • Anyone can perform &/or read research • Observe something • Ask a question, or “hypothesis”, about observation • Use hypothesis to make prediction • Test prediction by experimentation • Modify hypothesis after reviewing experimentation results • Repeat until satisfied hypothesis answers question • Ask more questions!

6. HYPOTHESIS & PREDICTIONS • A hypothesis is a question or theory based upon an observation • Often common sense, but common sense is neither common, nor true • Examples: • “Why do I get long-distance transports at the end of my shift?” • “Does anyone really drink ‘two beers’”?

7. SCIENTIFIC METHOD: EXAMPLE • Observation, Hypothesis & Prediction • “My Medical Director is a an awesome, short, chunky, red-headed female. Therefore I believe that ALL awesome Medical Directors are short, chunky, red-headed females” • Testing the Hypothesis / Methods: • “According to providers I asked, there are at least two awesome Cape Cod Medical Directors who are not red-headed, short, chunky or female” • Results: • “Because there are awesome Medical Directors who didn’t meet the hypothesis, I modified the hypothesis to “One awesome Medical Director is short, chunky & red-headed, but not all awesome Medical Directors must be physically similar in appearance” • Reproducibility & Clinical Applicability: • Easily be reproduced in any prehospital environment • Medical Director physical characteristics may not indicate awesomeness

8. EMS RESEARCH • The goal of EMS research is integrating results into clinical practice after an evidence-based initiative to improve patient outcomes in a competitive & cost-conscious healthcare market • Domains: • Clinical: Study of direct patient care • Systems:Effects of EMS system design & operations on resource utilization • Educational: Training methodology for prehospital providers

9. EMS RESEARCH HISTORY • EMS practices initially modeled after battlefield/hospital care • 1st paper published in 1966 (BMJ) showed patients “suffocated” due to poor EMS training • 1973 EMS Systems Act (Public Law 93-154) funded development of regional EMS systems, protocols & research • Pre-1980 only 3 “Efficacy Studies” published that evaluated if a practice or drug works in ideal conditions(but not necessarily in the field!) • A 1989editorial stated “Impressively Deficient” data for “efficacy of scope of EMS practice” • 1993 NHTSA FEMSC “EMS for Children Report” identified need for pediatric research • 2001 NHTSA National EMS Databasefounded to answer research questions • 1st paper evaluated prehospital ETI in pediatric cardiac arrest & found that most providers lacked training & that patients worse outcomes • Today: better funding, peer-reviewed journals & some clinically relevant research

10. EMS RESEARCH BARRIERS • Few trained researchers with minimal funding • Failure to translate research into clinical practice • Lack of integrated IT systems linking prehospital data with patient outcomes • Multiple locations / personnel / researchers • HIPPA, IRBs • Unstable patients often w/o diagnosis at time research conducted • Uncontrolled research environment • Critical nature limits practical & ethical experimental interventions • Difficulty in obtaining informed consent

11. CURRENT NATIONAL EMS RESEARCH (www.EMS.GOV/Research) • National EMS Assessment (2009-2011) analyzed 20,000 agencies, 82,000 vehicles, 826,000 personnel • EMS Evidence-Based Guidelines • National EMS Information System (NEMSIS) • Longitudinal EMS Demographic Study • National EMS Research Agenda • National EMS Assessment • EMS Workforce for the 21st Century: Feasibility for Safety & Surveillance • EMS Systems Configuration Study • EMS Performance Measures & Outcomes Evaluation • Motor Vehicle Occupant Safety Survey: Crash Injury & EMS

12. EMS RESEARCH ORGANIZATIONS / WEBSITES

13. PREHOSPITAL RESEARCH JOURNALS • JEMS is most widely-read print journal • Not peer-reviewed • Opinions, summaries & commentaries that are informative but not necessarily evidence-based • Peer-Reviewed Journals • Prehospital Emergency Care • Prehospital &Disaster Medicine • Annals of Emergency Medicine • Pediatric Emergency Care • Air Medical Journal

14. EVIDENCE-BASED MEDICINE (EBM) • Scientific method to answer a question & make patient care decisions • Establish “standards of care” after multiple studies replicate similar results, or a multi-center study shows undeniable results • Example: Maine Selective Spinal Clearance Study • In cost-cutting age, insurance often will not reimburse for non-EBM practices • Litigation possible if non-EBM procedures

15. PEER-REVIEWED RESEARCH • Rigorous review by persons of similar credentials to researchers • Maintains standards of quality & provides credibilityby critically evaluating research & exposing flaws prior to publication • A review will find the article: • Acceptable (rare) • Acceptable but requires revisions (common) • Not acceptable (common)

16. HOW AN ARTICLE GETS TO PRINT (TIMELINE) • Peer-reviewed EBM articles appear in many journals (some prestigious, some not) • Most research happens years before publication • Days 1-7 Hypothesis Formulated • Days 1-60 Methodology formulated; Funding application begins • Days 30-120 IRB proposal written, submitted, resubmitted • Months 4-5Identify & recruit investigators & subjects • Month 6-9 Investigator training; Research, data collection & article prep begins (background, methodology) • Months 9-? For RCTs 12-24 months, Chart Reviews usually 3-6 months • Year 2+ Abstract prepared & submitted for conference presentation; article prepared for submission • Year 2+ 8 Mo If accepted for revisions, editing begins. If refused, hunt for another journal begins • Year 2+ 10 Mo Article accepted; published 3-6 mos later unless special circumstances

17. JUDGING A RESEARCH PAPER • Credible source? • Applicable to EMS?  • Design appropriate to answer question being asked?  • Variables controlled to ensure reliability of results?  • Results conclusive/ logical? • Alternative explanations for results?  • Do findings impact current evidence or require changes in clinical practice? • How biased is the research?

18. BASIC VS APPLIED RESEARCH BASIC APPLIED / Clinical • Performed in lab or clinical setting as preliminary data collection to refine a hypothesis • Independent variables manipulated to observe & describe effect(s) on a dependent variable • Often examines properties of drugs or used in development of diagnostic tests • Every drug starts as a chemical whose properties are examined through experimentation • Real subjects in real-world situations • Effects of a drug, device, or procedure studied on humans • Test & prove (or disprove) therapies, procedures & protocols

19. LEVELS OF SCIENTIFIC EVIDENCE • Ia: Randomized Controlled Trial (RCT) meta-analysis • Ib: RCT (single) • IIa: Prospective Non-RCT • IIb: Experimental study (case series, animal studies) • III: Retrospective or Descriptive studies (case-control, comparative, correlation studies) • IV: Expert opinions, extrapolations • V: Rational conjecture Strongest Weakest

20. RANDOMIZED CONTROLLED TRIAL (RCT) • Subjects randomized into control & experimental groups • Groups evaluating for variables & outcomes of interest • Example: Outcomes of STEMI patients given aspirin vs those not given aspirin • Uses the scientific method & most “valid” research method

21. RCT EXAMPLE • Mattox KL, Bickell B, Pepe PE*, Burch J, Feliciano D. Prospective MAST study in 911 patients. J. Trauma. 1989;29:1104-12 • 911 adult trauma pts with SBP <90mmHg randomized by alternate days & transported to Ben Taub • Experimental Variable: MAST or no-MAST • Dependent Variable: survival from trauma • Independent Variables: etiology, age, race, sex, location of injury, trauma scores, injury severity scores • Results: Mortality 31% MAST group, 25% in non-MAST group • Conclusions: MAST trousers in trauma pts increases mortality

22. RCT: COHORT STUDIES • Subjects with a condition or receiving a treatment compared with another group not affected by condition / treatment under investigation • Different than a double blind study as this is an observational study only • No intervention by the researchers

23. EXAMPLE: COHORT STUDY • GottesmanBE, Gutman A et al. Radiation exposure in emergency physicians working in an urban ED: a prospective cohort study*. • HYPOTHESIS: EM MDs are exposed to radiation levels above NCRP limits • METHODS: Prospective cohort study conducted at a level I Trauma ED. Dosimeter radiation badges placed on all ED MDs & their phones carried 24/7/365. At the end of the study exposure dose for each subject calculated • RESULTS: 75 MDs enrolled. Compliance 99%. Annual extrapolated exposure for subject with highest radiation level was 50 mrem, below 5000 mrem exposure limit for health care workers • CONCLUSION: EM MDs working in an urban, academic, level I trauma center ED are not at risk of exceeding NCRP dose limits for ionizing radiation exposure *Am J Emerg Med. 2010 Nov;28(9):1037-40

24. RCT: DOUBLE BLIND STUDY • Neither subject(s) nor researchers knows whether subject receiving treatment, or control (placebo) treatment • Example: • AufderheideTP, et al. Resuscitation Outcomes Consortium (ROC) PRIMED trial: rationale & methodology for the impedance threshold device (ITD)* • AIM: Compare OOHCA survival to discharge between CPR+ITD vs CPR vsfake ITD • METHODS: Prospective, double-blind, RCT in OOHCA pts by EMS systems participating in the ROC. 1.4% absolute survival difference found in CPR + ITD • CONCLUSION: If ITD demonstrates hypothesized survival improvement, an estimated 2700 deaths from OOHCA could be prevented annually *Resuscitation. 2008 Aug;78(2):179-85

25. SYSTEMATIC REVIEWS (RCT META-ANALYSIS) • Summary of literature & statistical analysis after detailed search to determine effectiveness of interventions / procedures • Results of multiple studies evaluating a similar question are pooled & statistically analyzed • Highest level of scientific evidence as reduces possibility of false negative results • Poor quality studies are often excluded from the statistical analysis • Because patient numbers are so high (hundreds to thousands), minimal chance of inaccurate conclusion by “chance”

26. ANIMAL RESEARCH • Precursor to evaluating a human intervention • Computer models often initially used vs animal models for ethical reasons • Results often cannot be extrapolated to draw conclusions on what will happen in humans • Isoproterenol dosing based on rat studies resulted in 3500 British asthmatics dying from overdoses Pharma, 1971; vol18:272 • Domperidone withdrawn from market after 25 pts died from lethal arrhythmias not seen in dogs with 70 X toxic human dose Drugs, 1982, vol24:360-4 • Digitalis testing delayed in animals, while used safely for decades in humans because of toxic rodent effects Toxin vitro, 1992, vol6:47-52

27. EPIDEMIOLOGICAL / CORRELATION STUDY • Evaluate relationship between variables to determine if there is any correlation • “Positive”, “Negative”, “Null” • Often miscategorize “association” with “cause” • Positive correlation does not mean “A” causes “B” • Example: Obese persons drink soda, therefore soda causes obesity • Rely on memory recall or data documentation, which is often incomplete or unreliable • Used in surveys or retrospective chart reviews which may lead for a more controlled study

28. EPIDEMIOLOGICAL / CROSS-SECTIONAL SURVEYS • Collects data at a single point in time to evaluate a policy or public health issue • Example: A case of hepatitis reported at “Restaurant Q”, so all persons who ate at that restaurant are questioned to discover if they have symptoms consistent with hepatitis • Limited by event memory, difficulty in contacting subjects

29. EPIDEMIOLOGICAL / LONGITUDINAL OR CASE CONTROL STUDIES • Follows subjects over a period of time, serially asking research questions for study duration • Example: Framingham Heart Study has followed residents of Framingham, MA since 1948 looking for heart disease patterns • In case control studies, subjects with a condition are compared with those who do not • Observational, not interventional therefore classified as “epidemiological”

30. CASE REPORTS • Case Study / Case Report • Detailed info about a single subject or a small group of subjects • Examples: 1st time for new surgical technique, “weird” presentations of diseases • Case Series • Report on a series of subjects with one specific unifying factor but no control group • Examples: Outcomes of pelvic trauma patients with MAST trouser application

31. LITERATURE REVIEW • Exhaustive search of all relevant evidence-based literature related to a topic using multiple databases • Cochrane reviews (meta-analysis) • PubMed (NIH study repository) • CINAHL • Asking a question of Google or Wikipedia is NOT an exhaustive (nor reliable) literature review Accredited Wikipedia Resources

32. RETROSPECTIVE CHART REVIEWS / HISTORICAL STUDIES • Systematized searches for fact(s), then using information to describe, analyze & interpret the past • Example: • Bledsoe BE, Smith MG. Medical Helicopter Accidents in the United States: A 10-Year Review. • Methods: Review all US air medical helicopter accidents from 1997-2002 from NTSB database • Results: Majority of air medical helicopter accidents resulted from pilot error or mechanical failure • *J Trauma. 2004;56:1325-1329

33. EXPERT OPINIONS • Examples: • Systematic reviews • Narrative reviews • Pure opinion pieces • Position papers • Based on a literature review (including RCT meta-analysis) but considered low on the “Scientific Evidence” schematic due to not being an actual “study”

34. RATIONALE CONJECTURE • Lowest level of scientific validity but often accurate & important • Anecdotal reports often fit into this category, as do “interesting observations” • Often amusing due to “duh” factor

35. “DUH” EXAMPLE: Back MD. Knowing Your Mate Value: Sex-Specific Personality Effects on Accuracy of Expected Mate Choice*. • Males rated photos of women displaying “exploitability cues” indicating appropriateness for one-night stands vs “stability cues” appropriate for marriage or long-term dating • 22 exploitability cues correlated with “sexual receptivity”, including: • Lip licking/biting, appearing intoxicated, tight clothing, obesity, dumbconversations, rubbing breasts / genitals • Conclusions: Males prefer drunk, slutty**, dumb women for 1 night stands *PsycholSci. 2011 Aug;22(8):984-9. **Authors’ words, not mine!

36. PSEUDOSCIENCE: THINGS WE BELIEVE ARE TRUE THAT AREN’T • Epinephrine saves lives in OOHCA • No RCT has shown better outcomes with epinephrine vs placebo in OOHCA • Amiodarone improves outcomes in pediatric OOHCA • Only 1 RCT suggestsrisk > benefit of amiodarone (not statistically significant) • ATLS & the “Golden Hour” • Evidence suggests ATLS has less impact on outcomes than believed • Zero studies have shown anything magical about the 1st 60 minutes managing trauma patients

37. ARTICLE COMPONENTS • Author • Title • Key Words • Abstract • Introduction / Background • Methodology • Results • Discussion • Conclusion • References / Citations • Acknowledgements

38. AUTHOR INFORMATION • Primary, secondary & associate authors • Primary & secondary do majority of work • Associates edit, collect data, produce diagrams • If a multi-center, 1 author from each site listed • If >10 sites, authors listed in acknowledgements • Lists primary site / contact information • Dirty secret…many “authors” have never seen the research – they are department chairs, “famous” or “friends of famous”, all of which lend credence to research (& help get it published)

39. TITLE • “Headline”* clearly & briefly describing what paper examines • May be misleading to attract attention • Example: • “Cellphones linked to brain cancer” is sexier than … • “Environmental risk factors for cancers of the brain and nervous system: the use of ecological data to generate hypotheses.” *Keith Wesley

40. KEY WORDS • Important “search terms” • Different query results from: • “Prehospital” • “Paramedics” • “EMTs” • “EMS”

41. ABSTRACT • Short summary of study • Distills key study elements into a few easily understood sentences • After reviewing, the reader determines if article is “worth my time to read”

42. INTRODUCTION / BACKGROUND • Familiarize readers with subject & research field • Examines key question & reviews current relevant literature • Provides rationale for why research is important • Presents hypothesis attempting to answer the key question

43. METHODOLOGY • Evaluate research quality by showing how study conducted • Study design & population description • Data collection & interpretation methods • Provides details so other researchers can replicate & validate findings • Reader “assumes” articles published in a peer-reviewed journal are statistically valid • “Valid” data still may be misinterpreted by the authors or media

44. RESULTS • Objective description of data indicating statistical significance / insignificance using raw & calculated data • A “p value” <0.05 indicates result not likely to have happened by chance, and is “probably true” • Data interpretation reserved for discussion section • Results must be interpreted by reader keeping in mind validity of the hypothesis, study methodology & author bias

45. DISCUSSION • Study results interpreted & evaluated against existing body of literature • Attempts to connect results to bigger picture & show how results applied clinically • Statistically significant findings may not have much clinical application • Author’s opinion about results is valuable but taken with a grain of salt • Any bias or unexpected results reviewed & explained • No one starts a study expecting to “fail” • Example: If a author states “trend towards significance”, it means that the study failed to show a positive outcome, so the author “stretches the truth” to imply that data are really significant, but the study itself was flawed • Even well-designed / conducted studies limited by imperfect nature of EMS research

46. CONCLUSIONS • Summarizes info with recommendations based upon data analysis • Recommendations range from changing clinical practice to more research • Reader must weigh all parts of the study & determine if methodologically sound , clinically reproducible & valid enough to change or continue current practice • Race, gender, geography, economic status & “sensitive” factors should not be ignored due to political correctness • Example: Research links asthma incidence & severity with race/ socio-economic status; therefore providers should expect patients with more severe respiratory distress in urban, black & Hispanic areas vs rural Caucasian areas

47. REFERENCES / CITATIONS • Each cited author & paper plus quotes, ideas & content in a standard format • Source of further info on a topic • Decisions rarely made after one study • References provide further info on that subject in which you are interested

48. WHAT DOES THAT CITATION MEAN? Gottesman BE, Gutman A, et al. Radiation exposure in emergency physicians working in an urban ED: a prospective cohort study. Am J Emerg Med. 2010 Nov;28(9):1037-40. Epub2010 Mar 9. PMID: 20825934 Authors / Researchers “And Others”; usually part of the research team Title of study, describes the hypothesis & results PubMed ID# Electronic publication date: year, month, date Journal Name; does not indicate if peer-reviewed Print publication date: year, month; journal year(issue):pages

49. ACKNOWLEDGEMENTS & DISCLOSURES • Research expensive & often sponsored by a university or company • Authors must disclose financial or other support (i.e. airfare to a conference) to explain potential conflicts of interest • “Disclosures” indicate bias that may affect study’s outcome • If design adheres to scientific method, little chance for bias to affect outcomes • Example: If a study determines a drug to be ineffective / harmful, & if that study were sponsored by a competitor of the drug then readers should know that there is not only bias present, but potentially a data misinterpretation

50. NEMSRA RECOMMENDATIONS TO IMPROVE FUTURE EMS RESEARCH • Structured training programs • i.e. Fellowships • Collaborative Centers of Excellence combining investigators, resources & public policy • i.e. The Resuscitation Outcomes Consortium (The “ROC”) • i.e. Recognizing that prehospital care is an actual “academic” branch of medicine • Better evidence before implementing procedures, devices, or drugs • i.e. More RCTs & evidence-based practice • Standardized data collection using uniform prehospital data elements • i.e. Utsteincriteria • The FDA & Office for Human Research Protections should re-evaluate exception from informed consent • i.e. HIPPA and human subject testing exemptions • Investment in infrastructure to overcome obstacles impeding EMS research • i.e. IT, funding, data-sharing