Research Design and Sources of Evidence

1. Research Design and Sources of Evidence Chapter 3 Jane L. Forrest, EdD, RDHSyrene A. Miller, BA Pam R. Overman, BSDH, EdD Michael G. Newman, DDS

2. Purpose • The purpose of this chapter is to discuss sources of scientific evidence and characteristics of research designs that constitute the evidence. • Although EBDM emphasizes using randomized clinical trials and other quantifiable methods, this focus has evolved to include qualitative research and acknowledging that different research designs contribute to a continuum of knowledge.

3. Objectives • Identify what constitutes evidence. • Explain the difference between research and evidence. • Identify sources of primary and secondary evidence. • Discuss the difference between experimental and nonexperimental research. • Identify distinguishing characteristics of different research methods: Randomized Control, Cohort, Case Control, Case Series, and Case Report Studies. • Discuss the difference between quantitative and qualitative research and the role of qualitative research in EBDM. • Identify scientific sources of evidence to use in clinical decision making.

4. Sources of Evidence • Scientific evidence is the product of well-designed and well-controlled research investigations that minimize sources of bias. • Evidence is the synthesis of all valid research studies that answer a specific question. • Evolves over time as more research is conducted, underscoring the importance of keeping current with the scientific literature

5. Research and the Strength of Evidence • Evidence and Research often are usedinterchangeably. • Evidence is distinguished from research in that a single research study does not constitute "the evidence," but rather contributes to an overall body of knowledge that has been derived from multiple studies investigating the same area. • Evidence is considered the synthesis of all valid research studies that answer a specific question; however, in some cases, there may be only one study.

6. Building a Body of Scientific Evidence Figure 3-3. Primary vs. Secondary Research Primary Research Secondary Research Reviews of Already Conducted Research Individual Research Studies Study 1 Systematic Review & Meta-Analysis Statistical Analysis of Synthesized Results Study 2 Synthesized Results Study 3 Study 4

7. Primary: original research publications Quantitative: hypothesis testing; establishing cause and effect; data reported mathematically Qualitative research: exploratory; why and how of decision making; data reported in narrative terms Secondary: filtered or synthesized publication of the primary/original research Systematic reviews Meta-analyses Evidence-based article reviews Evidence-based clinical practice guidelines Evidence-Based Sources: Primary & Secondary

8. Quantitative Primary Research: Experimental Studies • Researcher controls or manipulates the variables under investigation, such as in testing the effectiveness of a treatment. • These studies are the most complex and include randomized controlled trials and controlled clinical trials. • The Randomized Controlled Trial (RCT) provides the strongest evidence for demonstrating cause and effect, i.e., the treatment has caused the effect, rather than it happening by chance.

9. Variables in Experimental Studies: RCT • All experiments have at least 1 independent and 1 dependent variable. • Concurrent enrollment of subjects and follow-up of subjects in experimental and control groups • Assignment of subjects through random process • Purpose of experiments is to estimate the extent to which the independent variable causes changes in the dependent variables. • Physically manipulate independent variable • Giving experimental group new drug and control group a placebo • Teaching 1 class with computers and 1 without

10. Key Characteristics of RCTs • Ability to randomly assign subjects to either the experimental or control group and to randomly allocate treatments • Prospective in nature and can include blind or double-blind strategies • A double-blind RCT is one in which neither the patient nor the investigator knows whether the patient is receiving the experimental treatment or the control treatment, e.g., studies involving therapies (pills/liquids/pastes). • More difficult to double blind studies when testing a new treatment, technique, or procedure where the investigator or patient can distinguish a difference. 

11. Experimental Research • Random assignment – equal chance of being in the experimental group or control group • Double-blind – investigators or examiners are not aware of which group the subjects are assigned • Inter-rater reliability – degree to which examiners are calibrated; achieve same results (e.g., 8 of 10 times) • Intra-rater reliability – degree to which an examiner achieves the same results with himself

12. Quantitative Primary Research: Nonexperimental Research • Studies in which the researcher does not give a treatment, give an intervention, or provide an exposure, i.e., data are gathered without intervening to control variables • Includes cohort studies, case control studies, case series, and case reports • Descriptive: used to systematically describe and interpret conditions or relationships that already exist

13. What Is a Cohort Study? • Makes observations about the association between a particular exposure or risk factor (tobacco) and the development of a disease (e.g., lung cancer) • Patients/subjects have been exposed to tobacco but do not have lung cancer • Followed over time and compared with a similar group that has not been exposed to the risk factor • Again, a treatment/intervention is not given by the investigator.

14. Prospective Cohort Study Design Exposed Unexposed Disease No Disease Disease No Disease Persons with and without the exposure of interest (e.g., tobacco) are identified at the initiation of the study. Information is then collected looking forward in time to identify disease outcomes. At the start of the study, neither group has the disease.

15. What Is a Case Control Study? • Patients who already have a certain condition are compared with people who do not • For example, lung cancer patients are asked how much they smoked in the past • Answers are compared with those from a matching sample without the condition • Less reliable than either RCTs or Cohort Studies. Statistical relationship between two conditions does not mean that one condition actually caused the other. • E.g., lung cancer rates are higher for people who earn less than $50,000 (who tend to smoke more), but this does not mean that they can reduce their cancer risk by getting a salary increase to over $50,000. Guide to Research Methods http://library.downstate.edu/ebm/2100.htm

16. Case Control Study Design Disease No Disease Exposed Unexposed Exposed Unexposed Persons with and without the disease of interest (caries, cancer) are identified at the initiation of the study. Information is then collected looking backward in time to identify potential exposures (sugar, tobacco) that could have contributed to getting the disease.

17. Qualitative Primary Research • Nonexperimental in that it conducts studies in natural settings in an attempt to understand an event from the point of view of the participants • It seeks to provide depth of understanding. • It answers questions such as what, how, and why. • It explores issues in more depth with those experiencing the issue, rather than testing a hypothesis to answer questions such as how many or what proportion. • Can generate new theory

18. Qualitative Primary Research (cont.) • Compliments quantitative research by attempting to clarify the meaning of how many or by providing a greater understanding of why an intervention works • Quantitative research may ask, "How many smokers have tried to quit?" whereas qualitative research explores "What stops smokers from quitting?" • The most important consideration in designing a study is to use the right methodology to answer the question.

19. Table 3-1: Characteristics of Quantitative Research Approaches6,7

20. Secondary Research: Systematic Reviews and Meta-Analysis • Secondary research is filtered or synthesized publications of the primary research literature. • Systematic reviews (SRs) and meta-analyses, evidence-based article reviews of already conducted research, and evidence-based clinical practice guidelines • Provides a way of managing large quantities of information by providing a summary of primary research studies that have investigated the same question • Uses explicit predefined criteria for retrieval of studies, assessment, and synthesis of evidence from individual RCTs and other well-controlled methods

21. Systematic vs. Literature Reviews

22. Secondary Research: Evidence-Based Journals and Article Reviews • New evidence-based resources are being developed for practitioners to facilitate integrating evidence into their clinical decision making, e.g., Journal of Evidence-Based Dental Practice (JEBDP), Evidence-Based Dentistry (EBD). • Provide concise and easy-to-read summaries of original and review articles selected from the biomedical literature based on specific inclusion criteria • Article reviews of already conducted research often consist of a 1- to 2-page structured abstract along with an expert commentary highlighting the most relevant and practical information of the study being reviewed.

23. Secondary Research: Practice Guidelines • Growing source of synthesized information on a specific topic • Guidelines are “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.” • The inclusion of scientific evidence within clinical practice guidelines has now become the standard, in that guidelines should incorporate the best available scientific evidence. SRs support this process by putting together all that is known about a topic in an objective manner.

24. Secondary Research: Practice Guidelines (cont.) Examples of clinical practice guidelines • ADA’s clinical recommendations on professionally applied topical fluoride • AHA’s guidelines on antibiotic premedication • ADHA guidelines on polishing procedures

25. Activities • Quiz • Exercise 3-1: Type of Study Exercises

26. Critical Thinking Questions • Discuss how quantitative and qualitative research are complimentary and provide an example of a study related to patient problems that would include both types of studies. [Example: how often patients floss (quantitative study) and what barriers do they encounter that prevent them from flossing every day (qualitative study)]. • Explain why a RCT is not always the appropriate research design to use. • Provide an example of when you would first conduct a traditional literature search prior to looking for a systematic review or meta-analysis.