1 / 53

Educational Research 102: Selecting the Best Study Design for your Research Question

Educational Research 102: Selecting the Best Study Design for your Research Question. Francis S. Nuthalapaty, MD. 2010 APGO Faculty Development Seminar. Disclosures. No relevant financial disclosures to declare. Learning Objectives. Describe types of research and study designs

fred
Download Presentation

Educational Research 102: Selecting the Best Study Design for your Research Question

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Educational Research 102:Selecting the Best Study Design for your Research Question Francis S. Nuthalapaty, MD 2010 APGO Faculty Development Seminar

  2. Disclosures No relevant financial disclosures to declare

  3. Learning Objectives • Describe types of research and study designs • Understand the characteristics of a good research question • Understand how to convert the research question into a hypothesis • Understand how to select the most appropriate study design to test the hypothesis

  4. What do we already know? TRUE FALSE 1. Issenberg SB, McGaghie WC, Petrusa ER, Gordon DL and Scalese RJ (2005) Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Medical Teacher 27, 2, pp 10-28. 2. Fletcher KE, Davis SQ, Underwood W, Mangrulkar RS, McMahon LF Jr, Saint S. Systematic review: effects of resident work hours on patient safety. Ann Intern Med. 2004 Dec 7;141(11):851-7.

  5. Write an educational research question in which you have interest

  6. Types of Research Empirical vs. Non-empirical Basic vs. Applied

  7. Empirical vs. Non-Empirical Research Empirical • Involves collection of data first hand Non-Empirical • No first hand data collection

  8. Example: Empirical or Non-empirical? The Effect of House Staff Working Hours on the Quality of Obstetric and Gynecologic Care Bailit, J et al, Obstet Gynecol 2009 • OBJECTIVE: To measure the effect of house staff working hours reforms on the quality of obstetric and gynecologic care.

  9. Example: Empirical or Non-Empirical? Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Issenberg SB et al, Med Teach 2005 • OBJECTIVE: Review and synthesize existing evidence in educational science that addresses the question, 'What are the features and uses of high-fidelity medical simulations that lead to most effective learning?'.

  10. Types of Research Empirical vs. Non-empirical Basic vs. Applied

  11. Basic vs. Applied Research Basic • Results apply to a great many people and situations • Results are related to general theory or to a general field of knowledge • Results need not have immediate or even clear implications Applied • Results are applicable only to a specific group of people in a particular situation. • Results are not necessarily related to a broader field of knowledge • Results must have immediate and clear implications for practice

  12. Example: Basic or Applied? Assessing Vaginal Surgical Skills Using Video Motion Analysis Diwadkar G et al, Obstet Gynecol 2009 • OBJECTIVE: To demonstrate the feasibility of using video motion analysis to quantitate a key step of vaginal hysterectomy and define measurable differences between novice and experienced surgical trainees during vaginal hysterectomy.

  13. Example: Basic or Applied? Effects of a Depression Education Program on Residents’ Knowledge, Attitudes, and Clinical Skills Learman L et al, Obstet Gynecol 2003 • OBJECTIVE: To determine whether an interactive educational program would improve obstetrics and gynecology Residents' knowledge, attitudes, confidence, and skills in caring for depressed patients.

  14. Research Methods What is the difference between Qualitative Research and Quantitative Research?

  15. Qualitative vs. Quantitative Research Qualitative • Descriptive and exploratory focus • Used to gain insight into attitudes, behaviors, values • Analysis of unstructured information • Narrative reporting Quantitative • Hypothesis driven • Used to identify association and/or causation • Analysis of discrete variables • Statistical reporting

  16. Quantitative Research • Experimental & Quasi-experimental • Non-Experimental • Causal-comparative • Correlational

  17. Quantitative Research • Experimental & Quasi-experimental • Assess effect of an independent variable on dependent variables • Comparison of 2 or more groups • Control over ‘treatment’ & measurement • Randomization • Control group

  18. Quantitative Research Designs Experimental Treatment Measurement Study Population Randomize Control Measurement Post-test only Control Group

  19. Quantitative Research Designs Experimental Measurement Treatment Measurement Study Population Randomize Control Measurement Measurement Pre-test/Post-test Control Group

  20. Quantitative Research Designs Experimental Measurement Treatment Measurement Control Measurement Measurement Study Population Treatment Measurement Randomize Control Measurement Solomon Four Group

  21. Quantitative Research Designs Quasi-Experimental Study Population Treatment Measurement One-shot Case Study

  22. Quantitative Research Designs Quasi-Experimental Study Population Measurement Treatment Measurement One-group Pre-test/Post-test

  23. Quantitative Research Designs Quasi-Experimental Study Group 1 Treatment 1 Measurement Study Group 2 Treatment 2 Measurement Static Group

  24. Quantitative Research Designs Quasi-Experimental Study Group 1 Measurement Treatment 1 Measurement Matching Study Group 2 Measurement Treatment 2 Measurement Pre-test/Post-test Matched Subjects

  25. Quantitative Research • Non-Experimental • Causal-comparative • Correlational

  26. Causal-Comparative • Baseline differences are pre-existing • Non-permutable: Ethnicity, Gender • Permutable: Teaching style • Determine the cause or consequences of differences • Associations can be identified • Causality cannot be determined

  27. Quantitative Research Designs Causal-Comparative Exposures Outcome Control Outcome Exploration of Effects

  28. Quantitative Research Designs Causal-Comparative Exposures Outcome Exposures Control Exploration of Causes

  29. Quantitative Research Designs Causal-Comparative Group 1 Exposure Outcome Group 2 Outcome Exploration of Consequences

  30. Correlational Research • Single group of subjects • Describe degree to which 2 or more quantitative variables are related • Help explain important human behaviors • Predict likely outcomes • Identifies associations • Causality cannot be determined

  31. Correlational Research

  32. Correlational Research • Correlation Coefficient • Calculated by regression • < .35 = only a slight relationship • .40 - .60 = possible theoretical value • .50 = minimum for crude predictions • >.65 = reasonably accurate predictions • >.85 = close relationship • Positive correlation = direct association • Negative correlation = inverse association

  33. Threats to Internal Validity • Subject characteristics • Loss of subjects • Maturation • Repeated measures • Statistical regression • Investigator bias

  34. External Validity • Can the findings from the study be generalized to larger populations? • Subject characteristics • Environment (lab vs. natural setting) • Did the subjects act differently b/c they were enrolled in a study (Hawthorne effect)?

  35. What are the steps in the research process?

  36. The Research Process Define Research Question Hole in the literature No hole in the literature Conduct Literature Review Refine Research Question Obtain IRB Approval Design Study Disseminate Results Collect Analyze Data

  37. FINER Research Questions • F – Feasible • I – Interesting • N – Novel • E – Ethical • R - Relevant

  38. FINER Research Questions • Feasible: • Can be investigated with available: • Time • Money • Space • Other resources

  39. FINER Research Questions • Interesting: • Interesting to you • Interesting to others in the field • Novel: • Will answers to the question advance knowledge in the field?

  40. FINER Research Questions • Ethical: • Will any physical or psychological harm come to anyone as a result of the research? • Protect participants from harm • Ensure confidentiality of data • Avoid knowing deception of participants • Relevant: • How might answers to this question improve educational practice?

  41. What is a research hypothesis and how is it different from a research question?

  42. Research Hypothesis A prediction regarding the possible outcomes of the study Wallen & Fraenkel. Educational Research: A Guide to the Process. 2nd Ed.

  43. Research Hypothesis • A statement which: • Summarizes the elements of the study • The sample • The design • The predictor and outcome variables Yoder E, MERC Workshop, 2009 AAMC Annual Meeting

  44. Research Hypothesis • Based upon a FINER research question • Simple – one difference or relationship • Specific – clearly stated, defined • Stated in advance – before data collection Yoder E, MERC Workshop, 2009 AAMC Annual Meeting

  45. Research Hypothesis • May be stated: • As the alternative hypothesis • Non-directional • The difference is unknown • Directional hypothesis • Existing studies/data suggest direction of relationships • As the null hypothesis Yoder E, MERC Workshop, 2009 AAMC Annual Meeting

  46. Research Hypothesis • Ha: There is a difference between groups or variables • Female medical students have a different IQ than male medical students • μf ≠ μmorμf > μm • H0: There is no difference between groups or variables • Female medical students have a different IQ than male medical students • μf = μm Yoder E, MERC Workshop, 2009 AAMC Annual Meeting

  47. Example: Hypothesis Improving Resident Competency in the Management of Shoulder Dystocia With Simulation Training Deering S, et al, Obstet Gynecol 2004 • METHODS: Residents from 2 training programs were randomized by year-group to a training session on shoulder dystocia (SD) management that used an obstetric birthing simulator or to a control group with no specific training. Both groups were subsequently tested on a standardized SD scenario, and a physician grader rated the resident's performance with a standardized evaluation sheet.

  48. Example: Hypothesis • What is the research question? • What is the null hypothesis? • What is the research study design? • Basic or Applied? • Qualitative or Quantitative? • Experimental or Non-Experimental

  49. Quantitative Research Designs Experimental Treatment Measurement Study Population Randomize Control Measurement Post-test only Control Group

  50. Small Group Activity • Share your research questions • Select one research question to convert to a hypothesis • Select the most appropriate study design

More Related