September 16, 2013

1. Foundations of Clinical Research: selecting and sampling populations to minimize biasAl Mushlin, MD, ScMDepartment of Public HealthWeill Medical College of Cornell University September 16, 2013

2. How Research Works ACTUAL STUDY RESEARCH QUESTION STUDY PLAN design implement Target population Intended sample Actual subjects Random & systematic error Random & systematic error Phenomena of interest Intended variables Actual measurements infer infer TRUTH IN THE UNIVERSE TRUTH IN THE STUDY FINDINGS IN THE STUDY

3. Two Kinds of Errors: errors solutions Study design and quality control Patient selection and sampling Sample size, correct measurements and statistical calculations • Systematic (bias) • Random (statistical)

4. Study Validity:Nothing is Perfect! • How well study measured what it intended to measure (internal validity) • How well study measured truth in the universe (internal plus external validity, "generalizability”) • Internal validity is necessary but not sufficient for external validity

5. Clinical Research Designs

6. Key Steps In Research • Identify the “problem in the universe” • Define the research question • Choose the study design • Specify the study population • Develop astudy protocol • Construct an operations manual • Do the study • Analyze data • Communicate results

7. The intended sample should be representative of the population of interest REASEARCH QUESTION STUDY PLAN design Intended sample Target population ERRORS Phenomena of interest Intended variables infer TRUTH IN THE STUDY TRUTH IN THE UNIVERSE External Validity

8. The first step is being clear about the population of interest • The population that is closest to the question in the universe • The group of patients for whom the question exists and the answer is important • we want the study findings to reflect (as close as possible) what is going on in the population of interest • The target population

9. The target population should “contain” the question in the universe REASEARCH QUESTION Target population Adults having THR Phenomena of interest Revision frequency/probability TRUTH IN THE UNIVERSE

10. The next step: the intended sample should represent the target population • Trade-off between generalizability and efficiency/feasibility • Common cause of systematic errors (selection bias) • critical step driven by research question and logistics

11. The intended sample should be representative of the population of interest REASEARCH QUESTION STUDY PLAN design Intended sample Target population Total sample of adults over 18 receiving THR at an orthopedic hospital Adults having THR ERRORS Phenomena of interest Intended variables Revision frequency/probability Repeat operation (or not) by 5 years infer TRUTH IN THE STUDY TRUTH IN THE UNIVERSE External Validity

12. What you want the actual study sample to be: • As close as possible to the intended sample (and thereby to the target population) • Representative enough to reduce systematic error (selection bias) • Complete enough (relative to the intended sample) to reduce further selection bias • Large enough to minimize random error • Acceptable in terms of cost and time to accomplish study

13. The actual subjects and measurements should be as close as possible to those intended STUDY PLAN ACTUAL STUDY implement Actual sample Intended sample Total sample of adults over 18 receiving THR at an orthopedic hospital 20K+ pts identified through the registry ERRORS Intended variables Actual measurements Revision frequency/probability infer Admission or self –report of THR revision FINDINGS IN STUDY TRUTH IN STUDY Internal Validity

14. What the study sample is: • Subset of the intended population available to investigator for analysis • The patients available for analysis, e.g., patients identified, consented, and actually responded to inquiry/questionnaires

15. Three steps: identifying the problem/ population “in the universe”, designing the best study you can, and then implementing it well. TARGET POPULATION design implement Designing and implementing INTENDED SAMPLE ACTUAL SAMPLE TRUTH IN THE UNIVERSE Drawing conclusions TRUTH IN THE STUDY FINDINGS IN THE STUDY infer infer

16. Defining the Samples:requires understanding who to include and who among those should not be studied • Inclusion Criteria • Exclusion Criteria

17. Inclusion Criteria • Specifying characteristics relevant to the research question • Demographic criteria • Clinical factors • Temporal specifications • Geographic areas/types of settings • Others

18. Exclusion Criteria • Subsets of the population that will not be studied due to: • High likelihood of non-participation • Inability to provide good data • At high risk of adverse effects • Characteristics that make study unethical

19. Why and When is there the Need to Sample? • If and when the intended and available population is larger than can be practically studied • The objective to select a smaller (i.e., more manageable) group of patients to study while still having the study represent the total patient population

20. Two General Ways to Sample • Convenience samples • Patients readily available to the investigator - (i.e. encountered attendees at a clinic at the investigator’s institution) • Systematic/consecutive samples • or • Probability samples • Selecting individuals so that each member of the population has a specified chance of being studied • Increases generalizability

21. Probability Samples • Random Sample • Enumerate patient population • Use random number generator to select study sample • Stratified random sample • Divide population into subgroups of interest (gender, ethnicity, age) • Random sample within subgroups • Cluster sample

22. Outline of the Study Protocol