Chapter 7

1. Chapter 7 Obtaining Subjects

2. Learning Objectives • Determine the valid connection between the study sample, study population, and theoretical population • Identify and assemble appropriate sampling frames • Select samples using appropriate sampling methods • Recruit and retain study participants • Maximize internal and external validity • Calculate the required sample sizes

3. Sampling • It is extremely rare that a study can include every last individual in a population of interest • Costly and time consuming • No fool-proof method for identifying everyone in the population • For this reason, most studies select a sample of individuals from the population of interest • Ideally, the sample will be a good “representation” of the population

4. Sampling Concepts • Target or theoretical population – entire group to which the study results pertain • Study population – members of the target population that are accessible by the researcher • Sampling frame – concrete list of members of the study population • Sample – Subgroup of individuals selected from the sampling frame

5. Sampling Process Target Population Study Population Sampling Frame Sample

6. Connection between Components • Representative – degree to which the sample includes individuals with similar or identical characteristics to those of the target population • Generalizable - generalizability is the degree to which study results are valid for members of the study population not included in the sample • The degree of both of these depends on the accuracy and rigor in which the sample was selected

7. Connection between Components Representative Generalizable

8. Sampling Methods • Probability-based methods – ability to assign a numeric value to the probability of any one person being chosen for the study • Ideally use a random method in which all individuals have an equal probability of being selected • Simple random • Systematic random • Stratified random

9. Simple Random Sample • Ideal but rarely achievable • Every individual in the sampling frame has the same probability of being selected for the sample • If we have a population size of N and want a sample size of n, then each individual should have a (n/N) probability of being selected • Say N=1000 and we want n=100 • p = 100/1000 = 0.10 or 10% chance of being selected • If each person has the same chance of being selected, there should be no systematic difference between those selected and those not selected – the sample would NOT be biased

10. Simple Random Selection • Assign everyone in the sampling frame a number from 1 to N • Generate numbers randomly • Random numbers tables (found in old stats. books) • Computerized number generators • Mechanical method like lottery ball selector • Select the individual with the corresponding number generated

11. Systematic Random Sample • Appropriate when it is not possible or practical to assign numbers to each individual in the frame • Appropriate when random numbers cannot be generated • Choose every kth individual where k = N/n • If N=1000 and n=100 then k = 1000/100

12. Systematic Random Selection • Organize individuals on a list or physically, such as boxes of biological specimens • Calculate k • Randomly select a number between 1 and k • Start with the unit with the randomly selected number on the list, then keep counting down the list and choose every kthindividual or specimen

13. Stratified Random Sample • Appropriate strategy to ensure that individuals with specific characteristics will be selected • Sampling frame units are first stratified or grouped according to the characteristic(s) of interest for inclusion (e.g., racial/ethnic groups), then individuals are selected randomly from each group or strata • Proportionate – individuals are selected to be the same proportion of the strata relative to the whole study population • Disproportionate – individuals are oversampled (at a greater proportion) or undersampled (at a lesser proportion) relative to the study population depending on the needs of the study

14. Stratified Random Selection • Make separate lists of individuals from the sampling frame for each strata of interest • For example, a list of females and a separate list for males • Assign numbers to each individual or record their original numbers separately for each strata • Generate random numbers and select that assigned number from each strata

15. Nonprobability Samples • Samples for which individuals have an unknown probability for being selected • Appropriate when no sampling frame is available, for example: • Homeless individuals who don’t visit shelters • Victims of domestic violence who don’t contact the police or seek medical care • Not possible to determine the representativeness of the sample, but this type of selection is the only one available for some types of target populations

16. Convenience Sampling • Include anyone who is eligible, no method of selection is used • Individuals who respond to advertisements or contact letters • Students in school class • Attendees at an event such as a health fair

17. Purposive Sampling • Sampling with a “purpose” – to include only individuals with relevant characteristics (e.g., injection drug users not in treatment) • Still, no sampling frame is available • Variations

18. Variations of Purposive Samples • Expert sample – include only individuals with expertise relevant to the study topic • Modal instance sample – include individuals who possess the “average” characteristics of the target population • Heterogeneity sample – include those who represent the broad array of characteristics of the target population • Quota sample – recruit a prescribed number of individuals with predetermined characteristics, then stop recruiting people with those characteristics when the desired number is achieved • Snowball sample – locate the first few individuals with the desired characteristics, ask them to give contact information of others like them, contact and recruit their recommendations

19. Note about Nonprobability Samples • Conclusions cannot be drawn about the generalizability of sample results to the target population because the sample representativeness cannot be evaluated • Such conclusions would be biased, by definition

20. Sample Recruitment and Selection • Process similar to “marketing” • Study must be promoted to the potential participants • “…the communication link between sellers and buyers for the purpose of influencing, informing, or persuading a potential buyer's purchasing decision” (Boone and Kurtz, 2011).

21. Method of Initial Contact • Ideally use the same method of communication that will be used for the data collection • in-person • traditional mail • telephone • internet • Initial contact can also be made through referrals by professionals (e.g., physicians, pharmacists, therapists)

22. Method of Initial Contact (cont.) • Introductory letter, script for a phone call or in-person contact, and brochures • Should be appropriate reading level and language for the target population • Must be reviewed and approved by the IRB

23. Informing the Potential Participant(for the purpose of persuasion) • Task necessary even before the informed consent process • Who is conducting the study (e.g., funding agency, investigator’s institution, name and contact information) • What is the purpose of the study and plan for disseminating results • What, generally, will be asked of the subject • When will the data collection take place • Where will the data collection take place

24. Elements of the Persuasion • Significance • Improving health for subject and/or society • Potential subject will contribute to the improvement • Legitimacy • Funding agency • Credentials of the investigator(s) • Protection • Confidentiality • Monitoring by the IRB

25. Incentives to Participate • Cash/material incentives are not considered benefits in the IRB review • BUT, they are very powerful persuaders for study participation • Proper value of incentive • Enough to be attractive • Not so much to be coercive

26. Screening • Often subject eligibility must be determined • Evaluate the presence of inclusionary criteria and absence of exclusionary criteria • Proper time to screen is after potential subjects have expressed willingness to participate • Details of the screening depend on the research question

28. Thinking Ahead to Retention • Value of cohort and experimental studies rely on retention of subjects from baseline to follow-up(s) • Incentive structure to maximize retention • Perhaps increasing value with each subsequent follow-up • Screening process can also include the collection of, sometimes extensive, contact information • Contact information should be verified and possibly updated through the course of the study

29. Validity of the Sample • Internal validity – extent to which the sample is a good representation of the study population • External validity – extent to which the sample is a good representation of the target or theoretical population • Without internal validity it is impossible to have external validity

30. Internal and External Validity Theoretical Population External Validity Study Population Internal Validity Sample

31. Threats to Validity • External threats • Improperly defined target population • Incomplete, inaccurate, or inappropriate sampling frame • Internal threats • Sampling errors • High non-response • High study attrition

32. Types of Threats • Selection bias - systematic error committed when sampling from the sampling frame; if a random procedure is planned and claimed but is not conducted properly, then selection bias may be the result • Non-response bias - selected subjects who consent to participate in the study are systematically different in a way that may affect results from selected persons who do not participate • Attrition bias - a systematic relevant difference between those retained and those lost to follow-up in cohort and experimental studies

34. Sample Size • How many subjects are needed for the study? • Statistical power – sample size is large enough to be able to detect hypothesized results that are statistically significant • Too small – limited statistical power • Too large – even weak and clinically insignificant results are found to be statistically significant • Related to Type I and Type II errors

35. Type I Error • Type I error - the error we make when we REJECT a TRUE null hypothesis (incorrectly concluding we HAVE support for our alternative hypothesis when we actually should NOT reject the null hypothesis) • Can be seen as a false positive result • Assign a maximum probability for making this error, called α, as usually 0.05 or no greater than a 5 percent chance of making this mistake • Alternatively, we are 95 percent confident that our result is not due to chance and may be a real result

36. Type II Error • Type II error - error we make when we do NOT reject a null hypothesis when the null hypothesis is really FALSE (incorrectly concluding we do NOT have support for our alternative hypothesis when we actually SHOULD reject the null hypothesis) • Can be seen as a false negative • Assign a maximum probability for making this error as usually 20 percent • Probability is represented by β, and (1-β) is referred to the statistical power or a study. • Want to have at least an 80 percent chance of rejecting a false null hypothesis • Want to have 80 percent power to detect real results in our study

37. Types I and II Errors

38. Sample Size and Statistical Power • Statistical power is largely dependent on sample size • Example of role of n in a confidence interval formula • The confidence interval around the mean estimate includes the standard value of Z (representing the alpha of 0.05) and sample size (n) in the denominator • The larger the n, the smaller the interval will be. The smaller the interval, the more precise is the estimate • Knowing or assuming the sample mean and standard deviation (σ) and using the Z value with 95 percent confidence, we can solve for n using the above formula • Solving for n is a power analysis.

39. Power Analysis • Precise analysis depends on the study design and appropriate assumptions • Examples follow • Assumptions are educated guesses about values needed to solve for n • Confidence level is typically 95% • Desired power is usually 80%