Sampling for Surveys

1. Sampling for Surveys

2. Surveys What is a survey? • A process of presenting a standard series of questions to a sample of persons. • The survey is the most widely used technique in criminology because it is best suited for looking at the complex social world. To capture that world accurately, we have to measure it “in situ.” … That means taking information from selected people, from where they are usually found. • Measures of many phenomena of interest are taken. The purpose is to accurately reflect the beliefs, attitudes, and behaviors of the sample in order to generalize accurate information to a target population.

3. Surveys • Survey research typically uses sampling rather than taking census. Sampling vs. Taking a Census • Sampling: selecting cases (elements)—or locating people (or other units of analysis)—from a target population in order to study the population. • Taking a Census: using all cases in an entire target (all elements) in order to study the population So why don’t we always take a census? • A “Sample” is a: • Noun: the group from whom data are (or were) gathered, and • Verb: to select cases that represent a population—not a musical term here • There are multiple ways to sample, but the goal is for the sample to maximally represent the target population

4. Sample vs. Population Sample Population

5. Sampling • Types of Samples (Multiple units of analysis can be sampled): Cases Persons in Field Studies Situations Archival Data Experiment Participants Persons answering a Survey • Depending on how the sample was generated, there are limits to how much findings can be generalized from it. • One aims for broad generalizability, but type of sampling is also determined by the: • complexities of the target population, and • researchers’ resources

6. Sampling Sampling Techniques • Nonprobability: Sampling methods that do not let us know in advance the likelihood of selecting for the sample each element or case from a population vs. • Probability: Sampling methods that allow us to know in advance how likely it is that any element of a population will be selected for the sample Knowing the chance of selection allows one to control sampling bias (under or overrepresentation of a population characteristic in a sample)

7. Nonprobability Sampling • Nonprobability (Very common in psychology, medicine, sociology) 1. Availability Sampling, convenience sampling Selection of cases based on what is easiest to do • Experiments • Exploratory and Qualitative research • Avoid this if you can 2. Quota Sampling Aspects of target population are known. Selects availability sample ensuring that it reflects known aspects of population

8. Nonprobability Sampling 3. Snowball Sampling Respondent-driven sampling, initial respondents refer others to the researcher • Usually used with hard-to-discover populations • Bias introduced by structured nature of affiliation • Can be improved with incentives to subjects to recruit a certain number of new respondents 4. Purposive Sampling Targets select people for a sample because of their unique position • Helps get understanding of systems or processes or information on a target population • Not representative of population in general

9. Nonprobability Sampling Critiques • Limited generalizability—one cannot judge representativeness. • Researchers should estimate who the sample represents . . . The sample at least represents populations that are similar to it. • Why use nonprobability samples? Nonprobability does not mean, “intentional attempt to get a sample that is not representative:” • Well-suited for exploratory and evaluation research • Sampling frames (lists from which samples are drawn) are at times inadequate or nonexistent • Quick, efficient • Can be effectively used to study and describe social and social psychological “processes” • Any research is limited, but not having research is worse. • Across samples, repeatedly finding the same results supports generalizability.

10. Probability Sampling Sampling Techniques • Probability Sampling: Sampling method reveals in advance the likelihood that any one element will be selected for the sample • Probability sampling begins with a sampling frame, or a list of all elements (or other units containing the elements) in a population. • E.g., Phone book, All Universities, Known Addresses, Subscribers to a magazine. • If a sampling frame is incomplete (which they usually are) then the accuracy of the sample is compromised. The researcher has the burden of assessing the sampling error or bias.

11. Probability Sampling 1. Simple Random Sampling Cases are identified strictly on the basis of chance. • Random number table to select from sampling frame • Random digit dialing • Equal probability of selection 2. Systematic Random Sampling First case selected randomly from list, subsequent cases are selected at equal intervals. • Typically the same as Simple Random Sampling • Be aware of periodicity

12. Probability Sampling 3. Cluster Sampling • Use when sampling frame is difficult to obtain, but clusters are identifiable. • Randomly select clusters, then use obtainable sampling frames within the clusters to select cases. Example: There is no national list of independent Baptists, but almost all independent Baptist churches can be identified. Members can be selected from membership lists. • Because clusters are generally homogeneous (e.g., all white churches) it is better to maximize the number of clusters and minimize number of cases from each cluster

13. Probability Sampling • Multistage cluster sampling • Selecting clusters in two or more hierarchical stages (e.g., selecting states, then selecting churches, then members) • Keep stages to a minimum because each stage produces sampling error; more stages, more error

14. Probability Sampling 4. Stratified Random Sampling Sampling frame divided into strata, cases drawn from each stratum randomly. • Small subpopulations of interest may yield too few cases in simple random sampling. To compensate, the researcher draws samples from each subpopulation independently. Example: Latino population of Santa Clara County is around 25%. A random sample of 100 would produce 20 – 30 Latinos—too few to generalize to Santa Clara County Latinos.

15. Probability Sampling 4. Stratified Random Sampling • Proportionate Stratified Sampling Ensuring that population proportions are reflected in proportions of each stratum of sample. • Population: 4% black, 25% Latino, 27% Asian, 44% white • Sample of 1,000: 40 black, 250 Latino, 270 Asian, 440 white • Disproportionate Stratified Sampling Population proportions are NOT reflected in proportions of each stratum of sample. • Population: 4% black, 25% Latino, 27% Asian, 44% white • Sample of 1,000: 250 black, 250 Latino, 250 Asian, 250 white • Idea is to get a lot of cases in each stratum • When combining all cases into one sample, use weighted averages

16. 2010 GSS Sampling • Full probability sample of US households—each household has an equal chance of being selected • Used stratified area probability sampling • At the household level, 1 adult is selected at random (Kish Table) • Sampling frame • Most cases came from a list of addresses from USPS (over 2/3) • Remaining cases from NORC-generated lists of households

17. 2010 GSS Sampling Stages used in four population area types, ending with random adult • Big MSAs (city), have USPS address—42% of population • Primary sampling unit: tract (1 -2K Housing units)—168 selected • Housing Unit Selected from USPS List • Intermediate MSAs or counties, have USPS address—30% of population • Primary sampling unit: MSA or part of county—30 selected • Secondary sampling unit: tract—120 selected • Housing unit Selected from USPS List • Rural counties and Intermediate areas (2) without adequate USPS address list—25% of population • Primary sampling unit: County, all or part—25 selected • Secondary sampling unit: Segment (constructed to contain 300 Housing Units)—100 selected • Housing unit from NORC-listed master • Big MSAs (city), without adequate USPS address list—3% of population • Primary sampling unit: Segment—12 selected • Housing unit from NORC-listed master

18. 2010 GSS Sampling Source: http://www.fcsm.gov/03papers/keynotespeaker.pdf, January 12, 2012 Stratum 3 = NORC list used

19. 2010 GSS Sampling • What are the implications of the General Social Survey’s sampling??? • The GSS is an adults-only survey of persons in households. Therefore, it underrepresents: • 18 – 24 year-olds (many not living in households—military, college, roaming) • 65 and over (many not living in households—vacations, RVs, assisted living) • Persons who live in large households (only one person per household is interviewed) • Homeless, criminal, and some poor (not in official households, in shelters, on streets, in apartments)

20. Probability Sampling Critiques • Just being random does not ensure that a sample is representative or that the research is good. • Limited Sampling Frame • Think of presidential phone polls: • Who is at home? Type of person, day of polling, etc. • Who has a land line? • Problems of non-response—random non-response okay, but systematic non-response is biasing • Phone surveys typically do not report response rate. They are often below 30% • How were questions worded: Measurement error • Problems of misspecified models: Leads to not asking the right questions

21. Probability Sampling Critiques • Is the Sample large enough? • Larger samples produce less sampling error • Too large is a waste of money • Big is good, but accurate and appropriate are better • Fraction of population sampled does not increase accuracy unless fraction is very large • Larger samples are needed when: • The population is more heterogeneous. • There are more variables of interest. • The weaker the effects, or the smaller the differences between groups, TO SUM: MORE COMPLEXITY REQUIRES LARGER SAMPLES