Determining How to Select a Sample

1. Determining How to Select a Sample

2. Basic Concepts in Sampling • Population:the entire group under study as defined by research objectives • Researchers define populations in specific terms such as “heads of households located in areas served by the company who are responsible for making the pest control decision.”

3. Basic Concepts in Sampling • Sample: a subset of the population that should represent the entire group • Sample unit: the basic level of investigation • Census: an accounting of the complete population

4. Basic Concepts in Sampling • Sampling error: any error in a survey that occurs because a sample is used • A sample frame: amaster list of the entire population • Sample frame error: the degree to which the sample frame fails to account for all of the population…a telephone book listing does not contain unlisted numbers

5. Reasons for Taking a Sample • Practical considerations such as cost and population size • Inability of researcher to analyze huge amounts of data generated by census • Samples can produce precise results

6. Two Basic Sampling Methods • Probability samples: ones in which members of the population have a known chance (probability) of being selected into the sample • Non-probability samples: instances in which the chances (probability) of selecting members from the population into the sample are unknown

7. Probability Sampling Methods • Simple random sampling • Systematic sampling • Cluster sampling • Stratified sampling

8. Probability Sampling Methods

9. Probability Sampling:Simple Random Sampling • Simple random sampling: the probability of being selected into the sample is “known” and equal for all members of the population • E.g., Blind Draw Method • Random Numbers Method (see MRI 12.1, p. 335)

10. Probability Sampling:Simple Random Sampling • Advantage: • Known and equal chance of selection • Disadvantages: • Complete accounting of population needed • Cumbersome to provide unique designations to every population member

11. Probability SamplingSystematic Sampling • Systematic sampling: way to select a random sample from a directory or list that is much more efficient than simple random sampling • Skip interval=population list size/sample size

12. Probability SamplingSystematic Sampling • Advantages: • Approximate known and equal chance of selection…it is a probability sample plan • Efficiency…do not need to designate every population member • Less expensive…faster than SRS • Disadvantage: • Small loss in sampling precision

13. Probability SamplingCluster Sampling • Cluster sampling: method in which the population is divided into groups, any of which can be considered a representative sample • Area sampling

14. Probability SamplingCluster Sampling • Advantage: • Economic efficiency…faster and less expensive than SRS • Disadvantage: • Cluster specification error…the more homogeneous the clusters, the more precise the sample results

15. Cluster Sampling • In cluster sampling the population is divided into subgroups, called “clusters.” • Each cluster should represent the population. • Area sampling is a form of cluster sampling – the geographic area is divided into clusters.

16. Cluster Sampling • One cluster may be selected to represent the entire area with the one-step area sample. • Several clusters may be selected using the two-step area sample.

17. A Two-Step Cluster Sample • A two-step cluster sample (sampling several clusters) is preferable to a one-step (selecting only one cluster) sample unless the clusters are homogeneous.

18. Stratified Sampling • When the researcher knows the answers to the research question are likely to vary by subgroups…

19. Stratified Sampling • Research Question: “To what extent do you value your college degree?” Answers are on a five point scale: 1= “Not valued at all” and 5= “Very highly valued” • We would expect the answers to vary depending on classification. Freshmen are likely to value less than seniors. We would expect the mean scores to be higher as classification goes up.

20. Stratified Sampling • Research Question: “To what extent do you value your college degree?” • We would also expect there to be more agreement (less variance) as classification goes up. That is, seniors should pretty much agree that there is value. Freshmen will have less agreement.

21. Stratified Sampling

22. Probability SamplingStratified Sampling • Stratified sampling: method in which the population is separated into different strata and a sample is taken from each stratum • Proportionate stratified sample • Disproportionate stratified sample

23. Probability SamplingStratified Sampling • Advantage: • More accurate overall sample of skewed population…see next slide for WHY • Disadvantage: • More complex sampling plan requiring different sample size for each stratum

24. Stratified Sampling • Why is stratified sampling more accurate when there are skewed populations? • The less variance in a group, the less sample size it takes to produce a precise answer. • Why? If 99% of the population (low variance) agreed on the choice of Brand A, it would be easy to make a precise estimate that the population preferred Brand A even with a small sample size.

25. Stratified Sampling • But, if 33% chose Brand A, and 23% chose B, and so on (high variance) it would be difficult to make a precise estimate of the population’s preferred brand…it would take a larger sample size…

26. Stratified Sampling • Stratified sampling allows the researcher to allocate more sample size to strata with less variance and less sample size to strata with less variance. Thus, for the same sample size, more precision is achieved. • This is normally accomplished by disproportionate sampling. Seniors would be sampled LESS than their proportionate share of the population and freshmen would be sampled more.

27. Stratified Sampling • Note that we would expect this question to be answered differently depending on student classification. Not only are the means different, variance is less as classification goes up…Seniors tend to agree more than Freshmen!

28. Nonprobability Sampling • With nonprobability sampling methods selection is not based on fairness, equity, or equal chance. • Convenience sampling • Judgment sampling • Referral sampling • Quota sampling

29. Nonprobability Sampling

30. Nonprobability Sampling • May not be representative but they are still used very often. Why? • Decision makers want fast, relatively inexpensive answers… nonprobability samples are faster and less costly than probability samples.

31. Nonprobability Sampling • May not be representative but they are still used very often. Why? • Decision makers can make a decision based upon what 100 or 200 or 300 people say…they don’t feel they need a probability sample.

32. Nonprobability Sampling • Convenience samples:samples drawn at the convenience of the interviewer • Error occurs in the form of members of the population who are infrequent or nonusers of that location

33. Nonprobability Sampling • Judgment samples:samples that require a judgment or an “educated guess” as to who should represent the population • Subjectivity enters in here, and certain members will have a smaller chance of selection than others

34. Nonprobability Sampling • Referral samples (snowball samples): samples which require respondents to provide the names of additional respondents • Members of the population who are less known, disliked, or whose opinions conflict with the respondent have a low probability of being selected

35. Nonprobability Sampling • Quota samples:samples that use a specific quota of certain types of individuals to be interviewed • Often used to ensure that convenience samples will have desired proportion of different respondent classes

36. Online Sampling Techniques • Random online intercept sampling: relies on a random selection of Web site visitors • Invitation online sampling: is when potential respondents are alerted that they may fill out a questionnaire that is hosted at a specific Web site

37. Online Sampling Techniques • Online panel sampling: refers to consumer or other respondent panels that are set up by marketing research companies for the explicit purpose of conducting surveys with representative samples • Other online sampling approaches

38. Developing a Sample Plan • Sample plan:definite sequence of steps that the researcher goes through in order to draw and ultimately arrive at the final sample

39. Developing a Sample Plan • Define the relevant population. • Obtain a listing of the population. • The incidence rate is the percentage of people on a list who qualify as members of the population • Design the sample plan (size and method).

40. Developing a Sample Plan • Draw the sample. • Substitution methods: • Drop-down substitution • Oversampling • Resampling

41. Developing a Sample Plan • Validate the sample. • Sample validationis a process in which the researcher inspects some characteristic(s) of the sample to judge how well it represents the population. • Resample, if necessary.

42. Determining the Size of a Sample

43. Sample Accuracy • Sample accuracy:refers to how close a random sample’s statistic is to the true population’s value it represents • Important points: • Sample size is not related to representativeness • Sample size is related to accuracy

44. Sample Size and Accuracy • Intuition: Which is more accurate: a large probability sample or a small probability sample? • The larger a probability sample is, the more accurate it is (less sample error).

45. A Picture Says 1,000 Words ± n 550 - 2000 = 1,450 4% - 2% = ±2% Probability sample accuracy (error) can be calculated with a simple formula, and expressed as a ± % number.

46. How to Interpret Sample Accuracy • From a report… • The sample is accurate ± 7% at the 95% level of confidence… • From a news article • The accuracy of this survey is ± 7%…

47. How to Interpret Sample Accuracy • Interpretation • Finding: 60% are aware of our brand • So… between 53% (60%-7%) and 67% (60%+7%) of the entire population is aware of our brand

48. Sample Size Axioms • To properly understand how to determine sample size, it helps to understand the following axioms…

49. Sample Size Axioms • The only perfectly accurate sample is a census. • A probability sample will always have some inaccuracy (sample error). • The larger a probability sample is, the more accurate it is (less sample error). • Probability sample accuracy (error) can be calculated with a simple formula, and expressed as a +- % number.

50. Sample Size Axioms • You can take any finding in the survey, replicate the survey with the same probability sample size, and you will be “very likely” to find the same finding within the +- range of the original finding. • In almost all cases, the accuracy (sample error) of a probability sample is independent of the size of the population.