1 / 26

Sampling Decisions in Your TC Program Evaluation

Sampling Decisions in Your TC Program Evaluation. TCEC. Overview. Why sampling? Terms Sampling methods Sample size Sample size and analysis Answers to program questions Resources Announcements. Why sampling?. Evidence from data drive the program

ilya
Download Presentation

Sampling Decisions in Your TC Program Evaluation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Sampling Decisionsin Your TC Program Evaluation TCEC

  2. Overview • Why sampling? • Terms • Sampling methods • Sample size • Sample size and analysis • Answers to program questions • Resources • Announcements

  3. Why sampling? • Evidence from data drive the program • Census (collecting data from each entity such as stores, tenants, population) may not be possible • A representative sample can do the trick

  4. Terms • Population = Entire group from which to collect data (e.g. county population; stores in a certain area, all city parks, etc.) • Census = all members of the population • Sample = a portion of the population

  5. More terms • Sampling unit = the objects, people, timeslots, etc. that are being sampled (stores, residents, casino patrons, etc) • Sampling frame = How you derive your sample (phone book, street corner traffic…)

  6. Sampling Methods • Simple Random • Stratified Random • Cluster • Convenience • Purposive

  7. Simple Random Each member of the population has the same chance of being in the sample

  8. Stratified Random A sample of equal size is drawn from different sub groups of the population Use example: TRL: different neighborhoods that have different foreseeable consequences if TRL has passed – you want equal representation of all groups

  9. Cluster Census of All members of the population in a group, e.g. all tobacco retailers A B C A = rural B = urban C = college town

  10. Purposive • Who or what you sample depends on the purpose of the use for the data • Examples • KIIs generally • POP: only those who smoke at the fair • Observation: only parks with tot lots

  11. Convenience • Reason: limited time and capacity (give reason) • Examples: • Intercept public opinion surveys • YTPS only when youth are available to do stings • Only housing residents who come to a housing association meeting

  12. Sample Size • Sample size= The number of population members you will use to collect data • Confidence level = The level of certainty (usually set at 95%) that your sample represents the whole population • Confidence interval = The percentage of error you expect in your results

  13. Population Confidence Confidence Sample size level interval size 150 95% 10 59 150 95% 6 96 150 95% 4 120

  14. Question What is the minimum POS sample size to make statistically valid decisions for a population of 1.8 million? Go to http://surveysystem.com Research Aids Sample Size Calculator

  15. Sample size calculator Determine Sample Size Confidence Level: _x_ 95% __99% Confidence Interval: 5 (2) Population:    1,800,000  Sample size needed: 384 (2398)

  16. What it means for analysis Example Survey Question: Please say if you agree or disagree with the following statement: “Smoking can shorten a person’s life.” __ Agree __ Disagree Let’s say 75% of those asked said “agree.” Analysis at confidence interval 5: We can say with 95% certainty that between 70 and 80 % (75 plus/minus 5) of the population in the county agree that smoking can shorten a person’s life. Analysis at confidence interval 2: We can say with 95% certainty that between 73 and 77 % (75 plus/minus 2) of the population in the county agree that smoking can shorten a person’s life.

  17. How to report In evaluation plan A public opinion survey will be conducted with a sample size of 384 county residents randomly selected from the phone book (or “through a convenience sample”), for a 95% confidence level and a + 5 % confidence interval for a total of 1.8 million county residents. (Note 1: you have to start out with a bigger sample size since a large percentage will decline to participate) (Note 2: if you analyze sub-groups of your survey, let’s say an ethnic groups’ collective responses, your sample size is much smaller)

  18. Determining statistical power afterwards Resource: http://www.greatbrook.com/survey_statistical_confidence.htm

  19. Requirements Q: Do I need to do sample size calculations? Reviewers like to see it but will not require it. An estimated sample size is acceptable if it is at an “acceptable” rate, with acceptable referring to reasonableness with regards to available resources (in this case “200” will usually satisfy reviewers).

  20. Other sample sizes (ss) • KII policy makers: ............(5-6) • POS w/MUH residents: ……use calculator/ estimate ss • Observation people at event:……………………………estimate total event participants/ calculate or estimate ss • Tobacco litter …………………See “Sampling Plan” in Tips and Tools # 8, Observation

  21. Representativeness Sampling method determines representativeness Less likely in Convenience sample (e.g. intercept survey) More likely in Random samples

  22. Question How do you deal with small sample sizes in a survey? If the survey is not yet completed – add to the sample If survey is completed – report results, add limitations explanation; acceptance of results is still quite possible

  23. Planning versus Analysis What to focus on during Planning “population” size, sample size, sampling method, Analysis Representativeness, limitations

  24. Resources TCEC website: • Tips and Tools # 8: Observation • Tips and Tools # 9: Sampling Decisions • OTIS project plans and reports Individual assistance: • Call TCEC at 530-752-9951 • E-mail at tobaccoeval@ucdavis.edu

  25. Announcements • New Evaluation Associate • TCEC will be on facebook • A webinar on Cost-effective TC Evaluation this Thursday at 10 (same phone number and website)

More Related