1 / 19

Presentation Structure

Control of Sampling Error Peter Lynn Institute for Social and Economic Research, University of Essex, UK. Presentation Structure. Objectives of Survey Design Survey Error Framework Coverage Error: Control Sampling Error: Control Design Effects: a key tool

arvin
Download Presentation

Presentation Structure

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Control of Sampling ErrorPeter LynnInstitute for Social and Economic Research,University of Essex, UK QMSS, Lugano, 13-8-2004 Lynn

  2. Presentation Structure • Objectives of Survey Design • Survey Error Framework • Coverage Error: Control • Sampling Error: Control • Design Effects: a key tool • Examples from European Social Survey QMSS, Lugano, 13-8-2004 Lynn

  3. Survey Design: Objectives Appropriate accuracy (cf. budget): • For what estimates? • What is appropriate? Estimates: • Typically, descriptives for sub-domains and total, and comparisons between sub-domains (inc. models) Appropriate: • high enough, cost effective QMSS, Lugano, 13-8-2004 Lynn

  4. Survey Errors QMSS, Lugano, 13-8-2004 Lynn

  5. Coverage Error Ideal aim: • Complete coverage • Implies zero coverage error Practical aim: • Very high coverage • Similar coverage for each sub-domain • Hopefully similar (and small) coverage bias QMSS, Lugano, 13-8-2004 Lynn

  6. Example: ESS Target population in each nation (domain): • all persons 15 years or older resident in private households within the borders of the nation, regardless of nationality, citizenship, language or legal status. Under-coverage in practice: • If language is barrier to interviewing; • If some addresses/households excluded (e.g. If electoral registers used as frame of addresses); • If illegal residents excluded (e.g. If population register used as frame). QMSS, Lugano, 13-8-2004 Lynn

  7. Sampling Error Aim: Maximum precision of estimates of between-domain differences Implication: same precision for each domain (for a given estimate) QMSS, Lugano, 13-8-2004 Lynn

  8. Sampling Error Affected by: • Sample size (n) • Population variance (S2) • Sample clustering • Sample stratification • Variable sampling fractions For SRS: Var(y)=(S2/n)(1-(n/N)) Effect of other 3 design features can be summarised by design effect QMSS, Lugano, 13-8-2004 Lynn

  9. Design Effect: A useful tool where ; ; ; QMSS, Lugano, 13-8-2004 Lynn

  10. Example: Use of Design Effects on ESS and where and So, challenge was to predict and for each nation QMSS, Lugano, 13-8-2004 Lynn

  11. Sample Design Process • ESS sampling panel set up • Each nation allocated to a panel member for bilateral liaison • Panel met 3 times and communicated regularly, to ensure consistency of approach • Each design had to be approved by whole panel QMSS, Lugano, 13-8-2004 Lynn

  12. Predicting • Unclustered designs (5) trivial: • For other designs, necessary to decide upon n and number of clusters and to predict eligibility rate and response rate • Under-estimated if RR under-estimated. E.g. Greece • Over-estimated if RR over-estimated. E.g. Italy , Spain , Czech Rep QMSS, Lugano, 13-8-2004 Lynn

  13. Predicting • Many countries assumed default value of 0.02 • A few countries assumed values between 0.03 and 0.05, either based on estimates from earlier surveys or because clustering units were particularly small • Post-fieldwork estimates showed large range across variables QMSS, Lugano, 13-8-2004 Lynn

  14. Predicting Planned variation in sampling fractions over strata (only NR is uncertain); Additionally, 3 forms of uncertainty encountered: • Re. Distribution of # persons aged 15+ per household • Re. Relationship between a proxy size measure used at PSU level and actual size measure of relevance • Re. Relationship between a proxy size measure at hhd/address level and actual number of persons aged 15+ QMSS, Lugano, 13-8-2004 Lynn

  15. Predictions of Zero variation (SE, DK, FI, HU, SI, BE) Known SFs only (DE, NO, PL) From recent surveys (CH, NL, UK) PSU size measures & SFs (IL) Hhold size dist. (AT, CZ, ES, GR) Hhold size measure & SF (LU) QMSS, Lugano, 13-8-2004 Lynn

  16. Overall Deff Predictions Good for many countries Some under-estimates compensated by larger m (GR, SI) Some modest under-estimates, e.g. CZ AT Two severe under-estimates: IL NO Poor RR prediction also affected m, e.g. IT, CZ QMSS, Lugano, 13-8-2004 Lynn

  17. Quality Improvement Round 1 procedures improved quality in several countries and cross-nationally Round 1 estimates will influence round 2 predictions (etc.) Guidelines to be amended in light of round 1 experience (e.g. m, dual designs, default roh) QMSS, Lugano, 13-8-2004 Lynn

  18. Final Comments None of these ideas are unique to cross-national surveys: apply to any comparative survey (= all surveys) We are forced explicitly to consider domain precision aims when fieldwork is organised separately for each domain But we should always do this: sample design should be appropriate for analysis aims QMSS, Lugano, 13-8-2004 Lynn

  19. Control of Sampling ErrorPeter LynnInstitute for Social and Economic Research,University of Essex, UK QMSS, Lugano, 13-8-2004 Lynn

More Related