1 / 16

Coverage error

Coverage error. Survey Research and Design Spring 2006 Class #3. Today’s objectives. Understand coverage error and how to handle it or reduce it Explain application exercise #1 (handout)

terry
Download Presentation

Coverage error

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Coverage error Survey Research and Design Spring 2006 Class #3

  2. Today’s objectives • Understand coverage error and how to handle it or reduce it • Explain application exercise #1 (handout) • Work in groups to develop constructs, target population, sampling frame, and to discuss possible issues of coverage Survey Research and Design (Umbach)

  3. Survey Process Define Research Objectives Choose Mode of Collection Choose Sampling Frame Construct and Pretest Questionnaire Design and Select Sample Recruit and Measure Sample Code and Edit Data Make Postsurvey Adjustments Perform Analysis Survey Research and Design (Umbach)

  4. Quality perspective (Groves, et al., p. 48) Measurement Representation _ Y Target Population m1 Construct Coverage Error Validity _ yc Measurement Yi Sampling Frame Sampling Error Measurement Error _ ys Sample Response yi Nonresponse Error Processing Error _ yr Respondents Edited Response yip Adjustment Error Postsurvey Adjustments _ yrw _ yprw Survey Statistic Survey Research and Design (Umbach)

  5. What is coverage error? • Gap between the target population and the sampling frame • The result of not allowing all members of the survey population to have an equal or nonzero chance of being sampled for participation • People that can never be sampled (telephone survey/no telephone) • Exists before the sample is drawn Survey Research and Design (Umbach)

  6. Discussion question • What do you do when your sampling frame misses your target population entirely or partially? • Redefine target population to fit frame better • Admit the possibility of coverage error in statistics describing original target population • What should you do? What are the tradeoffs? Survey Research and Design (Umbach)

  7. Coverage properties of frames • Four possible coverage problems: • Undercoverage • Ineligible units/overcoverage/foreign elements • Duplication • Clustering Survey Research and Design (Umbach)

  8. Example of coverage error • The CIRP Freshman Survey • Survey overview • Administration • Instrument Survey Research and Design (Umbach)

  9. Exercise • A day care center is conducting a survey of parents. They placed the notice and survey in each child’s folder. (Each child has a folder for communications from the center to the parents – parents are supposed to check the folder daily.) • A school refuses to give me contact information for students. Using the current semester’s course catalog, I randomly select courses that list an instructor’s name, contact the instructors for permission to administer my survey when their course meets, and then administer the survey in each course where the instructor agrees. Please discuss with your partner any coverage error that the two surveys might have. Survey Research and Design (Umbach)

  10. Target populations and their frame issues • Households and persons • Customers, employees or members of an organization • Organizations Survey Research and Design (Umbach)

  11. Dealing with coverage error • Remember, coverage error affects your statistics, so it depends on • The proportion of population not covered • Difference in statistic between covered and not covered elements • Strategies: • Thoroughly understand list generation • Dillman’s five questions Survey Research and Design (Umbach)

  12. Practical questions to ask about lists (Dillman, 2000) • Does the list contain everyone in the population? • Does the list include names of people who are not in the study population? • How is the list maintained and updated? • Are the same sample units included on the list more than once? • Does the list contain other information that can be used to improve the survey? Survey Research and Design (Umbach)

  13. Dealing with coverage error • Multiple frame/mixed-mode design • Filter questions • CIRP Freshman Survey example • Benefits of reducing coverage error have to be weighed against their costs Survey Research and Design (Umbach)

  14. Group project • What is the objective of your survey? Describe your constructs. • What is your target population? • What is your sampling frame? • What types of coverage issues might you expect? Survey Research and Design (Umbach)

  15. For next week… • Sample design and sampling error • Readings • Groves et al. Ch. 4 • Urdan (psych reserves) • NO Lohr reading Survey Research and Design (Umbach)

  16. One-minute paper • What was the most important thing you learned during this class • What important question remains unanswered? Survey Research and Design (Umbach)

More Related