1 / 25

Workshop 4c Data Collection

Workshop 4c Data Collection Stephanie J. Battles Energy Information Administration Dhaka, Bangladesh September 22, 1999 Data Collection Mail Survey Personal Interview Telephone Interview: Computer-Assisted Telephone Survey (CATI) Other Modes of Data Collection

adamdaniel
Download Presentation

Workshop 4c Data Collection

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Workshop 4cData Collection Stephanie J. Battles Energy Information Administration Dhaka, Bangladesh September 22, 1999

  2. Data Collection • Mail Survey • Personal Interview • Telephone Interview: Computer-Assisted Telephone Survey (CATI) • Other Modes of Data Collection • Multiple Modes of Data Collection • Survey Supplement Using A Subsample • Nonsampling Error • Nonresponse • Response

  3. Advantages Cost Effectiveness Small Staff Same Mailing Cost for 1 Question as it Costs for 100 Easily Target Population of Interest Studies Show That More Answers Are Provided Disadvantages Lower Response Rates Need to Have Good Planning Need Complete Mailing List No Opportunity for Follow-up Need Questions that are Simple and Short Legible/Easy to Follow Correct Language Can Respondent Read? Mail Survey(Self-Reported Data) • Design and Format Crucial • Future--CSAQ

  4. Mail SurveyExample: 1998 MECS • Three Different Forms, Forms A,B, and C • Form A: Collects Basic Data • Form B: Petroleum Refining • Form C: Establishments Who Ship Energy Offsite • 18,000 Establishments • Average Burden per Response--8.24 Hours • U.S. Census Bureau--Contractor • Two Mail Follow-Ups • Additional Telephone Follow-Up for the Largest Establishments Not Responding

  5. Advantages High Response Rates Are Possible Survey Questions Can be More Complex Opportunity for Follow-Up Questions Can Assist Respondents Who are Disabled, Aged or Other Problem Disadvantages Costly Need a Large Staff or Contract Personnel Natural Disaster Lose Portion of Sample May Not Get Person Most Capable of Answering the Questions Sample May Be in Restricted area Personal Interview(Face-to-Face)

  6. Personal Interview(Face-to-Face) • Interviewer Training (Classroom/Self-study/Both) • Introductory Letter Before Initial Contact • Purpose of the Survey • Why Participation is Important • Confidential Responses • Exhibit Cards • Multicontacts • Callbacks at Different Times of the Day • Multiwave • Reassigned Different Interviewers • Use of Incentives

  7. Personal InterviewExample-1997 RECS • Trainer Training • Three Days • Interviewer Training • Three Days (2 Sessions)/220 Interviewers • Washington D.C. • Topics Covered: • General Interviewing Techniques • RECS Background • Introduction to the CAPI system • Practice With Mock Interviews • Review of Administrative Requirements All Training Sessions Were Monitored by EIA Staff

  8. Personal InterviewExample-1997 RECS • Training Manual • Locating Exercises • Identifying Respondents and Getting In the Door • Mock Exercises (one person is the interviewer..the other is the respondent) • Instruction Manual • Administrative Procedures • Identifying RECS Households • Conducting the Interview • Question by Question Discussion • Training Used Computer (CAPI) • Letter Sent Before Initial Contact

  9. Personal InterviewExample-1997 RECS • Data Collection Results • Average Time: 29 Minutes • Interviewers Conducted an Average of 27 Interviews • Median Number of Visits--3 • 20% of the Interviews Were Were Verified by Mail or Telephone • Used Multicontact Approach • Different Time of Day • Used Multiwave Approach • Changed Interviewer • Mail Follow-Ups for Nonresponsive Households • Short Self-Administered Questionnaire • Sent Pen as an Incentive

  10. Personal InterviewExample-1997 RECS • Data Collection Response • 8,319 Listed Units--7,285 Eligible • Ineligible • Out of Scope, e.g. Business, Not Habitable • Vacant or Seasonal • 5,902 Completed (81% Response Rate) • 5% of the Personal Interviews Completed After the Second Wave (299) • 3 % of the Interviews Completed After the Mail Follow-up • 1,421 Households--Returned 10 % (181)

  11. High Response Rates South (81.1%) Midwest (80.2%) Rural Areas (85%) Single Family and Mobile Homes (80.5%) Low Response Rates Northeast (73.1%) Urban/Suburbs (76.4%) Large Apartment Buildings (72.1%) Personal InterviewExample-1997 RECS Response Rates and Household Characteristics

  12. Advantages Increase Quality Speed Programmed to Correctly Skip Automatically Edit Replies Produce Statistics Quickly Eliminates the Keying of the Data Disadvantages Costly Savings When the Survey is Large and/or Frequent and Questions Remain the Same Lower Response Rates Answering Machines No Telephone Up Front Costs Programming Telephone Interview(CATI) Most Telephone Surveys are CATI (Computer-Assisted Telephone Interviews)

  13. Other Modes of Data Collection BLS: Current Employment Statistics • Touchtone Data Entry (TDE) • Alternative to Mail Collection • Questionnaire is on a Computer • Respondent Answers Prerecorded Questions Using the Touchtone Buttons • Useful for a Few Numbers • Eliminates Key Entry • Voice Recognition (VR) • Recognizes Numbers “0 to 9”, “Yes” and “No” • Useful for A Few Numbers • Accent Recognition Problem Banking

  14. Other Modes of Data Collection • Facsimile Transmission (FAX) • Eliminates Mail Delays • Electronic Data Interchange (EDI) Collect Large Volumes of Data • Electronic Transmission is in a Standard Format • Errors are Reduced • Data are Instantly Available • Email Transmission • Paperless Collection • Eliminates Mail Delays • Errors/Costs Are Reduced • Confidentiality Issues EIA’s Financial Reporting System

  15. Multiple Modes of Data Collection • Survey Requires the Collection of Data from Different Respondent • Increase Survey Response Rates • Respondents May Favor One Mode Over Another • Certify Data Collected Are Correct • Reduce Costs

  16. Multiple Modes of Data CollectionExample: 1993 RECS • Personal Interview • Household/Housing Unit Data • Authorization Form • Mail Survey • Household Mail Survey • Obtain Data for Households Who Refused to Participate in the Personal Interview • Energy Supplier Survey • Natural Gas, Electricity, LPG, and Fuel Oil • Obtain Energy Consumption and Expenditure Data from the Household’s Energy Suppliers

  17. Multiple Modes of Data CollectionExample: 1993 RECS • Telephone Survey • Rental Agent Survey • Verify Information Received From Certain Households Who Did not Pay for All Their Energy or Paid A Third Party and Rented or Owned Living Quarters in A Multi-Unit Building

  18. Survey Supplement Using A Subsample • Obtain Detailed Data • Use if Too Costly for the Full Sample • Use if Response Burden is a Concern • Use Data to Develop a Model Based on Data Available From the Full Survey

  19. Survey Supplement Using A SubsampleExample: 1993 RECS Lighting Supplement • Purpose of the Supplement • Collect Data to Develop a Model to Predict Electricity Consumption for Lighting • Proportional Sample of All U.S. Households • 7% of the Completed Interviews • Lighting Model Used in Electricity Regression Equation Along Other End-Use Models, e.g. Space Heating.

  20. Nonsampling ErrorUnitNonresponse Error Unit Nonresponse Error: Unit nonresponse error occurs when a sampling unit does not participate in the survey , e.g. household, building or establishment. • Unit Nonresponse • Refusal/Failure to Return Questionnaire • Sensitive Information • Incorrect Address • Not Have Resources or Time To Reply • Not Interested • Not Like the Sponsor • Could Not Be Located • Natural Disaster • Recently Vacant/Destroyed • Telephone Disconnected

  21. Nonsampling ErrorUnit Nonresponse :Control of Error • Sampling Unit’s Record Carefully Labeled and Transmitted • Prior Notice • Follow-Up • Incentives • Questionnaire Design • Confidentially Ensured • Reweighting the Sample

  22. Nonsampling ErrorItemNonresponse Error Item Nonresponse Error: Item nonresponse error occurs when a particular item of interest is missing from a questionnaire. • Sensitive Question • Confidentiality Issue • Recall • Aged or Disabled • Fatigue/Burden • Question Long and Complex • Open Question • Records Not Available • No Advance Notice • Interviewer Error • Programming Problems

  23. Nonsampling ErrorItemNonresponse:Control of Error • Interviewer Training • Confidentiality Reminder • Use Memory Aids and Cues • Question Shorts and Simple • Questionnaire-Reasonable Length • Personal Interview, e.g. 60 Minutes • Understandable Skip Patterns • Limit Open Questions • Imputation

  24. Nonsampling ErrorResponse Error Response Error: Response error occurs when the item response is answered incorrectly. • Satisficing Answer • Doesn’t Care/Time Burden • Inability to Answer--A Guess • Deliberate Incorrect Response • Sensitive Question • Respondent Knows the “Correct Answer” • Invalid/Out of Date Records • Interviewer Incorrectly Records Response • Interviewer Bias

  25. Nonsampling ErrorResponse Error:Control of Error • Importance of the Survey and Accurate Responses • Use Memory Aids/Examples • Ensure Confidentiality of Responses • Advance Notice • Interviewer Training • Locate the Respondent With the Proper Knowledge • Quality Control of the Interviewing Process • Allow Respondents to Ask Questions

More Related