1 / 59

SURVEY LIFECYCLE

Vicki Crompton Mike Sivyer . SURVEY LIFECYCLE. STC SURVEY PROGRAM. The financial perspective…. Core statistical programs dictated by statutes, regulatory instruments and contractual obligations account for 93% of STC budget

glipton
Download Presentation

SURVEY LIFECYCLE

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Vicki Crompton Mike Sivyer SURVEY LIFECYCLE

  2. STC SURVEY PROGRAM The financial perspective…. • Core statistical programs dictated by statutes, regulatory instruments and contractual obligations account for 93% of STC budget • $510 million in base budget supplemented by $130 million in supplementary revenues ($45 million in cost-recovery surveys) • Social programs account for 65% of base budget, 75% of supplementary revenues

  3. STC SURVEY PROGRAM The statistical perspective… ACTIVE SURVEYS Social Surveys: 93 Economic Surveys: 292

  4. STC SURVEY PROGRAM The rationale………. • Produce information to support statutory and regulatory purposes • Produce information on the economic and socio-demographic structure of Canada • Produce information needed to support informed policy formulation, decision-making, research, and discussion • Produce information required to meet emerging issues and new challenges

  5. STC SURVEY PROGRAM Maintaining relevance………. • National Statistics Council • Network of subject-specific professional Advisory Committees • Federal Provincial Consultative Council • Stakeholder consultations

  6. STC SURVEY PROGRAM The deciding factors………. • Relevance • Public benefit • Cost • Response burden • Public sensitivies

  7. MAJOR COMPONENTSOF A SURVEY

  8. What the heck is a survey?

  9. WHAT IS A SURVEY? • The collection of information • From all or some units of a population • Employing well-defined concepts, methods and procedures • The compilation of such information into useful summary form

  10. CENSUS SAMPLE TYPES OF SURVEYS Includesall units of the population Population = total set of units to which survey results apply Includes some units of the population Sample = a representative part of the entire population used to reflect the entire population

  11. In Statistics Canada we also categorize by: Business vs Household vs Institution Voluntary vs Mandatory Interviewer Administered vs Self-completed Administrative Records vs Direct Collection Cross-classified vs Longitudinal Ad Hoc vs On-going/Core NOTE: These are NOT mutually exclusive! TYPES OF SURVEYS

  12. EXAMPLES OF SURVEYS WITHIN STC Regular (on-going/core) Surveys -Labour Force Survey - Retail Commodity Survey - Census of Population - Unified Enterprise Survey - Survey of Household Spending - Annual Survey of Manufactures - General Social Survey - Survey of Steel Inventory - Etc.

  13. EXAMPLES OF SURVEYS WITHIN STC Special (ad hoc/irregular) Surveys- Youth Smoking Survey- Survey of Graduates Who Moved to the USA- Child Care Survey- Household Internet Use Survey- Survey of Approaches to Educational Planning- Adult Education and Training Survey - Survey of Importance of Wildlife to Canadians - International Adult Literacy and Skills Survey

  14. EXAMPLES OF SURVEYS WITHIN STC Longitudinal Surveys - Survey of Labour and Income Dynamics - National Population Health Survey - National Longitudinal Survey of Children and Youth - Workplace and Employee Survey

  15. User Consultation and Content Determination Questionnaire Development Data Collection Data Processing Data Quality Measurement Dissemination

  16. Reasons for collecting the data Elaboration of information requirements How will the information/data be used? Identification of Population/Target Group What is the population to be studied? Are there any sub populations to be identified? SURVEY OBJECTIVES

  17. Identify Units of Analysis Individual Household Family - (Census or Economic) Business - Enterprise - Company - Establishment SURVEY OBJECTIVES

  18. Analysis Plan - Purpose/uses of the data - Statement of outputs Level of Geography Required for Analysis - National vs provincial - Urban/rural - Cities/economic regions Level of accuracy required SURVEY OBJECTIVES

  19. Feasibility of collecting data required Type of survey required (census/sample) Resources required Training Costs SURVEY DESIGN ISSUES

  20. Method of Collection Paper and Pencil Interview (PAPI) Computer Assisted Interviewing (CAI) By interviewer By respondent (incl. electronic transmission) Proxy vs Non-proxy Personal vs Telephone Interview vs Self-completed Special Testing Applications SURVEY DESIGN ISSUES

  21. Target/survey population Sampling units Population must be divisible into distinctive non-overlapping units called sampling units Each member of the population should belong to just one sampling unit Size of sample required Sample selection methods Sample Frame SAMPLE PLAN

  22. Means to access survey population A list containing the survey population For Example: A printed telephone book A computer database of registered motor vehicles Business Register Address Register Should contain all the relevant sampling units and all necessary and relevant information SAMPLE FRAME

  23. Sample of other surveys Labour Force Survey sample (supplements) Census of Population (post-censal) Canadian Community Health Survey Geographic Frames Lists of EAs or CMAs from census files SAMPLE FRAME

  24. First perform a literature search Other sources which asked similar questions Other surveys in STC Surveys from other countries Define/develop concepts and definitions Similar to other STC surveys Specific to this survey Easy to understand, be accepted and used by interviewers and respondents. CONTENT DEVLOPMENT

  25. COMPONENTS OF A SURVEY

  26. Wording of questions Easily understood by interviewer and respondent Easy to read - sounds natural – friendly Appropriate level of language Flows and skip patterns Easy to follow Logical to interviewer and respondent Capture all relevant information from respondents CONTENT DEVLOPMENT

  27. Questionnaire layout (print version) Easy to follow? Need additional instructions for Interviewer? CAI Application Ability to program Appropriate questions introduced at correct time User-friendly (for when completed by respondent) CONTENT DEVLOPMENT

  28. Determine which type of question is best for each question asked: Closed question - Identify answer categories to include Open-ended question (write-in) - Coding of write-in responses CONTENT DEVLOPMENT

  29. An important part of all major components Test wording of questions (in both official languages) Are concepts and definitions understood? Level of language Sequence of questions (flows/skips) Length of interview TESTING ACTIVITIES

  30. Focus groups Informal with colleagues Field procedures testing Pilot testing CAI Applications and other related programs TESTING ACTIVITIES

  31. EXTREMELY IMPORTANT SOMETIMES SACRIFICED DUE TO TIME AND COST CONSTRAINTS TESTING ACTIVITIES

  32. Interviewer training Sample Selection and send to Interviewers Send questionnaires /CAI application to Interviewers Necessary equipment in place (laptops, phones, reference materials, etc.) Field collection/ Interviewing Data capture & transmission Respondent relations DATA COLLECTION/FIELD ACTIVITIES

  33. STC Policy requires that we inform respondent of: Purpose of survey Confidentiality assurances Volunteer/mandatory aspects of survey Any projected linking of data Any projected sharing of data Normally start by sending a letter to respondents informing them of survey RESPONDENT RELATIONS ACTIVITIES

  34. Also have an introduction at the start of the interview process on the questionnaire RESPONDENT RELATIONS ACTIVITIES

  35. For business surveys there is a special program called the Key Provider Manager (KPM). The top 300 large complex enterprises in Canada account for one third of all economic activity. This can result in these enterprises being in many surveys leading to “respondent burden” and possible refusal to participate. RESPONDENT RELATIONS ACTIVITIES

  36. Duties of KPM are: To co-ordinate and facilitate flow of information between these enterprises and STC; To negotiate reporting arrangements with enterprise; To explain concepts, definitions reporting requirements; To do follow-up; Etc. RESPONDENT RELATIONS ACTIVITIES

  37. There are a series of steps that include: Editing Imputation Coding Creation of derived variables Weighting calculations Preparation of metadata Final file creation PROCESSING THE DATA

  38. Why edit the data? Data must be logical and consistent. Errors that may have been introduced into the data must be fixed. Errors can include: Interviewer errors Capture errors Respondent errors PROCESSING THE DATA

  39. What is included in editing? Inter-question relationships Paths and flows Coding of write-in responses Imputation for non-response or response error PROCESSING THE DATA

  40. Imputation activities can include: Replacing response error with true value Imputation of non-response Imputation of missing values based on other responses from same record Imputation of missing values with responses from a similar record Replace values for an entire section or record with responses from similar record Remove record from file PROCESSING THE DATA

  41. Imputation of Data from Other Records on File PROCESSING THE DATA

  42. Imputation of Data from Other Records on File PROCESSING THE DATA

  43. During the editing process there is also the creation of “ Derived Variables”. This can be the combination of responses of one question into a more useable format: e.g. Individual income into groups or ranges. Or the combination of two or more variables into a new variable: e. g. “Usually smoke cigarettes” with “Usual number of cigarettes smoked” to create “Type of smoker”: “heavy”, “moderate”, “light”. PROCESSING THE DATA

  44. It is during the editing stage that decisions and actions may be taken to suppress data This is done because of: Confidentiality risk Sensitivity of data Quality/reliability issues PROCESSING THE DATA

  45. One of the final activities of the processing stage is the calculation of the “weights” (sometime called the “sample weight”) The weight variable indicates the number of units in the population that are represented by each unit in the sample The sample weight corresponds to the inverse of the probability of selection of the unit in the sample PROCESSING THE DATA

  46. For example: In a survey of Ontario farms let’s assume that: Farm population = 153,000 (farms) Survey sample required = 9,000 Probability of selection = 9,000 / 153,000 = 1/17 Sample weight = 17 However…. PROCESSING THE DATA

  47. Most samples are based upon a complex sample design, with stratification, multiple stages of selection, and unequal probabilities of selection of respondents Weighting MUST take this into account Weighting also has to take into account any non-response PROCESSING THE DATA

More Related