EMR 6500: Survey Research

1. EMR 6500:Survey Research Dr. Chris L. S. Coryn Kristin A. Hobson Spring 2013

2. Agenda Stratified random sampling for proportions From questions to a questionnaire Budgeting survey research

3. Stratified Random Sampling

4. Estimate of Population Proportion

5. Example for a Population Proportion

6. Example for a Population Proportion

7. Example for a Population Proportion

8. Selecting the Sample Size for Estimating Population Proportions

9. Sample Size for Estimating Population Proportions

10. Sample Size for Estimating Population Proportions As with allocation methods for means and totals, allocation methods for stratified random samples for proportions can be equal, proportional, or optimal depending on the study’s purpose

11. From Questions to a Questionnaire

12. General Premises The design of a questionnaire should consider how to motivate the recipient to respond It should also avoid measurement errors, ranging from order effects to item nonresponse

13. Guidelines for Ordering Questions

14. Ordering Questions • General guidelines • Group related questions that cover similar topics, and begin with questions likely to be salient to nearly all respondents • Choose the first question carefully • Place sensitive or potentially objectionable questions near the end • Ask questions about events in the order that they occurred • Avoid unintended order effects

16. Guidelines for Creating a Common Visual Stimulus

17. Visual Stimulus • General guidelines • Establish consistency in the visual presentation of questions (across pages and screens) and use alignment and vertical spacing to help respondents organize information on the page • Use color and contrast to help respondents recognize the components of the questions and the navigational path through the survey • Visually group related information in regions through the use of contrast and enclosure • Use visual elements and properties consistently across questions to emphasize or deemphasize certain types of information • Avoid visual clutter • Minimize the use of matrixes and their complexity

20. Guidelines for Mail Questionnaires

21. Mail Questionnaires • General guidelines • Determine whether keypunching or optical imaging and scanning will be used, and assess the limitations imposed on designing and processing questionnaires • Construct paper questionnaires in booklet formats, and choose physical dimensions based upon printing and mailing considerations • Decide question layout and how questions will be arranged on each page • Use symbols, contrast, size, proximity, and pagination effectively when designing branching instructions to help respondents correctly execute them • Create interesting and informative front and back pages that will have wide appeal to respondents • Avoid placing questions side-by-side on a page so that respondents are asked to answer two questions at once

26. Guidelines for Web Questionnaires

27. Web Questionnaires • General guidelines • Decide whether an electronic alternative is appropriate • Choose how the survey will be programmed and hosted, commensurate with needs, skills, and sponsorship • Take steps to ensure that questions display similarity across platforms, browsers, and user settings • Decide how many questions will be presented on each page and how questions will be presented • Develop a screen format that emphasizes the respondent rather than the sponsor

28. Web Questionnaires • General guidelines • Use a consistent page layout across screens and visually emphasize questions information that respondents will need to complete the survey while deemphasizing information not essential to the task • Do not require responses unless absolutely necessary • Design survey-specific and item-specific error messages to help respondents troubleshoot • Evaluate carefully the use of interactive features, balancing improvements in measurement with the impact on respondent burden and the implications with mixed-mode surveys

29. Web Questionnaires • General guidelines • Use audiovisual capabilities sparingly, and evaluate the differential effect they have on respondents • Allow respondents to stop the survey and finish completing it at another time • Whenever possible, collect paradata that provide feedback on how respondents interact with questionnaire • Test the survey using a variety of platforms, connection speeds, browsers, and user-controlled settings, and test the database to ensure that items are collected and coded accurately • Take screenshots of each page of the final questionnaire for testing and documentation

32. Guidelines for Pretesting Questionnaires

33. Pretesting Questionnaires • General guidelines • Obtain feedback on the draft of the questionnaire from a number of people, each of whom has specialized of some aspect of questionnaire quality • Conduct cognitive interviews of the complete questionnaire in order to identify wording, question order, visual design, and navigational problems • When the stakes are high, consider experimental evaluations of questionnaire components • Conduct a small pilot study with a subsample of the population in order to evaluate interconnections among questions, the questionnaire, and implementation procedures

34. Budgeting Survey Research

35. General Cost Frameworks for Survey Research • Fixed-price • All costs are subsumed under a single dollar value, no matter time, effort, and other expenditures • Cost-reimbursable • Reimbursement for actual expenditures

36. Direct Costs • Personnel • Hourly staff (+7.65% FICA) • Salaried staff (+ 48.75% for full benefits) • Academic staff (+ 22.25% for full benefits) • Travel • Local transportation, flights, per diem, lodging • Other costs • Materials and supplies (e.g., office and operating supplies, telephone and conference calls, postage and shipping) • Publication and dissemination (e.g., standard printing, copying, production) • Participant costs and/or incentives • Consultants or subawards • Computer services

37. Indirect Costs • F&A (facilities and administration) • 49% is WMU’s current HHS approved rate • 24% is WMU’s current off-campus rate (requires rental space elsewhere) • Other rates can be negotiated based on institutional policies (e.g., existing policy of funders)

38. Typical Budgeting Methods Per task Per day (e.g., %FTE) In budgeting, both are often necessary

39. Consider… • A mail survey to 500 randomly selected recipients • Printing of surveys = $2.00/survey • Postage = $0.45/survey for delivery + $0.45/survey for return • Processing of surveys = 1 hour/survey @ $18/hour • $2/recipient incentive • What is the total cost for just the above, which does not include other costs?