1 / 84

Semester Review

Semester Review. The Tailored Design Method. The Tailored Design Method. Uses multiple motivational features in compatible and mutually supportive ways to encourage high quantity and quality of responses. The Tailored Design Method. Premised on social exchange perspective on human behavior

amandla
Download Presentation

Semester Review

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Semester Review

  2. The Tailored Design Method

  3. The Tailored Design Method Uses multiple motivational features in compatible and mutually supportive ways to encourage high quantity and quality of responses

  4. The Tailored Design Method Premised on social exchange perspective on human behavior Assumes that the likelihood of responding is greater when the expected rewards outweigh the anticipated costs

  5. The Tailored Design Method Gives attention to all aspects of contacting and communicating with respondents Encourages response by considering survey sponsorship, the nature of the population and variations within it, and content of questions

  6. The Tailored Design Method Emphasizes reducing errors of coverage, sampling, nonresponse, and measurement

  7. Coverage Error Occurs when all members of a population do not have a known, non-zero probability of selection Occurs when those who are excluded are different from those who are included

  8. Sampling Error Results from surveying only some rather than all members of a population Represented by B, the bound on the error of estimation

  9. Nonresponse Error • Occurs when people selected do not respond and are different than those who do • Nonresponse can occur at the level of items within a survey or at the level of the survey • MAR • MCAR

  10. Measurement Error Occurs when responses are inaccurate or imprecise Primarily related to poor layout and poor design and wording of questions

  11. Social Exchange and Surveys • Addresses three central questions about design and implementation • How can the perceived rewards for responding be increased? • How can the perceived costs of responding be reduced? • How can trust be established so that people believe the rewards will outweigh the costs of responding?

  12. Increasing Benefits Provide information about the survey Ask for help or advise Show positive regard Say thank you Support group values Give tangible rewards Make the questionnaire interesting Provide social validation Inform people that opportunities to respond are limited

  13. Decreasing Costs Make it convenient to respond Avoid subordinating language Make the questionnaire short and easy to complete Minimize requests for personal or sensitive information Emphasize similarity to other requests or tasks to which a person has already responded

  14. Establishing Trust Obtain sponsorship by legitimate authority Provide a token of appreciation in advance Make the task appear important Ensure confidentiality and security of information

  15. Features that can be Tailored • Survey mode • Singular or multiple • Sample design • Type of sample • Number of units sampled • Incentives • Type of incentive • Amount or cost of incentive • Before or after

  16. Features that can be Tailored • Contacts • Number of contacts • Timing of initial and subsequent contacts • Mode of each contact • Whether contacts will be personalized • Sponsorship information • Visual design of each contact • Text or words in each contact

  17. Features that can be Tailored • Additional materials • Whether to provide them at all • Type of materials (e.g., research report) • Visual design of materials • Text or wording of materials

  18. Features that can be Tailored • Questionnaire • Topics included • Length (duration, number of pages/screens, number of questions) • First page or screen • Visual design • Organization and order of questions • Navigation through questionnaire

  19. Features that can be Tailored • Individual questions • Topic (sensitive, of interest to the respondent) • Type (open-ended versus closed-ended) • Organization of information • Text or wording • Visual design

  20. Coverage and Sampling

  21. Central Terminology An element is an object on which a measurement is taken A population is a collection of elements to which an inference is made from a sample A sample is a collection of sampling units drawn from a frame or frames Sampling units are nonoverlapping collections of elements from the population that cover the entire population A frame is a list of sampling units

  22. Central Terminology • A completed sample is the units that respond • Sampling error is the result of collecting data from only a subset, rather than all, units from a frame • Again, represented by B, the bound on the error of estimation

  23. Coverage The degree to which the units in a sampling frame correspond to the population of interest Coverage is likely one of the most serious problems in most surveys

  24. Coverage and Frame Problems

  25. Reducing Coverage Error • Central questions: • Does the list contain everyone in the survey population? • Does the list include people who are not in the study population? • How is the list maintained and updated? • Are the same sample units included on the list more than once? • Does the list contain other information that can be used to improve the survey?

  26. An Overview of Crafting Good Questions

  27. Issues to Consider What survey mode(s) will be used to ask the questions? Is the question being repeated from another survey, and/or will answers be compared to previously collected data? Will respondents be willing and motivated to answer accurately? What type of information is the question asking for?

  28. Choosing Words and Forming Question • Make sure the question applies to the respondent • Make sure the question is technically accurate • Ask one question at a time • Use simple and familiar words • Use specific and concrete words to specify the concepts clearly • Use as few words as possible to pose the question • Use complete sentences with simple sentence structures • Make sure “yes” means yes and “no” means no • Be sure the question specifies the response task

  29. Visual Presentation of Survey Questions Use darker and/or larger print for the question and lighter and/or smaller print for answer choices and answer spaces Use spacing to create subgrouping within a question Visually standardize all answer spaces or response options Use visual design properties to emphasize elements that are important to the respondent and to deemphasize those that are not Make sure words and visual elements that make up the question send consistent messages Integrate special instructions into the question where they will be used rather than including them as freestanding entities Separate optional or occasionally needed instructions from the question stem by font or symbol variation Organize each question in a way that minimizes the need to reread portions in order to comprehend the response task Choose line spacing, font, and text size to ensure the legibility of the text

  30. From Questions to a Questionnaire

  31. General Premises The design of a questionnaire should consider how to motivate the recipient to respond It should also avoid measurement errors, ranging from order effects to item nonresponse

  32. Guidelines for Ordering Questions

  33. Ordering Questions • General guidelines • Group related questions that cover similar topics, and begin with questions likely to be salient to nearly all respondents • Choose the first question carefully • Place sensitive or potentially objectionable questions near the end • Ask questions about events in the order that they occurred • Avoid unintended order effects

  34. Guidelines for Creating a Common Visual Stimulus

  35. Visual Stimulus • General guidelines • Establish consistency in the visual presentation of questions (across pages and screens) and use alignment and vertical spacing to help respondents organize information on the page • Use color and contrast to help respondents recognize the components of the questions and the navigational path through the survey • Visually group related information in regions through the use of contrast and enclosure • Use visual elements and properties consistently across questions to emphasize or deemphasize certain types of information • Avoid visual clutter • Minimize the use of matrixes and their complexity

  36. Guidelines for Mail Questionnaires

  37. Mail Questionnaires • General guidelines • Determine whether keypunching or optical imaging and scanning will be used, and assess the limitations imposed on designing and processing questionnaires • Construct paper questionnaires in booklet formats, and choose physical dimensions based upon printing and mailing considerations • Decide question layout and how questions will be arranged on each page • Use symbols, contrast, size, proximity, and pagination effectively when designing branching instructions to help respondents correctly execute them • Create interesting and informative front and back pages that will have wide appeal to respondents • Avoid placing questions side-by-side on a page so that respondents are asked to answer two questions at once

  38. Guidelines for Web Questionnaires

  39. Web Questionnaires • General guidelines • Decide whether an electronic alternative is appropriate • Choose how the survey will be programmed and hosted, commensurate with needs, skills, and sponsorship • Take steps to ensure that questions display similarity across platforms, browsers, and user settings • Decide how many questions will be presented on each page and how questions will be presented • Develop a screen format that emphasizes the respondent rather than the sponsor

  40. Web Questionnaires • General guidelines • Use a consistent page layout across screens and visually emphasize questions information that respondents will need to complete the survey while deemphasizing information not essential to the task • Do not require responses unless absolutely necessary • Design survey-specific and item-specific error messages to help respondents troubleshoot • Evaluate carefully the use of interactive features, balancing improvements in measurement with the impact on respondent burden and the implications with mixed-mode surveys

  41. Web Questionnaires • General guidelines • Use audiovisual capabilities sparingly, and evaluate the differential effect they have on respondents • Allow respondents to stop the survey and finish completing it at another time • Whenever possible, collect paradata that provide feedback on how respondents interact with questionnaire • Test the survey using a variety of platforms, connection speeds, browsers, and user-controlled settings, and test the database to ensure that items are collected and coded accurately • Take screenshots of each page of the final questionnaire for testing and documentation

More Related