1 / 26

Chapter 6

Chapter 6. Tools for Interrogating Frequency Claims: Surveys and Polls. Detailed Learning Objectives: Question Wording and Observations. 1. Give examples of how question wording can change the results of a survey or poll.

thu
Download Presentation

Chapter 6

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 6 Tools for Interrogating Frequency Claims: Surveys and Polls

  2. Detailed Learning Objectives: Question Wording and Observations 1. Give examples of how question wording can change the results of a survey or poll. 2. Describe the different ways questions can be worded: open-ended, forced- choice, and using rating scales. 3. Explain how to increase the construct validity of questions by wording them carefully and by avoiding leading questions, double-barreled questions, and double negatives. 4. Explain how question order can change the meaning (and validity) of a question. 5. Explain ways to increase the construct validity of questions by preventing respondent shortcuts (such as nay-saying), biases (such as trying to look good), or simple inability to report. 6. Describe how observational techniques for measurement are different from survey techniques. 7. Explain ways to improve the construct validity of observations by reducing observer bias and target reactivity.

  3. Detailed Learning Objectives: Sampling 8. Indicate the population that a given sample might represent. 9. Define three sampling problems that lead to biased samples. 10. Understand why a random sample is more likely to be a representative sample and why representative samples have external validity to a particular population. 11. Explain five techniques for random sampling: simple random, multistage, cluster, stratified random sampling, and oversampling. 12. Describe three techniques of nonrandom sampling: purposive, convenience, and snowball sampling. 13. Explain why representative samples may be especially important for many frequency claims. 14. Consider times when an unrepresentative sample may be appropriate for a frequency claim. 15. Explain why it is more important to ask how a sample was collected rather than how large the sample is.

  4. Reading Quiz • A response set can be in the form of • Open ended questions • Leading questions • Yay-saying answers • A Likert scale • A survey question that has you rate a response from agree to disagree is an example of • A Likert scale • A double barreled question • The semantic differential format • Which of the following is NOT a way to prevent or avoid observer bias • Masked study design • Inclusion of filler questions • Measuring traces that the behavior leaves behind • Make unobtrusive observations • Choose the following that are correctly matched (can be more than one) • Random sampling—external validity • Random assignment—internal validity • Random sampling—internal validity • Random assignment—external validity

  5. Question Wording

  6. Construct Validity of Surveys and Polls • Choosing question formats • Writing well-worded questions • Encouraging accurate responses

  7. Choosing Question Formats Open-ended questions Forced-choice format Likert scale Semantic differential format (example shown)

  8. Writing Well-Worded Questions • Leading questions • Double-barreled questions • Double negatives (vs. negatively worded items) • Question order How can you write items that avoid these problems?

  9. Encouraging Accurate Responses • Response sets (three types) • Yea-saying/ nay-saying • Fence sitting • Faking good or bad • Saying more than we can know • Measuring subjectivity vs. objectivity What are some solutions to these problems?

  10. Behavioral Observations

  11. Construct Validity of Behavioral Observations • Examples of frequency claims based on observational data • Are observations better than self-reports? • Good observations are both reliable and valid

  12. Some Examples of Frequency Claims Based on Observational Data

  13. What are some solutions to these problems? Are Observations Better than Self-Reports? Observers may see what they expect to see Observers can affect what they see Masked research designs Observer effects Unobtrusive observations Wait it out Measure results of a behavior Observe ethically Figure 6.6 Unobtrusive observations This one-way mirror allows researchers to unobtrusively record the behaviors of children in a preschool classroom.

  14. Good Observations Are Both Reliable and Valid

  15. Good Observations Are Both Reliable and Valid Figure 6.8 Table 1 from Campos and colleagues’ (2009) study of family interactions. The table depicts the degree of interrater reliability for each of the behaviors coded.

  16. Sampling

  17. Generalizing to Others: Sampling Participants • Populations and samples • How to get a representative sample • When does external validity matter? • When a representative sample is not the top priority • Are bigger samples better samples?

  18. www.everydayresearchmethods.com

  19. Key Questions a. Read the first three paragraphs carefully. What do you think of the sampling techniques used in this poll? To what population can we generalize the results from this poll? b. Your friend comments, “wow! 375,000 people responded to this survey! We can definitely generalize this to the American population.” What do you say?

  20. Populations and Samples Population is ? Sample is ?

  21. Representative Samples Probability sampling Simple random sampling Cluster sampling Multistage sampling Stratified random sampling Oversampling Systematic sampling

  22. Unrepresentative Sampling Self-selected Purposive Convenience

  23. Homework • What do you think of the construct validity of the polls your group brought in? • What do you think about the external validity of the polls your group brought in? Working with your own examples

  24. More Practice Imagine that you are planning to estimate the price of the average book at the UD college bookstore. The bookstore carries 13,000 titles, but you plan to sample only 200 books. You will select a sample of 200 books, record the price of each book, and use the average of the 200 books to estimate the average price of the 13,000 books in the bookstore. Based on this information, answer the following questions: a. What is the sample in this study, and what is the population? b. How might you collect a simple random sample of books? c. How might you collect a stratified random sample? d. How might you collect a convenience sample? e. How might you collect a systematic random sample? f. How might you collect a cluster sample of 200 books?

More Related