Research Interrogation Tools: Consumer Guide

1. CHAPTER 3 Three Claims, Four Validities: Interrogation Tools for Consumers of Research

2. Teaching Notes Chapter 3 is the guiding frame for the textbook. In my own class, we spend three 50-minute class periods on this chapter. • The first day we focus on identifying claims and naming variables that are in the claims. • On the next two days, students practice asking questions about the four big validities and applying the three causal criteria to different examples.

3. Learning Objectives A year from now, you should still be able to: 1. Differentiate the three types of claims: frequency, association, and causal. 2. Ask appropriate questions to help you interrogate each of the four big validities: construct validity, statistical validity, external validity, and internal validity. 3. Explain which validities are most relevant for each of the three types of claims. 4. State which kind of research study is required to support a causal claim.

4. Detailed Learning Objectives 1. Explain how quasi-experiments can be either independent-groups designs or within-groups designs. 2. Define the following quasi-experimental designs: nonequivalent control group design, interrupted time-series design, and nonequivalent groups interrupted time-series design. 3. Explain whether different quasi-experimental designs avoid the following threats to internal validity: selection, maturation, history, regression, attrition, testing, instrumentation, observer bias, experimental demand, and placebo effects. 4. Using both the design and results, analyze whether a quasi-experimental design allows you to rule out internal validity threats. 5. Explain the trade-offs in using a quasi-experimental design.

5. Check Your Understanding

6. Variables • What is the difference between a variable and its levels? What might be the levels of the variable “favorite color”? • Explain why some variables can only be measured, not manipulated. Can “gender” be a manipulated variable? Can “frequency of worrying” be a manipulated variable? • What is the difference between a conceptual variable and the operational definition of a variable? How might the conceptual variables “frequency of worrying,” “intelligence,” and “stress” be operationalized by a researcher?

7. Three Claims • How many variables are there in a frequency claim? An association claim? A causal claim? • Which part of speech in a claim can help you differentiate between association and causal claims? • How are causal claims special, compared with the other two claim types? • What are the three criteria causal claims must satisfy?

8. Interrogating the Three Claims Using the Four Big Validities • What question(s) would you use to interrogate a study’s construct validity? • In your own words, describe at least two things that statistical validity addresses. • Define validity, using the term generalize in your definition. • Why is a correlational study not able to support a causal claim? • Why don’t researchers usually aim to achieve all four of the big validities at once?

9. Reading Quizzes

10. Reading Quiz 1. Three necessary criteria for causal claims are A. covariance, temporal precedence, and internal validity. B. association, construct validity, and generalizability. C. operationalization, temporal precedence, and construct validity. 2. To support an association claim, a study would have to have A. one variable.B. two manipulated variables.C. two or more measured variables. 3. Which of the following headlines is a causal claim? • Holding a gun may make you think others are too. • Younger people can’t read emotions on wrinkled faces • Strange but true: Babies born in the autumn are more likely to live to 100 • Check the baby! Many new moms show signs of OCD 4. According to the textbook, the conclusion that family meals prevent eating disorders cannot be supported because A. the study does not establish temporal precedence or internal validity.B. there is zero association between the variables.C. they covary positively.D. they manipulate too many variables.

11. Clicker Questions

12. Clicker Question—1 Which of the following is an example of an association claim? • “41% of people surveyed reported that they were having a good day.” • “Guzzling a 20-ounce bottle of soda ramps up cellular aging.” • “People who sit within two tables of the bartender have three more alcoholic drinks, on average, than those who sit three tables away.” • “Viewing a recent conflict as it would look one year in the future led to increased feelings of forgiveness.”

13. Clicker Question—2 You read: “People who sit within two tables of the bartender have three more alcoholic drinks, on average, than those who sit three tables away.” What are the variables in this claim? • sitting within two tables of the bartender versus sitting three tables away • sitting within two tables of the bartender • how far you sit from the bartender and how many drinks you have • having three extra alcoholic drinks

14. Clicker Question—3 You read: “People who sit within two tables of the bartender have three more alcoholic drinks, on average, than those who sit three tables away.” What kind of study was this probably based upon? • a correlational study • an experimental study

15. Clicker Question—4 Which of the following phrases would indicate that a researcher is making a causal claim? a. curbs b. seems to decrease c. may enhance d. All of the above should be treated as a causal claim.

16. Clicker Question—5 There are four big validities, and not all of them are important for every claim. Which validity is appropriate to interrogate for every study? • construct • statistical • internal • external

17. Clicker Question—6 There are four big validities, and not all of them are important for every claim. Which two validities are most often in a trade-off (that is, researchers give up one to prioritize the other)? • construct and statistical • internal and external • internal and statistical • construct and external

18. In-class Activities

19. 1: Practice Identifying Claims and Variables—1 For the headlines on the next slide: • Indicate if the claim is frequency, association, or cause. • For each claim, state the variable (or variables) at the conceptual level. • For each variable: state at the operational level (speculate about how it might be operationalized). • What kind of study would be needed to support this kind of claim?

20. 1: Practice Identifying Claims and Variables—2 • In an unequal society, feeling richer than others makes you a jerk. • Instagram could predict depression, new research suggests. • Giving kids sips of beer turns them into teenage drunks. • High “normal” blood sugar may still harm brain. • Want a higher GPA? Go to a private college. • Safe drivers spot hazards. • Active sex life may lead to improved job satisfaction. • Report: 16% of teens have considered suicide. • Breastfeeding may boost children’s IQ. • Breastfeeding rates hit new high in United States. • Many students don’t know when news is fake. • Worry may make women’s brains work overtime. • Happier families put kids to bed early.

21. 1: Practice Identifying Claims and Variables—3

22. 2: Three Claims, Four Validities Matrix—1 Find examples of a frequency claim, an association claim, or a causal claim in your magazines. Identify the variables in each claim. Write down a question to ask of the study, or of the journalist, for each of the appropriate validities. If you think a validity is not relevant, explain why not.

23. 2: Three Claims, Four Validities Matrix—2

24. Working It Through

29. www.everydayresearchmethods.com Blog Examples

30. Classifying Claims About Playeverydayresearchmethods.com

31. Correlational and experimental designs: Safe Drivers Spot Hazardseverydayresearchmethods.com

32. Correlational and experimental designs: Flavor and behavioreverydayresearchmethods.com

33. Is this causal claim justified? Watching cat videos improves your mood, focuseverydayresearchmethods.com

34. Is this causal claim justified? A messy desk makes you creative?everydayresearchmethods.com