Defining & Measuring Variables

1. Defining & Measuring Variables Research Basics

2. Overview • Defining Variables • Types of Measures • Scales (i.e., levels) of Measurement • Reliability • Validity

3. Defining Variables

4. Variables & Definitions • Variable • Something that varies and can be measured • Hypothetical constructs • Attributes/mechanisms that help explain/predict behavior • May not be directly observed • But may be associated with something that can be measured

5. Variables & Definitions • Conceptual Definitions: What you’d read in a textbook • Aggression???? • “A response intended to inflict pain or discomfort” (Averill, 1982, p. 30) • “Any form of behavior directed toward the goal of harming or injuring another living being who is motivated to avoid such treatment” (R.A. Baron, 1977, p. 7) • “The use of threat or force, in territory not clearly one’s own, without clear evidence that a majority of the emotionally involved people in that territory want such intervention” (R. K. White, 1984, p. 14)

6. Defining Variables • Operational definition “Operationalization” • Measurement or measurement procedure • How would you measure aggression?

7. Defining Variables • “Exercise leads to weight loss” • Exercise • Thumb twiddling 1 minute per week • 10 pushups each day • ½ hour aerobic activity, 5 days a week where heart rate is greater than 140 bpm • Weight loss • Pounds • Waist Size • Body fat content • Clothes size

8. Multiple Outcomes • Converging evidence is more convincing • Exercise leads to lower scale-weight • Exercise leads to lower scale-weight, smaller waist size, and reduced body fat • More measured outcomes increases the chances we’ve tapped into all relevant parts of our construct

9. Let’s practice • Creativity • Forgiveness • Love • Extraversion

10. “Southern Culture of Honor” • “Affronts are met w/ violent retribution” • Hypotheses • Perceive insults as damaging • Emotional/physiologically upset • Prepared to respond aggressively after insult (cognitive, physiologically, & behaviorally) (Cohen, Nisbett, Bowdle, & Schwarz, 1996)

11. “Southern Culture of Honor” • Hostility assessed • 2 observers measured emotional reactions • Face rating task • Guess emotion being expressed in pictures • Word completion task • __ight, gu__ (fight or gun) OR (light or gum) • Scenario completion task Cohen, Nisbett, Bowdle, & Schwarz, 1996)

12. Scenario Completion Task • It had only been about twenty minutes since they had arrived at the party when Jill pulled Steve aside, obviously bothered about something. • “What’s wrong?” asked Steve. • “It’s Larry, I mean, he knows that you and I are engaged, but he’s already made two passes at me tonight.” • Jill walked back into the crowd, and Steve decided to keep his eye on Larry. Sure enough, within five minutes Larry was reaching over and trying to kiss Jill. Cohen, Nisbett, Bowdle, & Schwarz, 1996)

13. Example: “Southern Culture of Honor” • Hostility assessed • Before & after cortisol levels • Hormone associated w/ stress, anxiety, & arousal • Before & after testosterone levels • Hormone associated w/ aggression & dominance • Desire to demonstrate toughness • Public opportunity to take shock in “electric shock stress test” • The “chicken game” • Firmness of handshake Cohen, Nisbett, Bowdle, & Schwarz, 1996)

14. What’s the point? • Many ways to operationalizevars • Didn’t use them all in one study though • Reliable measures? • Valid measures?

15. An operational definition should be… • Specific • Reliable • Valid

16. Approaches • Quantitative • Expressed numerically • Statistical tests available • Qualitative • In-depth • Grounded theory – use information to develop interpretation or theory • Identify ideas, then write theoretical narrative (broad themes) • Difficulties w/ analyses

17. How to measure? • Which measure provides the best indication of the underlying construct? • Self-reports • Behavioral, Cognitive, Affective • Tests • Personality, Achievement, Aptitude • Behavioral measures • Concern about reactivity • Physical measures • Each has advantages & disadvantages

18. Selecting Measures Scales of Measurement • Nominal Scales • Classification • Qualitative differences • Ordinal Scales • Classify & rank

19. Selecting Measures Scales of Measurement • Interval Scales • Classify, rank, & categories the same size • Arbitrary zero • Ratio Scales • Classify, rank, categories the same size, & anchored by zero point

20. Reliability

21. Choosing Your Measures • Reliability • Stability or consistency of measure Observed score = True Score + Measurement Error

22. Types of Error • 1. Observer Error • 2. Participant Changes • Stable attributes (E.g., suspicion, intelligence) • Transient states • Something about person that changes regularly • 3. Environmental Changes • 4. Test characteristics

23. Assessing Reliability • Test-retest reliability • Participants have changed • Respond in similar way • Practice Effects • Parallel-forms (or alternative-forms) reliability • Tests at time 1 and time 2: are they equivalent?

24. Assessing Reliability • Internal consistency • Can be measured with correlation coefficient • Split-half reliability - Split items on tests into equivalent halves

25. Assessing Reliability Judgments/Ratings by Multiple Observers • Interrater (interjudge) reliability • Extent of agreement between raters • Can be measured w/ correlation or percent agreement

26. Validity

27. Choosing Your Measures • Validity • Is the measure measuring what it is meant to? • Face Validity • Does the measure appear to measure what it is intended to measure? 2 + 2 = ________ 6 + 10 = ________ 7 + 3 = ________

28. What do you think this test measures? 1 = Strongly disagree 2 = Disagree 3 = Not sure 4 = Agree 5 = Strongly agree • I enjoy taking risks. • I prefer things that involve change and variety even if they make me feel uneasy at first. • I would enjoy fast driving. • I would do almost anything on a dare.

29. Types of Validity • Face Validity • When might we NOT want a face valid measure? • If we think it might influence participants’ answers • Content Validity • How well do the items represent the entire universe of items?

30. Types of Validity • Criterion validity • How well is the measure related to other variables that reflect the same construct? • Concurrent validity • Do scores correlate with other relevant current outcome/behavior? • IQ test should correlate with current school performance • Occupational aptitude test should correlate with current work performance • Predictive validity • Does it correlate with future outcomes? • According to theory

31. Types of Validity • Discriminant (Divergent) Validity • Looking for evidence that the measure is distinct from other behaviors/traits (i.e., not correlated)

32. Types of Validity • Face • Content • Criterion • Concurrent • Predictive • Discriminant (Divergent)

33. Relationship Between Reliability & Validity Can be reliable, without being valid. But can’t be valid, without being reliable.

34. Re-cap • Defining Variables • Types of Measures • Scales (i.e., levels) of Measurement • Reliability • Validity

35. Mini-Review What is the difference between reliability and validity? Provide an example on a measure that would have high content validity. Imagine I wanted to measure helping behavior. How might I operationalize this variable?