Scales and Measurement Chapters 11, 12

1. Scales and MeasurementChapters 11, 12

2. Constructs and Measurement • Construct Development • Identifying and defining what is to be measured • A construct is a hypothetical variable composed of different elements that are thought to be related (e.g., 5 questions tapping brand loyalty) • Measurement • Figuring out how to measure what you want to measure • Measure needs to be reliable and valid 

3. Different Types of Reliability • Internal Reliability • Extent to which items on a scale “hang together” or are correlated with one another • Cronbach’s alpha (covered in last class) • Split-half reliability (split measure into halves, correlate) • Test-Retest Reliability • Extent to which scores are stable over time • Have people complete questionnaire twice and correlate scores

4. Validity: Overview of Key Definitions • Validity (in general) • The extent to which conclusions drawn from a study are true • Internal Validity • When a researcher can clearly identify cause and effect relationships (i.e., there are no confounds) • External Validity • The extent to which what you find in your study can be generalized to your target population • Construct Validity • Extent to which your constructs of interest (e.g., sensation seeking) are accurately and completely identified (measured) • In other words, the extent to which you are actually measuring what you say you are measuring (your sensation seeking scale really does measure the true construct of sensation seeking)

5. Other Forms of Validity • Content Validity (Face Validity) • Extent to which a measure is appropriate according to experts in the domain of interest • Concurrent Validity (Convergent Validity) • Extent to which one measure of a construct overlaps with other similar measures of that construct • Discriminant Validity • Extent to which a measure of one construct does not overlap with measures of different constructs • Predictive Validity • Extent to which a measure of a construct can predict theoretically-relevant outcomes • Nomological Validity • How a construct fits within a broader set of related constructs

6. Key Idea • To develop reliable and valid measures which we can subsequently (and appropriately) use in statistical analyses, we must understand: • Properties of scales • How to design good questions that do not lead to biased or inconsistent responses

7. Measurement/Scaling Properties • Assignment • You can assign objects to categories • Order (Magnitude) • You can order objects in terms of having more or less of some quality • Distance (Equal Intervals) • The distance between adjacent points on the scale is identical • Origin (Absolute Zero Point) • Zero “means something” (absence of a given quality)

8. Types of Scales • Nominal Scale • Has Assignment Only (Political Party) • Ordinal • Has Assignment, Order (Rank Order of Finish in a Race) • Interval • Has Assignment, Order, Equal Intervals (Temperature) • Hybrid Ordinally-Interval Scale • Like an ordinal scale, but researcher “pretends” it is an interval scale (e.g., assumes 1 to 7 scale is an interval scale); commonly used in questionnaires • Ratio • Has Assignment, Order, Equal Intervals, Absolute Zero (Number of Cars)

9. What Type of Scale? • Number of Sweaters Purchased This Year? _______ • What is Your Ethnicity? • To what extent do you agree or disagree that Congress should have approved the $700 bailout? (1 = strongly disagree to 7 = strongly agree) • Please rank the following issues from most to least important (Iraq, Health Care, Economy, Environment) • What is your income? (5-10k; 11-15k; 16-20k; 20-25k; 25-30k)

10. On the Importance of Attitudes I believe both candidates bring strengths to the table I think McCain has a good grasp of national security issues I feel a strong connection with Obama I think Obama’s Plan for Health Care Reform is a good one I’m planning on voting for Obama I am inclined to vote for McCain I feel McCain would be a good leader

11. Three Components of Attitudes • Cognitive Component • How a person thinks about an attitude object (product, issue, candidate, idea) • Affective Component • How a person feels about an attitude object • Behavioral • A person’s behavioral predisposition to respond to an attitude object in a certain way

12. Three Components of Attitudes? I believe both candidates bring strengths to the table I think McCain has a good grasp of national security issues I feel a strong connection with Obama I think Obama’s Plan for Health Care Reform is a good one I’m planning on voting for Obama I am inclined to vote for McCain I feel McCain would be a good leader

13. Measuring Attitudes • While attitudes not perfect predictors of behavior, still very important • We need to understand how to measure attitudes accurately • Today, we’ll look at some creative ways to measure attitudes • And, some standard approaches taken in marketing research • As we do this, think about the types of questions we might want to ask in our restaurant concepts study

14. Creative Measures of Attitudes • Projective techniques (partially structured) • Physiological (GSR; Heart Rate; Blood Pressure) • Functional magnetic resonance imaging (fMRI) • Flushing toilets • Bogus pipeline • Lost letters and emails* • Implicit association test (IAT)*

15. You’ve Got Mail! Dear Peter Price, Thank you for applying for a Glassner Foundation Scholarship. As you know, these scholarships are highly competitive and are given only to a few select students. They cover tuition for four years at a state-funded university. There is also an additional $500 per year for academic supplies. Because of the large number of applicants this year we are late in sending out these notices. Because of the time sensitive nature of this material, we wanted to immediately inform you of the committee’s decision regarding your application. We are happy to inform you that you have been selected to receive a Glassner Scholarship. We ask that you respond within 48 hours. Due to the high number of applicants, we would like to extend the scholarship to others applicants if you do not accept the scholarship.

16. You’ve Got Mail! Dear Mohammed Hameed, Thank you for applying for a Glassner Foundation Scholarship. As you know, these scholarships are highly competitive and are given only to a few select students. They cover tuition for four years at a state-funded university. There is also an additional $500 per year for academic supplies. Because of the large number of applicants this year we are late in sending out these notices. Because of the time sensitive nature of this material, we wanted to immediately inform you of the committee’s decision regarding your application. We are happy to inform you that you have been selected to receive a Glassner Scholarship. We ask that you respond within 48 hours. Due to the high number of applicants, we would like to extend the scholarship to others applicants if you do not accept the scholarship.

17. You’ve Got Mail! Dear Mohammed Hameed, Thank you for applying for a Glassner Foundation Scholarship. As you know, these scholarships are highly competitive and are given only to a few select students. They cover tuition four four years at a state-funded university. There is also an additional $500 per year for academic supplies. Because of the large number of applicants this year we are late in sending out these notices. Because of the time sensitive nature of this material, we wanted to immediately inform you of the committee’s decision regarding your application. We regret to inform you that you have not been selected to receive a Glasser Scholarship. We ask that you respond within 48 hours. Due to the high number of applicants, we would like to extend the scholarship to others applicants if you do not accept the scholarship.

18. You’ve Got Mail!(Method) • Bushman & Bonacci (2004, JESP) • Pretested on Arab-American Prejudice • 2 weeks later, receive email intended for a different person • 2 IVs • Intended recipient had European-American (Peter Price) vs. Arab-American name (Mohammed Hameed) • Intended recipient won or didn’t win a scholarship (4 years support) • DV = willingness to return the email to the sender to indicate it was incorrectly delivered (must be done in 2 days)

19. You’ve Got Mail!(Results)

20. You’ve Got Mail!(Results)

21. You’ve Got Mail!(Results)

22. You’ve Got Mail!(Results) GOOD NEWS! YOU WON!! • P (returning email) went down as prejudice went up, especially when the intended recipient had an Arab-American name BAD NEWS…YOU DIDN’T WIN. • European-American: P (returning email) went down as prejudice went up • Arab-American: P (returning email) went up as prejudice went up. • People with Arab-American prejudice were actually more likely to return the bad news than the good news email when recipient was Arab American WHAT MIGHT EXPLAIN SUCH ARAB-AMERICAN PREJUDICE?

23. Terror Management Theory • Basic Assumption • Humans have developed “anxiety buffering mechanisms” against fear of death (cultural worldview, self-esteem, relationships) • Typical Method • Write about own death (mortality salience) or dental pain • Filler task • Judge others from ingroup vs. outgroup • Typical Result • We judge worldview threatening others more harshly under conditions of mortality salience • Support for Bush (vs. Kerry) goes up after MS prime

24. Implicit Association Task (IAT) • An implicit attitude is an attitude which people are not conscious of (or would not want to admit to) but which can be assessed via the associations people hold in their minds between evaluative words (good, bad) and attitude objects (Caucasian, Arab) • The IAT is a reaction time (RT) task in which subjects categorize words (positive, negative) and attitude objects (pictures of Caucasians, Arabs) on the left or right side of screen • To simplify, some trials involve categorizations that are consistent with an implicit attitude, others are inconsistent with the attitude • If there is an implicitly negative attitude toward Arabs, then reaction times should be slower to the inconsistent trials

25. The following set of trials is consistentwith an implicitly negativeattitude toward ArabsReaction times should berelatively fast if thereis an implicitly negative attitudetoward Arabs

26. Good Word or Caucasian Bad Word or Arab If you see a good word or a Caucasian, hit “e” (left) If you see a bad word or an Arab, hit “i” (right)

27. Good Word or Caucasian Bad Word or Arab

28. Good Word or Caucasian Bad Word or Arab Dirt

29. Good Word or Caucasian Bad Word or Arab

30. Good Word or Caucasian Bad Word or Arab Happy

31. Now Change Categorization(The following trials areinconsistent with implicit negative attitude toward Arabs)Reaction times should be slower on these trials if there is an implicitly negative attitude toward Arabs

32. Good Word or Arab Bad Word or Caucasian Categorization Switched

33. Good Word or Arab Bad Word or Caucasian

34. Good Word or Arab Bad Word or Caucasian Joy

35. Good Word or Arab Bad Word or Caucasian Death

36. Good Word or Arab Bad Word or Caucasian

37. What are we saying?! • Sensitive attitudes (e.g., racial prejudice) are sometimes hard to assess with a self-report scale (due to social desirability concerns) • The IAT measures associations (automatic attitudes) • Implicitly negative attitudes are not uncommon, but… • People can also override these automatic responses with controlled processes (so behavior is not discriminatory) • For more information, visit https://implicit.harvard.edu/implicit/ where you can take an IAT for the 2008 election (or other issues)

38. Verbal Rating Scales How satisfied were you with today’s meal? 1 2 3 4 5 6 7 Extremely Dissatisfied Somewhat Neither Somewhat Satisfied Extremely Dissatisfied Dissatisfied Satisfied Satisfied

39. Issues in Designing Verbal Rating Scales • Many measures taken by marketing researchers are verbal ratings • What do we need to consider when we develop verbal rating scales? • Number of categories • Forced vs. unforced scale • Balanced or unbalanced scale • Extent of verbal description • Should response categories be numbered or not • Comparative vs. noncomparative scale • Scale direction

40. Number of Response Categories? • To what extent are you satisfied with your current MP3 player? • Most researchers suggest between 5 and 7 categories; for example: 1 2 3 4 5 6 7 Extremely Dissatisfied Somewhat Neither Somewhat Satisfied Extremely Dissatisfied Dissatisfied Satisfied Satisfied • Too few does not give you enough information • Too many and it will be hard for people to discriminate between the options (e.g., a 100-point scale)

41. Forced vs. Unforced Scale? • How likely would you be to buy a car manufactured in Brazil? • Forced Scale (even number of options forces the respondent to lean one way or the other): 1 2 3 4 5 6 Very Unlikely Somewhat Somewhat Likely Very Unlikely Unlikely Likely Likely • Unforced scale gives people a neutral option: 1 2 3 4 5 6 7 Very Unlikely Somewhat Neither Somewhat Likely Very Unlikely Unlikely Likely Likely

42. Balanced vs. Unbalanced Scale? • How satisfied are you with your current hair stylist? • Balanced scale (same number of positive and negative options): 1 2 3 4 5 6 7 Extremely Dissatisfied Somewhat Neither Somewhat Satisfied Extremely Dissatisfied Dissatisfied Satisfied Satisfied • Unbalanced scale (here all options are positive): 1 2 3 4 5 6 7 SomewhatVery Satisfied Satisfied • Unbalanced scale can give biased results; unless distribution is naturally skewed to one side of the scale, should use balanced scale

43. Extent of Verbal Description? • The U.S. should invest in wind powered energy • Label endpoints or label all options? 1 2 3 4 5 6 7 Strongly Strongly Disagree Agree 1 2 3 4 5 6 7 Strongly Moderately Slightly Neither Agree Slightly Moderately Strongly Disagree DisagreeDisagree or Disagree Agree AgreeAgree • Labeling all options can aid in interpretation.

44. Should Categories be Numbered? • Toyota is an Environmentally Friendly Company Strongly Moderately Slightly Neither Agree Slightly Moderately Strongly Disagree DisagreeDisagree or Disagree Agree AgreeAgree 1 2 3 4 5 6 7 -3 -2 -1 0 1 2 3 • Numbers can help respondents understand scale • 1 to 7 scale quite common • But -3 to +3 can help interpretation of scale (disagree is negative, agree is positive); it may, however, overemphasize negativity • Judgment call; pretesting both scales could help identify problems Should we have numbers here?

45. Comparative vs. Noncomparative? • Noncomparative question • How would you evaluate Mintifresh toothpaste? • Comparative question • Compared to your current brand, how would you evaluate Mintifresh toothpaste? • Comparative questions establish the referent and can be useful if you need to know how your product compares to a specific competitor or the customer’s current brand • Noncomparative have the advantage of allowing the respondent to create their own referent, which can potentially improve accuracy

46. Direction of Scale? • Typical direction (lower values, negative connotation on left): Strongly Moderately Slightly Neither Agree Slightly Moderately Strongly Disagree DisagreeDisagree or Disagree Agree AgreeAgree 1 2 3 4 5 6 7 • Some scales are not valenced, so must be careful about positioning. Here we see a semantic differential scale, with amusing positioning: Unpleasant -2 -1 0 1 2 Pleasant Flimsy -2 -1 0 1 2 Sturdy Male -2 -1 0 1 2 Female • Hmmm…this arrangement suggests that males are to be evaluated negatively; must be careful in designing scales so as not to bias results

47. Other Types of Scales Rank order the restaurants above where 1 = most preferred, 3 = least preferred

48. Rank-Order andPaired Comparison Techniques • Rank-order technique • Advantages: easy to understand, typically what we do in real-life (vs. ratings) • Disadvantage: it may be that a person dislikes all of the options, so ranking not that informative • Paired comparisons • Take n products, compare each one to every other (in each pair, pick the one you prefer) • Gives you direct comparisons, but # paired comparisons can be very large as number of products to be compared increases

49. Semantic Differential Scale • Please provide your impression of Politician X: Dumb _____ _____ _____ _____ __X__ Smart Cold _____ _____ _____ __X__ _____ Warm Boring _____ _____ _____ _____ __X__ Funny Mean _____ _____ _____ __X__ _____ Nice • Please provide your impression of Politician Y: Dumb _____ __X__ _____ _____ _____ Smart Cold __X__ _____ _____ _____ _____ Warm Boring _____ _____ __X__ _____ ____ Funny Mean _____ __X__ _____ _____ _____ Nice • Profile Analysis of Politician X vs. Y: Dumb _____ _____ _____ _____ _____ Smart Cold _____ _____ _____ _____ _____ Warm Boring _____ _____ _____ _____ _____ Funny Mean _____ _____ _____ _____ _____ Nice X Y

50. Stapel Scale • Please provide your impression of Shoe Store X. (Use negative numbers if you feel it is inaccurate; positive numbers if you feel it is accurate): -5 -4 -3 -2 -1 Fast Service 1 2 3 4 5 -5 -4 -3 -2 -1 Friendly 1 2 3 4 5 -5 -4 -3 -2 -1 Knowledgeable 1 2 3 4 5 • Can draw comparative profile analysis (e.g., of various shoe stores) as we did with semantic differential scale (previous slide)