Choice-Based Conjoint Workshop October, 2010

1. Choice-Based Conjoint WorkshopOctober, 2010 With information provided by Sawtooth Software

2. Outline • Introduction to Conjoint Analysis • Formulating Experiments • Conjoint Methods -types • Use CBC software • Introduction to CBC analysis

3. Introduction to Conjoint Analysis • Identify Your Goal • Design your experiment – 5 stages • Interpreting part-worths and importance • A brief introduction to market simulations

4. Different Perspectives, Different Goals • Public wants all of the most desirable features of environmental assets at lowest possible cost • Providers want to maximize welfare by:1) minimizing costs of providing features 2) providing products/services that offer greater overall value than other alternatives

6. Demand or Preference Side of Equation • Focus first on demand/preference side of the equation • After figuring out what consumer wants, next assess whether it can be built/provided in a cost- effective manner

7. Products/Services are Composed of Features/Attributes • Olive Oil • Source, Price, Aroma, Size • Forest Harvesting Program Live trees after harvesting, Cost , Dead trees after harvesting, % of forest set aside from harvest • Plastic Bag Management • Cost/tax, % wildlife impact, durability, waste

8. Breaking the Problem Down • If we learn how consumer values the components of a product, we are in a better position to design those that improve profitability • If we learn how the public values the components of environmental goods and services, we are in a better position to design those goods and services to maximize societal welfare

9. How to Learn What Customers/Public Want? • One way: Ask Direct Questions about preference • What Brand do you prefer? • How much would you pay for it? • What color do you prefer • What size of container would you like? • Answers often trivial and unenlightening (e.g. respondents prefer low fees to high fees, medium size than large etc..)

10. How to Learn What Is Important? • One way: Ask Direct Questions about Importances • How important is it that you get the << >> that you want?

11. Stated Importances • Importance Ratings often have low discrimination:

12. Stated Importances • Answers often have low discrimination, with most answers falling in “very important” categories • If they were not important, we probably wouldn’t have included them in the research! • Answers sometimes useful for segmenting market, but still not very actionable • We still don’t exactly what product they want

13. What is Conjoint Analysis? • Research technique developed in early 70s • Measures how buyers value components of a product/service bundle • Dictionary definition-- “Conjoint: Joined together, combined.” • Marketer’s catch-phrase-- “Features CONsidered JOINTly”

14. How Does Conjoint Analysis Work? • We vary the product/service features (independent variables) to build many (usually 12 or more) product concepts • We ask respondents to rate/rank or choose among a subset of those product concepts (dependent variable) • Based on the respondents’ evaluations of the product concepts, we figure out how much unique value (utility) each of the features (attributes) added • (Regress dependent variable on independent variables; estimated betas equal to part worth utilities.)

15. Important Early Articles • Luce, Duncan and John Tukey (1964), “Simultaneous Conjoint Measurement: A New Type of Fundamental Measurement,” Journal of Mathematical Psychology, 1, 1-27 • Green, Paul and Vithala Rao (1971), “Conjoint Measurement for Quantifying Judgmental Data,” Journal of Marketing Research, 8 (Aug), 355-363 • Johnson, Richard (1974), “Trade-off Analysis of Consumer Values,” Journal of Marketing Research, 11 (May), 121-127 • Green, Paul and V. Srinivasan (1978), “Conjoint Analysis in Marketing: New Development with Implications for Research and Practice,” Journal of Marketing, 54 (Oct), 3-19 • Louviere, Jordan and George Woodworth (1983), “Design and Analysis of Simulated Consumer Choice or Allocation Experiments,” Journal of Marketing Research, 20 (Nov), 350-367

16. Traditional, Less-Effective Questions • How important is horsepower to you in a vehicle? • How important is fuel efficiency to you in a vehicle? • Which is more important to you, horsepower or fuel efficiency?

17. What’s So Good about Conjoint? • More realistic questions:Which product would you prefer . . .- 210 horsepower or - 140 horsepower - 17 MPG - 28 MPG • If choose left, you prefer Power. If you choose right, you prefer Fuel Economy • Rather than ask directly whether you prefer Power over Fuel Economy, we present realistic tradeoff scenarios and infer preferences from your product choices

18. What’s So Good about Conjoint? (cont.) • When respondents are forced to make difficult tradeoffs, we learn what they truly value • These values (utility scores) are associated with specific and actionable attribute levels relevant to the problem at hand

19. Building a Model

20. Building a Model • Inputs: • Attributes • Levels • Respondents • Prior Knowledge • External Data • Experimental Design • Conjoint Method • Outputs: • Utility Scores for each level • Importance Scores for each attribute • Ability to perform Simulations

21. Defining Attributes • Attributes are independent aspects of a product or a service (Brand, Price, Size, Color etc.) • How many attributes? • -Depends on research objectives • One rule of thumb was that no more than 6 0r 7 attributes is too much • May cause respondents to simplify, looking only at 2-3 most important • Attributes should be independent, mutually exclusive • Brand, quality and product life expectancy may all measure the same thing • Each attribute has varying degrees, or “levels” • Cost: $1, $2, $3 • Biodiversity Loss: 10, 50, 100 • Each level is assumed to be mutually exclusive of the others (a program has one and only one level of that attribute)

22. Rules for Formulating Attribute Levels • Attributes are assumed to be mutually exclusive • Attribute: Add-on features • Level 1= Sun roof • Level 2= GPS system • Level 3=DVD player • If you define levels in this way, you cannot determine the value of providing 2 or 3 of these features at the same time (or none of them)

23. Solutions • 8 level Attribute: • Features • None • Sunroof • GPS system • DVD Player • Sunroof, GPS • Sunroof, DVD • GPS, DVD • Sunroof, GPS, DVD • 3 Binary Attributes: • Sunroof: • None • Sunroof • GPS System • None • GPS • DVD Player • None • DVD Player

24. Rules for Formulating Attribute Levels • Levels should have concrete/unambiguous meaning • “very expensive” vs “ costs $575” • “weight: 5-7 kilos” vs “ weight 6 kilos” • -One description leaves meaning up to individual interpretation, while the other does not

25. Rules for Formulating Attribute Levels • Don’t include too many levels for any one attribute • The usual number is about 3-5 levels per attribute • Make sure levels from your attributes can combine freely with one another without resulting in utterly impossible combinations (very unlikely combinations OK)

26. Attribute Examples Brand A B C Cost $1 $2 $3 Color Red Black Blue …or graphics as well can be levels.

27. Suggestions for Determining Which Attributes & Levels to Include • Talk to all stakeholders • Focus Groups • Search of competitors websites, sales materials

28. Other Inputs into the Model

29. Conjoint Utilities (Part Worths) • Numeric values that reflect how desirable different features are:Feature UtilityVanilla 2.5Chocolate 1.825¢ 5.335¢ 3.250¢ 1.4 • The higher the utility, the better

30. Interpreting Conjoint Utilities • Interval scaled data (no ratio operations!) • You cannot compare one level from one attribute with one level from another attribute, since conjoint utilities are scaled to an arbitrary constant within each attribute (often zero-centered) • You CAN compare differences between two levels of one attribute versus two levels of another attribute (an addition operation)

31. Conjoint Importances • Ratio scaled data • Measure of how much influence each attribute has on people’s choices • Best minus worst level of each attribute, then percentaged:Vanilla - Chocolate (2.5 - 1.8) = 0.7 15.2%25¢ - 50¢ (5.3 - 1.4) = 3.9 84.8% ----- -------- Totals: 4.6 100.0% • Importances are directly affected by the range of levels you choose for each attribute

32. Market Simulations • Make alternative program/services scenarios and predict which program/services respondents would choose • Accumulate (aggregate) respondent predictions to make “Shares of Preference” (some refer to them as “market shares”)

33. Market Simulation Example • Predict market shares for 35¢ Vanilla cone vs. 25¢ Chocolate cone for Respondent #1:Vanilla (2.5) + 35¢ (3.2) = 5.7Chocolate (1.8) + 25¢ (5.3) = 7.1 • Respondent #1 “chooses” 25¢ Chocolate cone! • Repeat for rest of respondents. . .

34. Market Simulation Results • Predict responses for 500 respondents, and we might see “shares of preference” like: • 65% of respondents prefer the 25¢ Chocolate cone

35. So you want to do a conjoint…..

36. Step 1: Begin With the End in Mind • What is the objective of the research? • How much will public be willing to pay for biological control feature? • Will farmers switch and adopt a different varieties? • The better you define the root problem, the better your research will be!

37. Step 2: Plan Your Analysis • Identify how clients need to use data • Deliver analysis plan to clients as part of research proposal • Makes sure that objectives and deliverables are clear

38. Step 3: Define Attributes and Levels • How many attributes? Depends on research objectives • More than 6 attributes may cause respondents to simplify, looking only at 3-4 most important • Attributes should be independent, mutually exclusive • Brand, quality, product life expectancy may all measure the same thing

39. Rules for Formulating Attribute Levels • Levels are assumed to be mutually exclusiveAttribute: Add-on featureslevel 1: Manuallevel 2: Biological Controllevel 3: Chemical • If you define levels in this way, you cannot determine the value of providing two or three of these features at the same time (or none of them)

40. Rules for Formulating Attribute Levels • Don’t include too many levels for any one attribute • The usual number is about 3 to 5 levels per attribute • One temptation is to include many levels for price, so we can estimate people’s preferences for each • Better approach usually is to interpolate between fewer more precisely measured levels for “not asked about” prices • Cover the range of probable values

41. Rules for Formulating Attribute Levels • Make sure levels from your attributes can combine freely with one another without resulting in utterly impossible combinations (very unlikely combinations OK)

42. Medium Low High Representing Levels • Text • “High Performance Sports Car” • Pictures / Graphics • Sample boards • Allows respondents to touch or feel samples for tactile attributes (towel softness, greeting card paper quality, etc.) • “Null” Level – Has Stereo vs. __________

43. Step 3: Choose a conjoint methodStep 4: Identify Research Constraints • Sample issues • Sample Population >200 • Length of survey (how long can I keep their attention) • Fielding issues • Budget • Client sophistication

44. Example of a Pair of Soda Product Profile Scenarios

45. How to tell if you are in over your head… • You should be okay if… • Small number of attributes • Attributes freely combine with one another • Large sample, even after adjusting for subgroup analysis • Your client can describe the analysis, attributes in one paragraph or less, and you can then explain it to a six year old with little difficulty!

46. Section 2 • Intro to Choice-Based Conjoint (Discrete Choice Modeling)

47. Setting Up CBC Interview (Definitions)Concept 1 Concept 2 Concept 3 Concept 4How many concepts per task?How many tasks per survey? Task

48. How Many Concepts per Task? • Generally, 2 to 5 concepts are used • Attribute text length, graphical representation affect the decision

49. How Many Tasks per Survey? • Respondents are expensive to recruit. It makes sense to ask respondents multiple choice tasks. • Respondents take about 7 minutes on average to answer 20 tasks (~20 seconds per task) • CBC is very flexible in terms of how many tasks to include. Minimum is just one task! (but you’ll need huge sample size, and will face limitations in analysis) • Typical choice is 12 to 18 choice tasks

50. Test CBC Design • What is a Design? The sum total of information about the attribute levels being shown in the CBC tasks across all respondents. (The independent variable matrix) • If you use ANY prohibitions, or use few questionnaire versions, you MUST test your design • Failure to test the design can invalidate your study • CBC/Web automatically tests your design when it generates the design file--pay attention to the test!