Tales from the Lab

1. Tales from the Lab Reports from the field --psychology experiments relevant to Usability Professionals Paul Sas March 16, 2004

2. Unlike my CHI2003 Tutorial User bias & judgment: The Subjective side of decisionmaking Part I. Studies on judgment and decision making 90 minute review of findings on JDM + 4.5 Hours more, trying to squeeze in a semester of content Paul Sas

3. Goals of this talk • NOT an Ed Tufte dot of infinite data density • NOT a demonstration of what happens with Choice Overload Rather: • Tales. Many psychological principles best shown through stories • Suggestive applications for Designers and User Testing Experts Paul Sas

4. Heuristics, defined • Not an arcane term to usability experts • Quick, ready rules of thumb • In JDM, experiments focus on the biases that fall out from these rules of thumb in many well-defined cases Paul Sas

5. One demonstration of a Heuristic Paul Sas

6. E.g.: Representativeness Heuristic Q: Which delivery room will have more days during which the number of boys delivered will outnumber the girls? Hospital A -- 15 deliveries/day Hospital B -- 45 deliveries/day Or is it C: Just as likely to happen at A as at B Paul Sas

7. Law of small numbers (Tversky and Kahneman) Representative Heuristic: If X resembles Y, people judge the likelihood as high that X will generate/cause Y. • A delivery room is a delivery room (for more than 60% of the people polled) • Undue confidence in early trends • Unreasonably expect replication. • Underestimate confidence interval width • Rarely attribute deviation to sampling Paul Sas

8. Relevance to Designers/Testers • Remember that a user test is a very useful heuristic • Emphasize to your audience that it necessarily has biases • Don't bet your job on the patterns of preferences observed Paul Sas

9. Less is More (Hsee) • Music Dictionary A • 10,000 entries • condition is like new • Music dictionary B • 20,000 entries • cover is torn • When evaluated separately: A $24 B $20 • When evaluated jointly: A $19 B $27 Paul Sas

10. Less is More, again • China set A • 24 pieces of plateware • Full set for 8 • China set B • 32 pieces of plateware intact out of 40 • Includes all 24 of A, plus 8 (with 8 more chipped) • When evaluated separately: A $24 B $20 • When evaluated jointly: A $19 B $27 Paul Sas

11. Separate v Joint Evaluation (JE) • SE: analogous to purchasing one item from an auction, w/o comparative shopping • JE: Occurs when choosing from a range of items • Ease of evaluability drives decision • Lay rationalism: a tendency to overweight attributes that appear rationalistic, such quantity and economic value, and downplay attributes that appear subjective Paul Sas

12. Primed to feel or Primed to Calculate(Rottenstreich) • Box of Madonna CDs • If evaluated from an affective/emotional valence, then quantity is irrelevant • If viewed as tokens with exchange value, then more equals more • Primed to feel: No more price for box of 10 CDs than for 5 CDs • Primed to calculate: Box of 10 CDs receives higher valuation than 5 Paul Sas

13. What does this tell Designers? • It is possible to move people toward an emotionally valent sphere where there is no longer cost-competition • Steve Jobs at Apple is superb at transcending the commodity space of price Paul Sas

14. Anchoring (Ariely) • People's valuation is incredibly sensitive to random sources of information • Classic: Roulette wheel's impact on Africa • Auction to earn $ for stress: Digits of SSN were starting point (2 or 3) • Low anchored SSN: earned $.08 • High anchored SSN: earned $.59 Paul Sas

15. Can Designers Use Anchoring? • In a competitive market, it's not possible to whisper a price(Yet realtors get 6%, and waiters 15) • Newly developed niches can be creatively partitioned. • Shipping fees, handling fees, etc. Paul Sas

16. Paradoxes of Hedonomics • Experienced vs. Remembered Utility • Certain dimensions are easy to evaluate: • Most intense moment • Last moment • Other dimensions are impossible to guess: • Average height on a roller coaster ride • Distance covered on a roller coaster Paul Sas

17. Peak and End Rule (Kahneman) • Experienced vs. Remembered Utility • "Our mind does not make movies; it takes snapshots" • Rather than guess the total amount of suffering, people recall the worst instant, and the last instant. • If you increase the amount of suffering, but arrange for the last minutes to be less intense, people report the longer period as less painful Paul Sas

18. Peak and End Rule -- Pictograph Paul Sas

19. Peak and End Rule for User Design • Jared Spool's "Truth About Download Time" • Nielsen reports the most popular sites took an average of 8 seconds to download, whereas the pages of the less popular sites took an average of 19 seconds. "He therefore concludes that users will be annoyed … by pages that take any longer than about 10 seconds to load." • no correlation between download times and perceived speeds reported by our users. Amazon.com, rated as one of the fastest sites by users, was really the slowest (average: 36 seconds). • a strong correlation between perceived download time and whether users successfully completed their tasks on a site Paul Sas

20. Uncertainty Effects (Shafir) • Buy the trip to Hawaii • If you pass, you'll take the trip to celebrate • If you fail, you'll go on the same trip to console yourself • Choose to buy 32%; Choose to not buy, 7% • Pay $5 fee to wait, 61% • Not knowing why I'm going feels irresponsible. Paul Sas

21. More choice causes less purchasing (Iyengar) • Every other hour, a set of 24 jams/jellies to sample. On odd hours, only 6 jams available. • Choice of sampling any of 24 jams: 3% redeemed coupon. • Choice of sampling any of 6 jams: 30% redeemed coupon. • Recdently validated in 401K selection behavior Paul Sas

22. Solutions to Choice Overload • "Oracles" (a magus, not a 'wizard') • Investors preferred the portfolio selected by a professional investment manager to the portfolio they selected themselves, when comparing the implied distribution of outcomes [Benartzi and Thaler (2001)] Paul Sas

23. Eliciting Goals that Matter • Can't simply affirm past efficacy • Distraction fails as well Paul Sas

24. Delmore Effect • Related domain, outside the single-most important, will improve effectiveness Paul Sas

25. Creating Personas to Design For • Linda is 31 years old, single, outspoken and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. • Linda is a teacher in elementary school. • Linda works in a bookstore and takes Yoga classes. • Linda is active in the feminist movement. • Linda is a psychiatric social worker. • Linda is a member of the League of Women Voters. • Linda is a bank teller. • Linda is an insurance salesperson. • Linda is a bank teller and is active in the feminist movement. Paul Sas

26. Conjunction Errors • Heuristics lead to violations • Linda = feminist bank teller • 85% Respondents displayed the predicted order (H>F for Linda) • Explanation of P(F and H)>P(F) • Adding features increases representativeness – but cannot increase probability Paul Sas

27. Descriptive Choice: Prospect Theory Paul Sas

28. Sources for followup • Less is More (Chris Hsee)http://gsbwww.uchicago.edu/fac/christopher.hsee/vita/ • Primed for Feeling (Yuval Rottenstreich)http://gsb.uchicago.edu/fac/yuval.rottenstreich/ • Anchoring (Dan Ariely)http://web.mit.edu/ariely/www/papers.html • Peak and End Rule (Daniel Kahneman)http://www.nobel.se/economics/laureates/2002/kahneman-lecture.html • Choice Overload (Sheena Iyengar)http://www.columbia.edu/~ss957/articles.html • Delmore Effect (Paul [Whitmore] Sas)http://www-psych.stanford.edu/~wit/PhDraft.pdf Paul Sas