An Open Problem: Characterizing Random Utility Models of Best-Worst Choice A. J. Marley

2. An Open Problem: Characterizing Random Utility Models of Best-Worst Choice • A. J. Marley University of Victoria, Victoria DIMACS Workshop on Polyhedral Combinatorics of Random Utility, May 2006

4. Research Supported by • The Australian Research Council (ARC) • The Natural Sciences and Engineering Research Council (Canada)

5. Summary • An illustrative empirical study • Definitions and theoretical examples • Random utility models for best-worst choice • Characterizing random utility models of best worst choice • (Further theoretical results on Thurstonian models of best-worst choice)

6. Public Issues • Preserving the environment • Quality medical care • Cost of living • Crime and public safety • National unity/Canada’s future • Level of taxation • Poverty in Canada • Drinking and driving • A quality food supply • The drug problem • Food safety

7. Question & Layout Used for First Best-Worst Task Identify the public issue that causes you the most concern and the public issue that causes you the least concern in each of the following sets of issues. Check only one issue for each of the Least and Most Columns. LEAST ISSUE MOST LEAST ISSUE MOST Preserving the environment Cost of living Level of Taxation A quality food supply Quality medical care Quality medical care The drug problem Poverty in Canada Question & Layout for Best-Worst Task (example shows 2 questions)

8. Issue Food safety 1 0 2 0 3 1 4 0 5 0 6 1 7 1 8 1 9 0 10 1 11 0 12 1 # times present 6 Preserving the environment 1 0 0 1 0 0 0 1 1 1 0 1 6 Cost of living 0 1 0 0 1 0 1 0 1 1 0 1 6 Level of taxation 1 0 1 0 0 0 1 0 1 1 1 0 6 National unity/Canada’s future 0 0 1 0 1 0 0 1 1 0 1 1 6 Crime & public safety 0 0 0 1 0 1 1 0 1 0 1 1 6 A quality food supply 0 1 1 1 0 0 0 0 0 1 1 1 6 Quality medical care 1 1 0 0 0 0 1 1 0 0 1 1 6 Drinking & driving 0 0 0 1 1 0 1 1 0 1 1 0 6 The drug problem 1 0 0 0 1 1 0 0 0 1 1 1 6 Poverty in Canada 0 1 0 0 0 1 0 1 1 1 1 0 6 # Issues in set 4 4 4 4 4 4 6 6 8 8 8 8 Fractional factorial design for choice sets (Best-Worst task)

9. Comparison of Most-Least totals Vs MNL estimates

10. Summary • An illustrative empirical study • Definitions and theoretical examples • Random utility models for best-worst choice • Characterizing random utility models of best worst choice • (Further theoretical results on Thurstonian models of best-worst choice)

18. Consistent Random Ranking Models And the Open Problem

20. Observations • A consistent random ranking model on (all the subsets of) a set T has consistent margins • Conversely, if a set of best-worst choice probabilities satisfy a random ranking model (next slide), then one can define a set of best and worst choice probabilities through the margins of the ranking probabilities • Thus, the major task is to characterize a random ranking model on best-worst choice probabilities

26. Summary • A major open problem is to characterize the class of (best, worst and) best-worst choice probabilities that satisfy a consistent random ranking model • We have a characterization of the maxdiff model for best-worst choice. This model is the most frequently used in applications • The maxdiff model (with its ‘natural’ extension to best and worst choices) is not compatible with a consistent random ranking model

27. Thank you for listening

35. Summary • A major open problem is to characterize the class of consistent random ranking models of best, worst and best-worst choice • We have a characterization of the maxdiff model for best-worst choice. This model is the most frequently used in applications • The maxdiff model (with its ‘natural’ extension to best and worst choices) is not compatible with a consistent random ranking model

36. Thank you for listening