Practical Design for Discrete Choice Experiments

1. Practical Design for Discrete Choice Experiments Bradley Jones, SAS Institute August 13, 2008

2. Discrete Choice Experiment Setup • Respondents indicate the alternative they prefer most in each choice set • Alternatives are called profiles • Each profile is a combination of attribute levels • Choice sets typically consist of two, three or four profiles DEMA 2008, August 2008, Cambridge 2

3. Example: Marketing a new laptop computer Attributes Levels Hard Drive 40 GB 80 GB Speed 1.5 GHz 2.0 GHz Battery Life 4 hours 6 hours Price $1,000 $1,200 $1,500 DEMA 2008, August 2008, Cambridge 3

4. Sample Choice Set Check the box for the laptop you prefer. Hard Disk Speed Battery Price 40Gig 1.5GHz 6hours $1,000 40Gig 2.0GHz 4hours $1,500 Profile 1 Profile 2 DEMA 2008, August 2008, Cambridge 4

5. Statistical model • multinomial logit model • based on the random utilities model • where xjs represents the attribute levels and βis the set of parameter values • probability of choosing alternative j in choice set s DEMA 2008, August 2008, Cambridge 5

6. Design optimality criterion • Dcriterion-minimizethe determinant of the variance matrix of the estimators: • Equivalently– maximize the determinant of the information matrix, M. DEMA 2008, August 2008, Cambridge 6

7. Dependence on the unknown parameter, β Bayesian optimal designs: • construct a prior distribution for the parameters • find design thatperforms best on average • Sándor & Wedel (2001, 2002, 2005) DEMA 2008, August 2008, Cambridge 7

8. Design for Nonlinear Models To design an informative experiment ….. You need to know something about the response function ….. And about the parameter values. DEMA 2008, August 2008, Cambridge 8

9. Bayesian D-Optimal Design Bayesian ideas are natural to cope with the fact that the information matrix, M, depends on b. Chaloner and Larntz (1986) developed a Bayesian D-Optimality criterion: Φ(d) = ∫ log det[M(b;d)] p(b) db DEMA 2008, August 2008, Cambridge 9

10. Computing Bayesian D-Optimal Designs A major impediment to Bayesian D-optimal design has been COMPUTATIONAL. The integral over bcan be VERY SLOW. It must be computed MANY TIMES in the course of finding an optimal design. DEMA 2008, August 2008, Cambridge 10

11. Bayesian Computations Gotwalt, Jones and Steinberg (2007) use a quadraturemethod, due to Mysovskikh. This method is guaranteed to exactly integrate all polynomials up to 5th degree and all odd-degree monomials. With p parameters, it requires just O(p2) function evaluations. DEMA 2008, August 2008, Cambridge 11

12. Mysovskikhquadrature Assume a normal prior with independence. • Center the integral about the prior mean. • Scale each variable by its standard deviation. • Integrate over distance from the prior mean and, at each distance, over a spherical shell. DEMA 2008, August 2008, Cambridge 12

13. Mysovskikhquadrature continued… Radial integral: Generalized Gauss-LaGuerrequadrature, with an extra point at the origin. Spherical integrals: The Mysovskikhquadraturescheme. DEMA 2008, August 2008, Cambridge 13

14. Spherical integral A simplex, its edge midpoints on the sphere, and the inverses of all of these points. Simplex point weights: p(7-p)/2(p+1)2(p+2). Mid-point weights: 2(p-1)2/p(p+1)2(p+2). DEMA 2008, August 2008, Cambridge 14

15. Quadrature points in two dimensions Each point is both a simplex point and a mid-point. All weights equal1/6. Simplex point Mid-point DEMA 2008, August 2008, Cambridge 15

16. Study Description 16 Respondents – 8 developers 8 sales & marketing 9 Male 7 Female 2 Surveys with 6 choice sets in each Respondents were assigned randomly to surveys blocked by job function DEMA 2008, August 2008, Cambridge 16

17. Software Demonstration DEMA 2008, August 2008, Cambridge 17

18. Conclusions Discrete Choice Conjoint Experiments require design methods for nonlinear models. D-Optimal Bayesian designs reduce the dependence of the design on the unknown parameters. New quadrature methods make computation of these designs much faster. Commercial software makes carrying out such studies simple and efficient. DEMA 2008, August 2008, Cambridge 18

19. References Atkinson, A. C. and Donev, A. N. (1992). Optimum Experimental Designs, Oxford U.K.: Clarendon Press. Cassity C.R., (1965) “Abscissas, Coefficients, and Error Term for the Generalized Gauss-LaguerreQuadrature Formula Using the Zero Ordinate,” Mathematics ofComputation, 19, 287-296. Chaloner, K. and Verdinelli, I. (1995). Bayesian experimental design: a re-view, Statistical Science 10: 273-304. Grossmann, H., Holling, H. and Schwabe, R. (2002). Advances in optimum experimental design for conjoint analysis and discrete choice models, in Advances in Econometrics, Econometric Models in Marketing, Vol. 16, Franses, P. H. and Montgomery, A. L., eds. Amsterdam: JAI Press, 93-117. Gotwalt, C., Jones, B. and Steinberg, D. (2009) Fast Computation of Designs Robust to Parameter Uncertainty for Nonlinear Settings accepted at Technometrics. Huber, J. and Zwerina, K. (1996). The importance of utility balance in efficient choice designs, Journal of Marketing Research 33: 307-317. McFadden, D. (1974). Conditional logit analysis of qualitative choice behavior, in Frontiers in Econometrics, Zarembka, P., ed. New York: Academic Press, 105-142. Meyer, R. K. and Nachtsheim, C. J. (1995). The coordinate-exchange algorithm for constructing exact optimal experimental designs, Technometrics37: 60-69. Monahan, J. and Genz, A. (1997). Spherical-radial integration rules for Bayesian computation, Journal of the American Statistical Association 92: 664-674. Sandor, Z. and Wedel, M. (2001). Designing conjoint choice experiments using managers' prior beliefs, Journal of Marketing Research 38: 430-444. DEMA 2008, August 2008, Cambridge 19