Designing experiments for efficient WTP estimates

1. Designing experiments for efficient WTP estimates Ric Scarpa Prepared for the Choice Modelling Workshop 1st and 2nd of May Brisbane Powerhouse, New Farm Brisbane

2. The background • Profession is exploring designs that incorporate a-priori beliefs on taste intensities • As soon as one moves away from zero taste intensities (indeed an unlikely prior) large efficiency gains are achievable • In non-market valuation the focus is not on accurate taste intensities estimation, but on WTP estimation

3. Structure of the talk • Background on estimation of WTP variance • C-efficiency extended to choice modeling • Some challenges • Some criteria for efficient design • Results from an exploration in the MNL context with utility in the preference space • Future research

4. Variance of WTP from preference-space RUMs • In a typical RUM setting specification of utility is carried out in the preference space • U=kkxk+ $x$ • WTPk =-k / $, with all  estimated by ML • Under ML assumptions  ~ N(b, ACV) • Var(WTPk ) = Var (-k / $) • Received wisdom indicates that approximations to this can be achieved by the so called Krinsky and Robb procedure

5. Krinsky and Robb… an accident? • It is unclear as to why this has become the state of practice and substituted the delta method • Examining the literature one finds: • Krinsky, I. & Robb, A.L. On Approximating the Statistical Properties of Elasticities Review of Economics and Statistics, 1986, 68, 715-719 detection of a problem with delta method • Krinsky, I. & Robb, A.L. On Approximating the Statistical Properties of Elasticities: A Correction Review of Economics and Statistics, 1990, 72, 189-190  acknowledgement that they made a mistake in the first paper • Env. economics literature never took in or referred to the 1990 “Correction” and went on practicing K&R 86, which is basically a parametric bootstrap • However, the delta method works, as indicated by the correction and illustrated in other fields

6. Delta method and C-efficiency • DM offers a close form approximation via Taylor series that can be used to measure C-efficiency using priors on beta • C-efficiency is a form of M-efficiency (M=managerial) in that it is the variance of a managerial quantity that is of importance • It was proposed in the early 90’ by B. Kanninen in CVM design

7. Delta method • Based on Slutsky’s theorem and the properties of the ML estimator • Take any continuous function at least twice differentiable g(). Using the first two terms of a Taylor series approximation to expand it around the estimates one obtains: Where is the vector of K first derivatives, the gradient of g(.), and ' indicates the transpose.

8. Differences with CVM • Of course in SP CM we have k attributes and this complicates things as the dimension of WTP is k-1 • One can use an algorithm that finds the design allocation of attribute in alternatives and their levels that minimize the sum of variances of WTP • This search is unlikely to produce a balanced outcome because attributes with large WTP will have larger variances • For example, the minimum may be reached by achieving a very small variance for one attribute while leaving the variance for another attribute higher than what is desirable. • A potential solution is to use weighting in the construction of the C-error, assigning higher weights to attributes whose variance one wants to reduce most

9. Additional criteria in design • Min-max and max-min criteria can be useful in this context • maximizing the minimum t-value for the WTP: • or equivalently, that of minimizing the number of design replicates necessary to achieve the desired significance level for WTP:

10. Differences with D-efficiency • WTP-efficiency typically does not require maximization outcomes to be linked with continuous variables in the design (e.g. cost) to be placed at the extreme levels of the range • It is dependent on the covariances of the taste intensities, and they typically involve more trade-offs than D-efficient designs

11. Specificity of method in env. economics practice • Most env. econ. Studies include a SQ alternative • Efficiency measures differ in the presence or absence of a SQ constant • Recent studies have also started to pay attention to the effect of scale (Gumbel heteroskedasticity) • Finally, in WTP-based benefit transfer exercises from many different locations the pivot (reference points of respondent may vary) and this also has an impact on optimal WTP-design depending on whether the objective is WTP prediction or choice prediction

13. How does C-efficiency trade off other criteria in practice? • Results of a desk study done with J. Rose (ITLS Sydney)

15. Future research • Adaptive learning design (sequential updating, best with CAPI) • Individual-specific design (pivoting and individual scale factors matter) • Fast implementation designs (one balancing attribute on fixed frame e.g. cost as for Kanninen 2002) • Behaviourally valid designs (maximizing respondent efficiency)