250 likes | 391 Views
Biases and Path Dependency in the Even Swaps Method. Raimo P. Hämäläinen Tuomas J. Lahtinen raimo.hamalainen@aalto.fi , tuomas.j.lahtinen@aalto.fi Systems Analysis Laboratory Aalto University, Finland sal.aalto.fi. Path dependency needs attention.
E N D
Biases and Path Dependency in the Even Swaps Method Raimo P. Hämäläinen Tuomas J. Lahtinen raimo.hamalainen@aalto.fi, tuomas.j.lahtinen@aalto.fi Systems Analysis Laboratory Aalto University, Finland sal.aalto.fi
Path dependency needs attention • Decision support processes often carried out in a sequence of steps • Behavioral biases along the path lead to dynamic effects Biases affect the path and the path affects which biases are likely to take place • Even Swaps method based on sequence of trade-offs • Interactive processes in multicriteria optimization also consist of sequential steps
Even Swaps Smart Choices (1999) Even Swaps is part of the PrOACT approach
Even Swaps elimination process Even swap: Alternative swapped to preferentially equivalent one that differs in two attributes • Carry out even swaps that make • Alternatives dominated There is another alternative, which is equal or better than this in every attribute, and better at least in one attribute • Attributes irrelevant Each alternative has the same value on this attribute • These can be eliminated • Asequence of swaps is carried out until the most preferred alternative remains
25 78 Practically dominated by Montana Dominated by Lombard Commute time removed as irrelevant (Slightly better in Monthly Cost, but equal or worse in all other attributes) Office selection problem (Hammond, Keeney, Raiffa 1999) An even swap
Different paths can be followed Paths consist of different sequences of trade-off judgments DM can experience the paths differently Each path should lead to the same choice - does this happen?
Phenomena related to paths • Anchoring to initial comparison tasks and judgments • Reference point changes along the path Loss aversion (Tversky and Kaheman 1991) • Elimination of alternatives and attributes changes the DM’s perception of the problem Context dependent preferences (Tversky and Simonson 1992) • Effects related to the measuring stick attribute Tempting to always use money as the measuring stick Scale compatibility (Tversky et al. 1988)
Loss aversion in even swaps Modified alternative becomes more attractive than the preceding one Contradicts preferential equality assumption of even swaps Loss aversion gives extra weight for losses Even swap: a reference change in one attribute is compensated by a change in another attribute • If reference change is a loss – compensatory gain overstated • If reference change is a gain – compensatory loss understated
Scale compatibility bias Attribute used as the measuring stick gets extra weight in trade-offs (Slovic 1990, Delquie 1993) Response: 10€ (10€ equals 30 min) 20 min (10€ equals 20 min) • Trade-off question: • How much should you pay to compensate for saving 30 minutes of commuting time? • How much should you save in commuting time to compensate for payment of 10 euros? The weight of commuting time is higher when it is used as the measuring stick This affects even swaps
Experiment • Students (83) from Aalto University used Even Swaps with the Smart-Swaps software Summer job selection task Apartment selection task • Subjects carried out both tasks on two or three paths Pricing path: Money used as the measuring stick Hours path: Working hours used as the measuring stick Smart-Swaps path (2 versions): Path suggested by the software Fixed reference path: All swaps carried out in a single alternative
Experiment leads to six comparisons Same subjects in all four comparisons Same subjects in both comparisons • Statistical analysis by McNemar’s test with binomial statistic • Outcomes of the same subject compared on pairs of paths in each decision task
Results • On every pair of paths over 50% of subjects ended up with different outcomes • Not only due to random inconsistencies: Path dependency exists • Results can be explained by scale compatibility and loss aversion • Here we present some of the results
Pricing path vs. Smart-Swaps path More subjects select a high salary job on the pricing path (one-way p: 0.002) More subjects select a low rent apartment on the pricing path (one-way p: 0.09) • Pricing path favors alternatives that are best in the money related attribute • This can be explained by scale compatibility – money is used as the measuring stick on pricing path
Swaps only in one alternative • Task: two jobs, B and D • When swaps are carried out in B, 50% of the subjects select it. • When swaps are carried out in D, 21% of the subjects select B. • Alternative is favored when all swaps are carried out in it • One-way p-value: 0.004
Explanations • Alternative is favored when all swaps are carried out in it • Loss aversion causes an alternative to become more attractive in each swap • Misunderstanding trade-offs (Keeney 2002): People can feel that they should benefit from the trade-off ”I am willing to trade-off” vs. ”I am indifferent between the two alternatives”
Reducing trade-off biases Experiment with 82 subjects, reference group given typical instructions Treatment group: Think of trade-off judgment from two reference points or Think of trade-off judgment with two measuring sticks Results: • Loss aversion bias reduced in treatment group • Scale compatibility bias not reduced Too much weight for the attribute that was first used as the measuring stick
What needs to be done? • Sensitivity analysis practically infeasible Focus on the process especially important
Support learning Good practise in preference modeling (Payne et al. 1999, Anderson and Clemen 2013) • Carry out the process on multiple paths to identify path dependency – Discuss with the DM • Present trade-off questions in multiple ways • Converging sequence of preference statements to decide the trade-off (Keeney 2002)
Design the process to cancel out biasesKleinmuntz (1990) • Reducing scale compatibility bias: Select measuring stick attribute in which alternatives are initially close to each other • Alternatives become more attractive in each swap: Carry out the same number of swaps in all the alternatives
Debiasing? Our Even Swaps experiment: Scale compatibility and loss aversion bias coefficients • Used in DA by Bleichrodt et al. 2001, Anderson and Hobbs 2002, Jacobi and Hobbs 2007 • Normative use can be problematic Credibility and transparency issues • Analyst can use the estimates of biases to support DM’s learning
Conclusions • Path dependency is a real phenomenon • DM constructs preferences during the DA process (Slovic 1995) • Challenge to design processes which alleviate path dependency • Any DA process consists of steps Do paths have an impact? • Path dependency needs attention also in interactive MCO methods • Learning is essential • Software can provide help
References Anderson, R. M., Clemen, R. 2013. Toward an Improved Methodology to Construct and Reconcile Decision Analytic Preference Judgments, Decision Analysis, 10(2), 121-134. Anderson, R. M., Hobbs, B. F. 2002. Using a Bayesian Approach to Quantify Scale Compatibility Bias. Management Science, 48(12), 1555-1568. Bleichrodt, H. J., Pinto, J. L., Wakker, P. 2001. Making descriptive use of prospect theory to improve the prescriptive use of expected utility. Management Science, 47(11), 1498-1514. Delquié, P. 1993. Inconsistent Trade-offs between Attributes: New Evidence in Preference Assessment Biases. Management Science, 39(11), 1382-1395. Hammond, J.S., Keeney, R.L., Raiffa, H., 1999. Smart Choices: A practical guide to making better decisions. Harvard Business School Press, Boston, MA. Jacobi, S. K., Hobbs, B. F. 2007. Quantifying and mitigating the splitting bias and other value tree-induced weighting biases, Decision Analysis, 4(4), 194-210. Keeney, R. 2002. Common mistakes in making value trade-offs. Operations research, 50, 935-945.
References Kleinmuntz, D. K. 1990. Decomposition and control of error in decision-analytical model. Insights in decision making: A tribute to Hillel J. Einhorn, 107-126. Payne, J. W., Bettman, J. R. Schkade, D. A. 1999. Measuring constructed preferences: Towards a building code, Journal of Risk and Uncertainty, 19(1-3), 243-270. Slovic, P. 1995. The construction of preference. American Psychologist, 50(5), 364. Slovic, P., Griffin, D., Tversky, A. 1990. Compatibility effects in judgment and choice. Insights in decision making: A tribute to Hillel J. Einhorn, 5-27. Tversky, A., Sattath, S., Slovic, P. 1988. Contingent Weighting in Judgment and Choice. Psychological Review, 94(3), 371-384. Tversky, A., Kahneman, D. 1991. Loss Aversion in Riskless Choice: A Reference-Dependent Model. Quarterly Journal of Economics, 106(4), 1039-1061. Tversky, A., Simonson, I. 1993. Context-dependent preferences. Management Science, 39(10), 1179-1189.