On the Psychology of Prediction Kahneman & Tversky

by Gordon Peyton On the Psychology of PredictionKahneman & Tversky

On the Psychology of Prediction • Type of predictions discussed • Category prediction • Future type of employment • Numerical prediction • Future salary • Goal to discuss • Methodological issues • Sources of unjustified confidence • Fallacious Intuitions concerning regression effects

Category PredictionBase Rate, Similarity, & Likelihood • 1stcolumn • Baseline estimate mean of Grad student field of study distribution( in %) • 2nd column • Similarity Group • Received personality sketch of Tom W. (see bottom-left) • Estimate of Tom’s similarity of typical student in each of the fields • 3rd column • Prediction Group • Subjects received additional information (see below)

Results • Correlation between judged likelihood & similarity = .97 • Correlation between judged likelihood & base rate = .65 • More than 95% of subjects judged that Tom W. is more likely to study computer science than humanities or education • As Psychology graduate students the subjects were aware of the higher rate of students in latter fields

Issues • Lower confidence in projective test (27%) vs. HS senior interest and plan reports (53%) • Violation of Bayes’ rule • HS projective test not a good representation of Tom W. as a graduate student • Relevant information for statistical prediction • Prior or background information • i.e. base rates • Specific evidence concerning the individual • i.e. description of Tom W. • Expected accuracy of prediction • i.e. estimated probability of hits

Manipulation of expected Accuracy • Hypothesis: “Manipulation of expectation of accuracy does not affect the pattern of predictions.” • Same Design as Tom W. study • With high accuracy group (55%) & Low accuracy group (27%)

Results • High accuracy group mean estimates p = .7 vs. low accuracy group mean estimate p = .56 • (t = 3.72, p <.001 • No significant difference between high- & low-accuracy groups (p = .13 v p= .16) • (t = .42, df = 103) • Results were pooled the same way as in Tom W. study (see Table 2) • Again base rates were ignored

Quote • In statistical Theory one is allowed to ignore base rate only when on expects to be infallible (Kahneman, D. & Tversky, A., 1973).

Prior vs. Individuating Evidence O = Odds that Description belongs to an engineer rather than a lawyer Q = Odds that Description belongs to an lawyer rather than a engineer R = Likelihood ratio of particular description

Conclusions Results Follow up Description as informative as null Results where still 50% for low and high engineer groups Again prior odds were not accounted for • High engineer group • p = .55 • Low engineer group • P= .50 • Prior information on likelihood ignored (70/30; 30/70)

Summary • Kahneman & Tvorsky showed that even insignificant descriptions lead to bias in decision making by ignoring basic principles of statistics, namely to account for prior probability when confronted with specific information. • What does that mean for the HF engineer in regards to i.e. design?

Numerical Prediction • Normative prediction fundamental rule: • Variability of prediction should reflect predictive accuracy • Meaning the lower predictive accuracy the lower is variability • Representative hypothesis: Prediction and evaluation should coincide.

Prediction of Outcomes vs. Evaluation of inputs • Adjectives study: • Subjects were given five adjective describing character • Reports study: • Subjects were given detailed reports • Both groups had same descriptions • Evaluation Group • Estimate the students class standing • Prediction Group • Predict GPA at the end of the school year • Only discrepancy in adjectives study, namely predictions were consistently higher

Results • Adjectives Study • t = 1.25, df = 72, ns • Evaluation Group • SD = 25.7 • Prediction Group • SD = 24.0 • Reports Study • t = .75, df = 98, ns • Evaluation Group • SD = 22.2 • Prediction Group • SD = 21.4 • Differences in results were not significant • Knowledge of reduced accuracy in predictions did not cause their predictions be more regressive than their evaluations

Prediction vs. Translation • Subject supposed to predict the GPA of 10 hypothetical students • Prediction Criteria: • 1: Percentile GPA • Past performance • 2:Mental Concentration • Valid but unreliable • 3: Sense of Humor • Not a measure of academic ability

Summary of Fallacys • Representative Hypothesis was supported • In contrast to the normative model • Predictions were no more regressive than evaluations or judgments • Illusion of validity • People are prone to experience high confidence in highly fallible Judgments • Failure to acquire proper notion of regression

Topics Addressed from Prior Paper • Representativeness • Insensitivity to sample size • Insensitivity to predictability • Illusion of validity • Misconceptions of regression

Availability • Biases due to retrievability of instances • Familiarity and salience • Biases due to effectiveness of a search set • Abstracts perceived to occur more frequently than search words • Biases due to imaginability • Risk overestimation if dangers can be readily conceived • Risk underestimation if risks are hard to conceive • Illusionary Correlation • Overestimation of frequency of co-occurrence

Adjustment & Anchoring • Insufficient adjustment • Descending sequence estimate higher than ascending estimate ( i.e. 4 x 3 x 2 x 1 vs. 1 x 2 x3 x 4) • Biases in the evaluation of conjunctive and disjunctive events • Chain like structure of conjunctions leads to overestimation • Funnel like structure of disjunctions leads to underestimations • Anchoring in assessment of subjective probability distributions

Conclusion • Described three heuristics • Representativeness • Used for judgment of probability in and object of event • Availability • Used for judgment of frequency of a class or plausibility of a particular event • Adjustment from an anchor • Used in numerical prediction when a relevant values is present

Food for Thought • Poker question • On the river in a two handed game of Texas Hold’em Jim K. has roughly 2 million chips in his stack, the pot in the middle is 1.2 mil. His opponent John D. just bet 100k. Jim notices that John D. sits up straight, slightly bend forward, leaning towards him on the table, which indicates to him that John has a good hand due to the prior experience at this table. He has nothing in the moment, but knows that he has with his two hearts, including the queen, about a 20% chance to make a Queen high flush, which would be 98% of the time the winning hand with the current cards on the table. He decides to call. • Should he have called?

On the Psychology of Prediction Kahneman & Tversky