Bayesian Models

1. Honors 207, Intro to Cognitive Science David Allbritton An introduction to Bayes' Theorem and Bayesian models of human cognition Bayesian Models

2. Bayes Theorem: An Introduction • What is Bayes' Theorem? • What does it mean? • An application: Calculating a probability • What are distributions? • Bayes' Theorem using distributions • An application to cognitive modelling: perception

3. What is Bayes' Theorem? • A theorem of probability theory • Concerns “conditional probabilities” • Example 1: Probability class is cancelled if it snows • P (A | B) • “Probability of A given B,” where A = “class is cancelled” and B = “it snows.” • Example 2: Probability that it snowed, if class was cancelled • P (B | A) • Bayes' Theorem tells how these two conditional probabilities are related

4. P (B | A) * P (A) P (A | B) = --------------------------- P (B) likelihood * prior posterior probability = --------------------------- normalizing constant What is Bayes Theorem? (cont.)

5. What does it mean? • The “prior” is the probability of A as estimated before you knew anything about B; the prior probability of A. • The “likelihood” is the new information you have that will change your estimate of the probability of A; it is the likelihood of B if A were true. • The “normalizing constant” just turns the resulting quantity into a probability (a number between 0 and 1); it is not that interesting to us.

6. An application: Calculating a probability "A cab was involved in a hit and run accident. There are two cab companies in town, the green (85% of the cabs in the city) and the blue (15%). A witness said that the cab in the accident was blue. Tests showed that the witness is 80% reliable in identifying cabs." Question: What is the probability that the cab in the accident was blue? A = The cab is blue. B = The witness says the cab is blue. P (A) = prior probability that the cab is blue P (B) = overall probability that the witness says the cab is blue P (B | A) = likelihood the witness says the cab is blue when it really is blue P (A | B) = posterior probability the cab is blue given that the witness says it is P(A) = .15 P(B|A) = .8 P(B) = P(B|A)*P(A) + P(B|~A)*P(~A) = .8*.15 + .2*.85 = .29 P (A | B) = [ P(B | A) * P(A) ] / P(B) = .8 * .15 / .29 = .41

7. What are distributions? • Demonstration: • http://condor.depaul.edu/~dallbrit/extra/psy241/chi-square-demonstration3.xls • Terms • Probability density function: area under the curve = 1 • Normal distribution, Gaussian distribution • Standard deviation • Uniform distribution

8. Bayes Theorem using distributions • Posterior, prior and likelihood are not single values, but functions over distributions • P(theta | phi) = P(phi | theta) * P(theta) / P(phi) • Posterior dist. = C * likelihood dist. * prior dist. • C is just a normalizing constant to make the posterior distribution sum to 1, and can be ignored here • Because the posterior is a distribution, need a decision rule to use it to choose a value to output

9. An application to cognitive modeling: perception • Task: guess the distal angle theta that produced the observed proximal angle phi • The viewing angle is unknown, so theta is unknown • p (theta | phi) = posterior; result of combining our prior knowledge and perceptual information • p (phi | theta) = likelihood; probabilities for various values of theta to produce the observed value of phi • p (theta) = prior; probabilities of various values of theta that were known before phi was observed • Gain function = rewards for correct guess, penalties for incorrect guesses • Decision rule = function of posterior and gain function

10. More information: • http://en.wikipedia.org/wiki/Bayes'_theorem • http://yudkowsky.net/bayes/bayes.html