1 / 7

Bayes Rule

Learn about the Bayes' Rule, product rule, conditional independence, and naive Bayes algorithm for assessing diagnostic and causal probabilities in uncertain scenarios. Get insights into the applications and calculations involved in Bayesian statistics.

rosag
Download Presentation

Bayes Rule

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bayes Rule Which is shorthand for: CS 3243 - Uncertainty

  2. Bayes' Rule • Product rule P(ab) = P(a | b) P(b) = P(b | a) P(a)  Bayes' rule: P(a | b) = P(b | a) P(a) / P(b) • or in distribution form P(Y|X) = P(X|Y) P(Y) / P(X) = αP(X|Y) P(Y) • Useful for assessing diagnostic probability from causal probability: • P(Cause|Effect) = P(Effect|Cause) P(Cause) / P(Effect) • E.g., let M be meningitis, S be stiff neck: P(m|s) = P(s|m) P(m) / P(s) = 0.5 × 0.0002 / 0.05 = 0.0002 • Note: posterior probability of meningitis still very small! CS 3243 - Uncertainty

  3. Bayes' Rule and conditional independence P(Cavity | toothache  catch) = α · P(toothache  catch | Cavity) P(Cavity) = α · P(toothache | Cavity) P(catch | Cavity) P(Cavity) • This is an example of a naïve Bayes model: P(Cause,Effect1, … ,Effectn) = P(Cause) πiP(Effecti|Cause) • Total number of parameters is linear in n CS 3243 - Uncertainty

  4. Naïve Bayes Classifier • Calculate most probable function value Vmap = argmax P(vj| a1,a2, … , an) = argmax P(a1,a2, … , an| vj) P(vj) P(a1,a2, … , an) = argmax P(a1,a2, … , an| vj) P(vj) Naïve assumption: P(a1,a2, … , an) = P(a1)P(a2) … P(an) CS 3243 - Uncertainty

  5. Naïve Bayes Algorithm NaïveBayesLearn(examples)For each target value vj P’(vj)← estimate P(vj) For each attribute value ai of each attribute a P’(ai|vj) ←estimate P(ai|vj) ClassfyingNewInstance(x)vnb= argmax P’(vj)ΠP’(ai|vj) ajε x vjεV CS 3243 - Uncertainty

  6. An Example (due to MIT’s open coursework slides) R1(1,1) = 1/5: fraction of all positive examples that have feature 1 = 1 R1(0,1) = 4/5: fraction of all positive examples that have feature 1 = 0 R1(1,0) = 5/5: fraction of all negative examples that have feature 1 = 1 R1(0,0) = 0/5: fraction of all negative examples that have feature 1 = 0 Continue calculation of R2(1,0) … CS 3243 - Uncertainty

  7. An Example (due to MIT’s open coursework slides) (1,1) (0,1) (1,0) (0,0) R1 = 1/5, 4/5, 5/5, 0/5 R2 = 1/5, 4/5, 2/5, 3/5 R3 = 4/5, 1/5, 1/5, 4/5 R4 = 2/5, 3/5, 4/5, 1/5 New x = <0, 0, 1, 1> S(1) = R1(0,1)*R2(0,1)*R3(1,1)*R4(1,1) = .205 S(0) = R1(0,0)*R2(0,0)*R3(1,0)*R4(1,0) = 0 S(1) > S(0), so predict v = 1. CS 3243 - Uncertainty

More Related