450 likes | 474 Views
Learn the basics of Bayesian networks, why we use them, how to build them by hand and from data, their applications, and their relationship to other models.
E N D
Bayesian Networks Lecture 1:Basics and Knowledge-Based Construction Lecture 1 is based on David Heckerman’s Tutorial slides. (Microsoft Research) Requirements: 50% home works; 50% Exam or a project
What I hope you will get out ofthis course... • What are Bayesian networks? • Why do we use them? • How do we build them by hand? • How do we build them from data? • What are some applications? • What is their relationship to other models? • What are the properties of conditional independence that make these models appropriate? • Usage in genetic linkage analysis
Applications of hand-built Bayes Nets • Answer Wizard 95, Office Assistant 97,2000 • Troubleshooters in Windows 98 • Lymph node pathology • Trauma care • NASA mission control Some Applications of learned Bayes Nets • Clustering users on the web (MSNBC) • Classifying Text (spam filtering)
Some factors that support intelligence • Knowledge representation • Reasoning • Learning / adapting
Outline for today • Basics • Knowledge-based construction • Probabilistic inference • Applications of hand-built BNs at Microsoft
Bayesian Networks: History • 1920s: Wright -- analysis of crop failure • 1950s: I.J. Good -- causality • Early 1980s: Howard and Matheson, Pearl • Other names: • directed acyclic graphical (DAG) models • belief networks • causal networks • probabilistic networks • influence diagrams • knowledge maps
Fuel Battery Fuel Gauge Engine Turns Over Start Bayesian Network p(f) p(b) p(g|f,b) p(t|b) p(s|f,t) Directed Acyclic Graph, annotated with prob distributions
Fuel Battery p(f) p(b) Fuel Gauge p(g|f,b) Engine Turns Over p(t|b) Start p(s|f,t) BN structure: Definition Missing arcs encode independencies such that
Independencies in a Bayes net Example: Many other independencies are entailed by (*): can be read from the graph using d-separation (Pearl)
Fuel TurnOver Start Explaining Away and Induced Dependencies "explaining away" "induced dependencies"
T F Fuel (empty, not) TurnOver (yes, no) S Start (yes, no) Local distributions Table: p(S=y|T=n,F=e) = 0.0 p(S=y|T=n,F=n) = 0.0 p(S=y|T=y,F=e) = 0.0 p(S=y|T=y,F=n) = 0.99
T F Fuel (empty, not) TurnOver TurnOver (yes, no) S no yes Fuel Start (yes, no) p(start)=0 not empty empty p(start)=0 p(start)=0.99 Local distributions Tree:
node parents Lots of possibilities for a local distribution... • y = discrete node: any probabilistic classifier • Decision tree • Neural net • y= continuous node: any probabilistic regression model • Linear regression with Gaussian noise • Neural net
Class ... Input 1 Input 2 Input n Naïve Bayes Classifier discrete
Hidden Markov Model discrete, hidden H1 H2 H3 H4 H5 ... ... X1 X2 X3 X4 X5 observations
Feed-Forward Neural Network X1 X1 X1 inputs hidden layer sigmoid Y1 Y2 Y3 outputs (binary) sigmoid
Outline • Basics • Knowledge-based construction • Probabilistic inference • Decision making • Applications of hand-built BNs at Microsoft
Building a Bayes net by hand(ok, now we're starting to be Bayesian) • Define variables • Assess the structure • Assess the local probability distributions
What is a variable? • Collectively exhaustive, mutually exclusive values Error Occured No Error
Clarity Test: Is the variable knowable in principle • Is it raining? {Where, when, how many inches?} • Is it hot? {T 100F , T < 100F} • Is user’s personality dominant or submissive? {numerical result of standardized personality test}
Assessing structure(one approach) • Choose an ordering for the variables • For each variable, identify parents Pai such that
TurnOver Gauge Start Battery Example Fuel
TurnOver Gauge Start Battery Example Fuel p(f)
TurnOver Gauge Start Battery Example Fuel p(b|f)=p(b) p(f)
TurnOver Gauge Start Battery Example Fuel p(b|f)=p(b) p(f) p(t|b,f)=p(t|b)
TurnOver Gauge Start Battery Example Fuel p(b|f)=p(b) p(f) p(t|b,f)=p(t|b) p(g|f,b,t)=p(g|f,b)
Fuel TurnOver Gauge Start Battery Example p(b|f)=p(b) p(f) p(t|b,f)=p(t|b) p(g|f,b,t)=p(g|f,b) p(s|f,b,t,g)=p(s|f,t) p(f,b,t,g,s) = p(f) p(b) p(t|b) p(g|f,b) p(s|f,t)
Start Gauge Fuel Battery TurnOver Why is this the wrong way?Variable order can be critical
Fuel Battery Gauge TurnOver Start A better way:Use causal knowledge
Fuel TurnOver Gauge Start Battery Conditional Independence Simplifies Probabilistic Inference
Portion of BN for print troubleshooting (see Breese & Heckerman, 1996)
Lumière Project (see Horvitz, Breese, Heckerman, Hovel & Rommelse 1998) User’s Goals User’s Needs User Activity
Studies with Human Subjects • “Wizard of OZ” experiments at MS Usability Labs User Actions Typed Advice Inexperienced user Expert Advisor
Several classes of evidence Activities with Relevance to User’s Needs • Search: e.g., menu surfing • Introspection: e.g., sudden pause, slowing of command stream • Focus of attention: e.g, selected objects • Undesired effects: e.g., command/undo, dialogue opened and cancelled • Inefficient command sequences • Goal-specific sequences of actions .
Summary so far Bayes nets are useful because... • They encode independence explicitly • more parsimonious models • efficient inference • They encode independence graphically • Easier explanation • Easier encoding • They sometimes correspond to causal models • Easier explanation • Easier encoding • Modularity leads to easier maintenance
Teenage Bayes MICRONEWS 97: Microsoft Researchers Exchange Brainpower with Eighth-grader Teenager Designs Award-Winning Science Project .. For her science project, which she called "Dr. Sigmund Microchip," Tovar wanted to create a computer program to diagnose the probability of certain personality types. With only answers from a few questions, the program was able to accurately diagnose the correct personality type 90 percent of the time.
Artificial Intelligence is a promising fieldalways was, always will be.