Local structures; Causal Independence, Context-sepcific independance

1. Local structures;Causal Independence,Context-sepcific independance COMPSCI 276 Fall 2007

2. Reducing parameters of families • Determinizm • Causal independence • Context-specific independanc • Continunous variables Local structure

3. Local structure

4. Causal Independence • Event X has two possible causes: A,B. It is hard to elicit P(X|A,B) but it is easy to determine P(X|A) and P(X|B). • Example: several diseases causes a symptom. • Effect of A on X is independent from the effect of B on X • Causal Independence, using canonical models: Noisy-O, Noisy AND, noisy-max A B X Local structure

5. Binary OR A B X A B P(X=0|A,B) P(X=1|A,B) 0 0 1 0 0 1 0 1 1 0 0 1 1 1 0 1 Local structure

6. Noisy-OR A B “noise” is associated with each edge described by noise parameter   [0,1] : Let q b=0.2, qa=0.1 P(x=0|a,b)= (1-a)(1-b) P(x=1|a,b)=1-(1-a)(1-b) a b X A B P(X=0|A,B) P(X=1|A,B) 0 0 1 0 0 1 0.1 0.9 qi=P(X=0|A_i=1,…else =0) 1 0 0.2 0.8 1 1 0.02 0.98 Local structure

7. Noisy-OR with Leak A B Use leak probability 0  [0,1] when both parents are false: Let a=0.2, b=0.1, 0 = 0.0001 P(x=0|a,b)= (1-0)(1-a)a(1-b)b P(x=0|a,b)=1-(1-0)(1-a)a(1-b)b a b X A B P(X=0|A,B) P(X=1|A,B) 0 0 0.9999 0.0001 0 1 0.1 0.9 1 0 0.2 0.8 1 1 0.02 0.98 Local structure

8. Closed Form Bel(X) - 1 Given: noisy-or CPT P(x|u) noise parameters i Tu = {i: Ui = 1} Define: qi = 1 - I, Then: q_i is the probability that the inhibitor for u_i is active while the Local structure

9. Closed Form Bel(X) - 2 Using Iterative Belief Propagation: Set piix = pix (uk=1). Then we can show that: Local structure

10. Causal Influence Defined X1 X1 X1 Definition 2 Let Y be a random variable with k parents X1,…,Xk. The CPT P(Y|X1,…Xk) exhibits independence of causal influence (ICI) if it is described via a network fragment of the structure shown in on the left where CPT of Z is a deterministic functions f. Z0 Z1 Z2 Zk Z Y Local structure

11. Local structure

12. Local structure

13. Local structure

14. Local structure

15. Local structure

16. Context Specific Independence • When there is conditional independence in some specific variable assignment Local structure

17. Local structure

18. Local structure

19. Local structure

20. Local structure

21. The impact during inference • Causal independence in polytrees is linear during inference • Causal independence in general can sometime be exploited but not always • CSI can be exploited by using operation (product and summation) over trees. Local structure

22. Representing CSI • Using decision trees • Using decision graphs Local structure

23. A student’s example Difficulty Intelligence Grade SAT Apply Letter Job Local structure

24. Tree CPD • If the student does not apply, SAT and L are irrelevant • Tree-CPD for job A a0 a1 S (0.8,0.2) s0 s1 L (0.1,0.9) l0 l1 (0.9,0.1) (0.4,0.6) Local structure

25. Definition of CPD-tree • A CPD-tree of a CPD P(Z|pa_Z) is a tree whose leaves are labeled by P(Z) and internal nodes correspond to parents branching over their values. Local structure

26. Choice Letter1 Letter2 Job Captures irrelevant variables C c1 c2 L1 L2 l10 l11 l20 l21 (0.9,0.1) (0.3,0.7) (0.8,0.2) (0.1,0.9) Local structure

27. Choice Letter1 Letter2 Letter Job Multiplexer CPD • A CPD P(Y|A,Z1,Z2,…,Zk) is a multiplexer iff Val(A)=1,2,…k, and • P(Y|A,Z1,…Zk)=Z_a Local structure

28. Rule-based representation • A CPD-tree that correponds to rules. A a0 a1 B C b0 b1 c0 c1 C (0.1,0.9) B (0.2,0.8) c0 c1 b0 b1 (0.3,0.7) (0.4,0.6) (0.3,0.7) (0.5,0.5) Local structure

29. N(m, s) Gaussian Distribution Local structure

30. 0.4 gaussian(x,0,1) gaussian(x,1,1) 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 -3 -2 -1 0 1 2 3 N(m, s) Local structure

31. 0.4 gaussian(x,0,1) gaussian(x,0,2) 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 -3 -2 -1 0 1 2 3 N(m, s) Local structure

32. Multivariate Gaussian Definition: Let X1,…,Xn. Be a set of random variables. A multivariate Gaussian distribution over X1,…,Xn is a parameterized by an n-dimensional mean vector  and an n x n positive definitive covariance matrix . It defines a joint density via: Local structure

33. Linear Gaussian Distribution Definition: Let Y be a continuous node with continuous parents X1,…,Xk. We say that Y has a linear Gaussian model if it can be described using parameters 0, …,k and 2 such that: P(y| x1,…,xk)=N (0 + 1x1 +…,kxk ; 2 ) Local structure

34. X Y X Y X Y Local structure

35. Local structure

36. Linear Gaussian Network Definition Linear Gaussian Bayesian network is a Bayesian network all of whose variables are continuous and where all of the CPTs are linear Gaussians. Linear Gaussian BN  Multivariate Gaussian =>Linear Gaussian BN has a compact representation Local structure

37. Hybrid Models • Continuous Node, Discrete Parents (CLG) • Define density function for each instantiation of parents • Discrete Node, Continuous Parents • Treshold • Sigmoid Local structure

38. Continuous Node, Discrete Parents Definition: Let X be a continuous node, and let U={U1,U2,…,Un} be its discrete parents and Y={Y1,Y2,…,Yk} be its continuous parents. We say that X has a conditional linear Gaussian (CLG) CPT if, for every value uD(U), we have a a set of (k+1) coefficients au,0, au,1, …, au,k+1 and a variance u2 such that: Local structure

39. CLG Network Definition: A Bayesian network is called a CLG network if every discrete node has only discrete parents, and every continuous node has a CLG CPT. Local structure

40. Discrete Node, Continuous ParentsThreshold Model Local structure

41. Discrete Node, Continuous ParentsSigmoid Binomial Logit Definition: Let Y be a binary-valued random variable with k continuous-valued parents X1,…Xk. The CPT P(Y|X1…Xk) is a linear sigmoid (also called binomial logit) if there are (k+1) weights w0,w1,…,wk such that: Local structure

42. 1 sigmoid(0.1*x) sigmoid(0.5*x) 0.95 sigmoid(0.9*x) 0.9 0.85 0.8 0.75 0.7 0.65 0.6 0.55 0.5 0 5 10 15 20 Local structure

43. References • Judea Pearl “Probabilistic Reasoning in Inteeligent Systems”, section 4.3 • Nir Friedman, Daphne Koller “Bayesian Network and Beyond” Local structure