Acceleration of Inductive Inference of Causal Diagrams

1. Acceleration of Inductive Inference of Causal Diagrams Olexandr S. Balabanov Institute of Software Systems of NAS of Ukraine bas@isofts.kiev.ua

2. Statistical Data (Sample) Structure of Data Generation Process (Causal Model) No prior knowledge No Temporal Order of Variables Generic Task: Our Goal : to speed up a model induction We obey a constraint-basedapproach to model induction

3. Edges Model skeleton identification via searching for separators Edge orientation Separators Structure of model Calculating parameter values Data 3 Phases of Causal Inference

4. A constraint-based algorithm deletes an edge X—Y when finds a fact that variables X and Y are conditionally independent under some condition. The algorithm tries to find a separator for each pair of variables. The key idea of PC-algorithm is: to include in a tentative separator for pair (X, Y) only those variables which are supposedly adjacent to X or to Y.

5. But still a task of searching for separators remains computationally very expensive even for networks of moderate density. The worst situation: When there edge X—Y exists, the PC continues an attempts to find a separator for (X,Y). The algorithm would examine all subsets of Adj(X) and all subsets of Adj(Y) as tentative separators.

6. It is especiall desirable to recognize the edge presence as early as possible. Also very useful is to come with tests of low rankwhenever possible. This means to find minimal separators. Idea to achieve the goal – to exploit pairwise Markov properties of ADG-model, concept of locally-minimal separator and their logical consequences.

7. We have developed several rules of inductive inference acceleration. These rules perform: 1) recognition of edge presence; 2) recognition of edge absence; 3) deleting some variables from list of candidates in supposed separator; 4) recognition of some variables as obligate members of respective separator (if it exists at all).

8. One of the most effective rules: Rule of ‘placing aside’ : If there Ds(Z;X;Y) &ØDs(Z;;Y)holds in model G, then vertex Z is not a member of any locally-minimal d-separator for pair (X,Y)in G.

9. If we equip an algorithm (like PC) with just the two rules (placing aside rule and “lack of separator’s pivot” rule), then the algorithm would recover a forest (or poly-forest) by executing tests of zero- and first-rank. In particular, algorithm Razor-1.1 would identify a forest, presented below, by tests of 0-rank and 1-rank only. Basic PC algorithm for the same model would work out test up to 9-rank.

10. 10 1 9 2 8 7 3 4 6 5 Algorithm Razor-1.1 (or even simpler one, but with the two rules) requires tests of first rank at max. PC algorithm requires tests of 8-rank at max.

11. 15 4 13 9 2 1 11 12 10 14 3 5 6 7 8 More complicated and realistic example. This structure consists of 15 vertices and 30 edges. Razor-1.1 requires tests of 4th rank at max. PC algorithm requires tests of 8th rank at max. .

12. Below –results of inference from data samples. ADG structures were generated randomly for 20 vertices (variables) and number of edges = 40 –70. Variables – binary and ternary. Model’s parameters –also randomly generated. Sample size = 20000.

13. PC Razor Experimental results: Performance 20 vertices (variables), 50 edges.

14. PC Razor Razor PC Inference Errors 20 vertices (variables), 50 edges. Notice : These results present uncomfortable cases (with binary and ternary variables and random parameters).

15. Extension Most of the rules of inductive inference acceleration may be extended to the case of causal diagrams with latent variables (some corrections to the algorithm should be done). Algorithm needs to be upgraded for the case of causally-insufficient models. Conclusion As demonstrated, algorithm equipped with proposed rules performs learning Bayesian nets (of moderate density) multiple times faster then PC algorithm. At the same time, number of errors grows much more slowly. Thus inductive inference acceleration rules facilitate fast identification of skeleton of causal model.

16. Thanks for attention Recent publications • Balabanov A.S. Minimal separators in dependency structures: Properties and identification. Cybernetics and Systems Analysis. – Vol. 44, – No 6, 2008, – P.803–815. – Springer N.Y. • Balabanov A. S. Construction of minimal d-separators in a dependency system. Cybernetics and Systems Analysis. – Vol. 45, – No 5, –2009. – P. 703–713. • Balabanov O. S. Accelerating algorithms for Bayesian network recovery. Adaptation to structures without cycles (in Ukrainian). Problems in programming journal, – 2011. – No 1. – P.63–69. – Kiev, Ukraine, ISBN 1727-4907. • Balabanov O.S., O.S. Gapyeyev, A.M. Gupal, S.S. Rzhepetskyy. Fast algorithm for learning Bayesian networks from data. Journal of Automation and Information Sciences. – Vol. 43, – No 10, – 2011, to appear.