310 likes | 476 Views
A Cooperative Coevolutionary Genetic Algorithm for Learning Bayesian Network Structures. Arthur Carvalho a3carval@cs.uwaterloo.ca. Outline. Bayesian Networks CCGA Experiments Conclusion. Bayesian Networks. AI technique Diagnosis, predictions, modelling knowledge Graphical model
E N D
A Cooperative CoevolutionaryGenetic Algorithm for Learning Bayesian Network Structures Arthur Carvalho a3carval@cs.uwaterloo.ca
Outline • Bayesian Networks • CCGA • Experiments • Conclusion
Bayesian Networks • AI technique • Diagnosis, predictions, modelling knowledge • Graphical model • Represents a joint distribution over a set of random variables • Exploits conditional independence • Concise, natural representation
Bayesian Networks X1 X2 X3
Bayesian Networks X1 X2 X3
Bayesian Networks • Directed acyclic graph (DAG) • Nodes: random variables • Edges: direct influence of one variable on another • Each node is associated with a conditional probability distribution (CPD)
Bayesian Networks • Learning the structure of the network (DAG) • Structure learning problem • Learning parameters that define the CPDs • Parameter estimation • Maximum Likelihood estimation
Bayesian Networks • Structure learning problem in fully observable datasets • Find a DAG that maximizes P(DAG| Data) • [Cooper & Herskovits, 92]
Bayesian Networks • NP-Hard [Chickeringet al, 1994] • The number of possible structures is superexponential in the number of nodes [Robinson, 1977] • For a network with n nodes, the number of different structures is:
Outline • Bayesian Networks • CCGA • Experiments • Conclusion
CCGA • Structure learning task can be decomposed into two dependent subtasks • To find an optimal ordering of the nodes • To find an optimal connectivity matrix
CCGA D B A D C
CCGA D B A D C
CCGA D B A D C
CCGA D B A D C
CCGA D B A A C
CCGA D B A A C
CCGA D B A A C
CCGA D B A B C
CCGA D B A C
CCGA • Two subpopulations • Binary (edges) • Permutation (nodes) • Cooperative Coevolutionary Genetic Algorithm (CCGA) • Each subpopulation is coevolve using a canonical GA
CCGA • Evaluating individual species • Each subpopulation member is combined with both the best known individual and a random individual from the other subpopulation • The fitness function is applied to the two resulting solutions • The highest value is the fitness • CCGA-2 [Potter & De Jong, 1994]
Outline • Bayesian Networks • CCGA • Experiments • Conclusion
Experiments • Setup • K2 algorithm [Cooper & Herskovits, 1992] • Alarm network • 37 nodes and 46 edges • Insurance network • 27 nodes and 52 edges • Three datasets • 1000, 3000, and 5000 instances • 100 executions
Experiments • Parameters:
Experiments • Alarm network
Experiments • Insurance network
Outline • Bayesian Networks • CCGA • Experiments • Conclusion
Conclusion • New algorithm to solve the structure learning problem • Novel representation • Good performance • Future work • Incomplete datasets • Graph-related problems
Thank you! Source code and datasets available at: www.cs.uwaterloo.ca/~a3carval/softwares/CCGA_code.rar Arthur Carvalho a3carval@cs.uwaterloo.ca