450 likes | 578 Views
Polynomial Discrete Time Cellular Neural Networks. Eduardo Gomez-Ramirez † Giovanni Egidio Pazienza‡ † LIDETEA, POSGRADO E INVESTIGACION Universidad La Salle – México, D.F. ‡ Department d’Electronica, EALS Universitat “Ramon Llull” – Barcelona, Spain. Outline.
E N D
Polynomial Discrete Time Cellular Neural Networks Eduardo Gomez-Ramirez † Giovanni Egidio Pazienza‡ † LIDETEA, POSGRADO E INVESTIGACION Universidad La Salle –México, D.F. ‡ Department d’Electronica, EALS Universitat “Ramon Llull” – Barcelona, Spain
Outline • Cellular Neural Networks (CNN) • Introduction and Objective • Genetic Algorithms (GA) • Polynomial Discrete Time CNNs (PDTCNNs) • XOR Problem • Game of Life • Learning vs Design • Conclusions and future work
CNN: Introduction • CNN for complex task (linearly nonseparable data) • Multilayer CNNs • Include more degrees of freedom for the output state of each layer • Search in a finite set of templates • Single layer: Polynomial CNNs
Objective • Improve the representation power of a single layer CNN including a simple nonlinear term to solve problems with linearly nonseparable data (XOR)
CNN: mathematical model • The simplified mathematical model is: where xc is the state of the cell, uc the input and yc the output
CNN: Discrete Model • Computing x(∞), the model can be represented as using the following activation function
GA: main steps proposed • Steps: • Crossover C(Fg) • Mutation M(*) • Adding random parent Ag()
a1 I=0 b1 c1 a2 I=0 b2 c2 GA: Crossover • Individual 1 • Individual 2
GA: Mutation where rU(0,1) is a random variable with uniform distribution defined on a probability space (,,P),
a1 I=0 b1 c1 a2 I=0 b2 c2 GA: Mutation (resolution) • Individual 1 • Individual 2
Polynomial Discrete Time Cellular Neural Network THEOREM 1: (Weierstrass’s Approximation Theorem) Let g be a continuous real valued function defined on a closed interval [a,b]. Then, given any positive, there exists a polynomial y (which may depend on ) with real coefficients such that: For every x [a,b].
Polynomial Discrete Time Cellular Neural Network • THEOREM 2 *: Any Boolean Function of n-variables can be realized using a Polynomial Threshold gates of order sn. The quadratic threshold gate can be defined: And s is the number of inputs and T is the threshold constant. * N. J. Nilsson. The Mathematical Foundations of Learning Machines. McGraw Hill, New York, 1990.
PDTCNN: Solving XOR problem Some papers: • Z. Yang, Y. Nishio, A. Ushida, Templates and algorithms for two-layer cellular neural networks. IJCNN’02, 2002. • F. Chen, G. He, G. Chen & X. Xu, Implementation of Arbitrary Boolean Functions via CNN. CNNA’06, 2006.
PDTCNN:Solving XOR problem • M. Balsi, Generalized CNN: Potentials of a CNN with Non-Uniform Weights. CNNA-92, 2002 . • E. Bilgili, I. C. Göknar and O. N. Ucan, Cellular neural network with trapezoidal activation function. Int. J. Circ. Theor. Appl., 2005
Learning parameters • Initialpop=20000 • Number of fathers=7 • Maximum number of random parents to be add = 3 • Kpro=0.8 • Increment=1 • Mutation Probability=0.15
The Game of Life (I) • The Game of Life (GoL) is a totalistic cellular automaton consisting in a two-dimensional grid cells, that may be either alive (black) or dead (white).
The Game of Life (II) • The state of each cell varies according to the following rules: • Birth: a cell that is dead at time t becomes alive at time t + 1 only if exactly 3 of its neighbors were alive at time t; • Survival: a cell that was living at time t will remain alive at t + 1 if and only if it had exactly 2 or 3 alive neighbors at time t.
The Game of Life (III) • Every sufficient well-stated mathematical problem can be reduced to a question about Life; • It is possible to make a life computer (logic gates, storage etc.); • Life is universal: it can be programmed to perform any desired calculation; • Given a large enough Life space and enough time, self-reproducing animals will emerge... • The whole universe is a CA! (E.Fredkin, MIT).
A The Game of Life – NOT gate
CNN & GoL • Multilayer CNN (Chua, Roska) – 1990 • Activation function (Chua, Roska) – 1990 • CNN-UM (Roska,Chua) -1990 • CNN Universal Cells (Dogaru, Chua) – 1999 Simplicity vs. Computational power
Polynomial CNN (I) What’s g(ud,yd)?
Polynomial CNN (II) • In the simplest case g(ud, yd) is a second degree polynomial, whose general form is
Polynomial CNN (III) Thanks to some considerations we find that
uc and appear in the state equation direct link with totalistic Cellular Automata Polynomial CNN (IV)
Black pixel= +1 White pixel= -1 GoL: Rules (II) • Rule 1: a cell will be alive at least 3 of the 9 cells in its 3 × 3 neigh. are alive pixel centr. = 1 (black) Σ neigh. = -2 (5 w, 2 b) next state = -1 (white)
Black pixel= +1 White pixel= -1 GoL: Rules (III) • Rule 2: a cell will be alive if at most 3 of its 8 neighbors are alive pixel centr. = 1 (black) Σ neigh. = -2 (5 w, 2 b) next state = 1 (black)
Design algorithm (I) • First iteration: we try to perform the first rule (a cell will be alive at least 3 of the 9 cells in its 3 × 3 neigh. are alive) • If Y(0)=0 bc=1 bp=1 i=3
Design algorithm (II) • Second iteration: we try to accomplish the second rule (a cell will be alive if at most 3 of its 8 neighbors are alive)
Design algorithm (III) • Hyp: pc=0
Templates found using learning • Coming soon...
Conclusions (I) • In general: • In some cases it is possible to reduce a multilayer DTCNN to a single layer PDTCNN • Thanks to the GoL we can explore the capacity of PDTCNNs for Universal Machine
Conclusions (II) • About learning: • The resolution used reduces the search space • The step “Add random parent” improves the behavior to avoid local minimas • About design • We give a simple algorithm to design templates for the Polynomial CNN
Future Work • Implementations of mathematical morphology functions with PDTCNNs
Polynomial Discrete Time Cellular Neural Networks Eduardo Gomez-Ramirez Giovanni Egidio Pazienza egr@ci.ulsa.mx gpazienza@salle.url.edu