A Hybrid neural network model for consciousness

1. A Hybrid neural network model for consciousness LIN Jie, JIN Xiao-gang, YANG Jian-gang (Institute of Artificial Intelligence, Zhejiang University, Hangzhou 310027, China) Date of publication: Mar, 2004 Presented by: Bhuban M Seth, Joydip Datta Under the guidance of: Prof. Dr. Pushpak Bhattacharyya

2. Motivation • Ultimate goal of Artificial Neural Net is to imitate a human brain. • But human brain is too complex to understand. • Question: What is a consciousness and How it is generated in brain? • Is there any hierarchical organization in the brain? • How can we incorporate these newfound insights of human brain into an ANN?

3. Understanding the brain(Different Approaches) • Taylor (1994): Relational Mind • Rakovic (1997): hierarchically organized and interconnected paradigm for information processing inside the brain. • Vitiello (2003): Quantum Model • Rennie et. al. (2002): Evoked potential

4. Where all these things leads to? • Cognitive processes are carried out at different levels in the brain. • Higher levels may be reduced to lower levels. • Thus, higher levels of complex brain functions require a number of neural modules to cooperate together. • Example: We see a rose, smell the fragrance and remember some memory… this way a conscious state of mind emerges in a thinking process.

5. Level 1: Physical Mnemonic Layer • Physical Mnemonic Layers (PML) capture input from external senses and produce a feature vector (patterns) from them. Many modular PMLs run in parallel. • There may be two kinds of external inputs: • Arousal Inputs: Reach only up to recognition Layer – Do not take part in Associative Recognition • Aware Inputs: Reaches Abstract Thinking Layer and may take part in Associative Recognition • The feature vectors are input to recognition layer

6. Level 2: Recognition Layer • It is a searching tree composed of layered storage neurons. • It receives a pattern from the PML.

7. Level 3: The Global Workspace • It belongs to the Abstract Thinking Layer. • It describes the state of Consciousness. • It can project the abstract information it has and mobilize different parts of the brain. • This global availability of information define the conscious state of mind.

9. Recognition Layer • Divided into levels • Each level consists of number of knowledge clusters • Input is the pattern formed by Physical Mnemonic Layer (PML) • This pattern is compared with stored patterns at all levels

10. Recognition Layer • If the pattern is similar to some existing patterns • it will be recognized. • Else, the pattern will be saved (New neurons will be created) • Similarity is measured by resonant coefficient

11. Inherent Frequency • Definition: • It is the feature vector of a pattern. • Inherent frequency of a neuron group k that • memorizes a knowledge pattern can be described by • the weights from one neuron in the group to other • members, as K=[wl, w2 ..... wi .... ].

12. Similar Patterns • Similarity of two patterns A, B are determined by their Resonant Coefficient R(A,B). • The resonant coefficient is a kind of delta similarity relation satisfying the following properties: • Reflexive: R(A, A)=1 • Symmetric: R(A,B) = R(B,A) • And 1 - | R(A,C) – R(B,C) | >= R(A,B) --(Upper bound) R(A,B) >= max(0, R(A,C)+R(B,C)-1 ) --(Lower bound)

13. Example • Suppose a series of four-dimension patterns Pi (i=0,1,2 .... ,9) formed by PML models enter RL. Say, Pi is the binary format of i as • P3=[0,0,1,1], P5=[0,1,0,1], P1=[0,0,0,1]. • We can define resonant coefficient R(Pi, Pj) as R(Pi,Pj) = 1 – (XOR(Pi and Pj)/ 4) • Then R(P0,P0)=1,R(P0,P1)=0.75,R(P0,P2)=0.75 , R(P0,P3)=0.5 and so on.

14. Resonant Space • It is a representation of pattern showing similarity between them. • Definition: It is a space of patterns to which any other pattern can be compared to evaluate resonant coefficient. • A pattern P is represented in resonant space by a single point, whose projection on an axis represents the resonant coefficient between the pattern corresponding to the axis and the pattern P.

15. Resonant Space(contd…) The resonant space formed by patterns P0 and P5

16. Cntd… Consider a resonant space Rn with n patterns Pi and the resonant coefficient R(Pi, Pj) between any two patterns Pi and Pj . From the resonant space formed by n patterns , a pattern Pm may be represented on Rn as: where is the unit vector along Pi axis.

17. Threshold in Recognition Layer • Definition: The thresholds exist in RL corresponding to different levels (numbered from zero to TOP): to>tL>tL+l>tTOP, patterns are clustered at those levels. For example, at level L, two patterns ~ belong to the same cluster if and only if tL>f(u,v)>tL+I. There also exists a highest threshold tmax and two patterns are recognized to be the same if f(u,v)>tmax.

18. Abstract Thinking Layer • It can associatively compare (and recognize) different types of inputs. • It can broadcast it’s contents to the nervous system as a whole allowing different modules to interact. • E.g. The ATL cat take input from the auditory and the vision subsystem and while associatively recognizing the inputs it can mobilize the olfactory subsystem.

19. Abstract Thinking Layer

20. Abstract Thinking Layer • The ATL is an Bi-directional Backpropagation network (BBP). • A1 and A2 are both input to of the BBP. • The computation is interleaved: only one-way learning is going on at a particular interval. • The structure (no of neurons in different layers of the BBP) of the ATL may vary depending on the inputs. • A subset of the neurons are excited at a time while rest of them are inhibited. This in general represents the consciousness.

21. Consciousness in ATL • Dynamic workspace states are self sustained and follow one another in a continuous stream, without external help • Consciousness generation requires a stable activation loop. • The system enters a stable state V* (attractor) when there no more change in the state possible:V* = V(t+∆t) = V(t), ∆t > 0

22. Time span threshold in GW • Establishment of stable state requires a minimal duration. • There is a temporal span of successive workspace states. • If patterns from several subsystems appear in the ATL longer than some Time span threshold then a conscious state emerges. • Otherwise they can not establish a self sustained activation loop – They are called sub-consciousness.

23. Conclusion • Different levels exists in consciousness generation process. • Partial recognition layer threshold helps to form clusters within RL unconsciously. • Strong pattern that persists for more than a time span threshold can accomplish associative recognition resulting in consciousness.

24. Background Study • Wikipedia articles on: Brain, Human Brain, Cerebral Cortex, Hippocampus etc (different parts of brain), Neuron, Action Potential, Depolarizing, Hyperpolarizing, Inhibited Neurons, Excited Neurons, Axon Hillock, Back-propagation, Neural Back-propagation, Resting potential, Layered perceptron, MLP, Electrical Inductance, Electrical Resonance etc. • Hierarchical Learning in Neural Network: http://www.cs.iastate.edu/~baojie/acad/current/hnn/hnn.htm • A Bi-Directional Multilayer PerceptronM. JEDRA, A. EL OUARDIGHI, A. ESSAID and M. LIMOURILaboratoire Conception & Systèmes, Faculté des Sciences, Avenue IbnBatouta, B.P. 1014, Rabat10 000, Morocco, e-mail: jedra@fsr.ac.ma