1 / 15

Landscapes of the brain and mind

Wan Ahmad Tajuddin Wan Abdullah* Complex Systems Group Department of Physics Universiti Malaya 50603 Kuala Lumpur *http://fizik.um.edu.my/cgi-bin/hitkat?wat. 3 rd MPSGC KUALA LUMPUR 2007. Landscapes of the brain and mind. 3 rd MPSGC KUALA LUMPUR 2007.

mike_john
Download Presentation

Landscapes of the brain and mind

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Wan Ahmad Tajuddin Wan Abdullah* Complex Systems Group Department of Physics Universiti Malaya 50603 Kuala Lumpur *http://fizik.um.edu.my/cgi-bin/hitkat?wat 3rd MPSGC KUALA LUMPUR 2007 Landscapes of the brain and mind

  2. 3rd MPSGC KUALA LUMPUR 2007 The Little-Hopfield Neural Network Model McCulloch-Pitts neurons: • Binary: Vi = 0,1 • Summed inputs: hi = Σj TijVj • Theshold: Vi→H(hi-Ui)

  3. 3rd MPSGC KUALA LUMPUR 2007 The Little-Hopfield Neural Network Model Network: N coupled nonlinear equations – Vi(t+τ) = H(ΣjTijVj(t)-Ui) solve simultaneously!

  4. 3rd MPSGC KUALA LUMPUR 2007 The Little-Hopfield Neural Network Model If • exchange symmetry in synaptic strengths Tij = Tji • no self-interactions Tii = 0 dynamics understood in terms of a Lyapunov function E = - ½ Σi Σj TijViVj+Σi UiVi Look: ΔE = - hiΔVi monotone decreasing wrt neuron dynamics cf conservative forces  potential function cf spin systems (bipolar neurons, Si = 1)

  5. Energy Configuration 3rd MPSGC KUALA LUMPUR 2007 The Little-Hopfield Neural Network Model 'energy' landscape gradient descent energy minimum ≡ stable configurations Energy Configuration

  6. 3rd MPSGC KUALA LUMPUR 2007 The Little-Hopfield Neural Network Model Combinatorial optimization – • map combinatorial choices to neuron configuration • map cost function to energy function • obtain synaptic weights • let network relax to minimum energy configuration

  7. 3rd MPSGC KUALA LUMPUR 2007 The Little-Hopfield Neural Network Model Associative memory – optimize which ‘image’ stored nearest to input ‘image’ (initial configuration) Tij := Σ(r) (2Vi (r) - 1) (2Vj(r) - 1) Ui = 0 Cooper-Hopfield prescription Check: E = - ½ Σi Σj Σ(r) (2Vi(r) - 1) (2Vj (r) - 1)ViVj minimum when Vi~ Vi(r) • spurious memories – local minima • forgetting • temperature – simulated annealing • basins of attraction

  8. 3rd MPSGC KUALA LUMPUR 2007 Energy landscapes Minima = stable states / solutions Global minima = good solutions Local minima = spurious states / solutions Ruggedness (measured by e.g. correlations) = difficulty in finding solution

  9. 3rd MPSGC KUALA LUMPUR 2007 Logic Programming Bird(x)Have_feathers(x),Fly(x). x Bird if x Have_feathers and x Fly. Fly(Tweety). Tweety Fly. Have_feathers(Tweety). Tweety Have_feather. Have_fur(Sylvester). Sylvester Have_fur.  Bird(Tweety) Horn clauses – at most 1 logical atom in consequent

  10. 3rd MPSGC KUALA LUMPUR 2007 Logic Programming on Little-Hopfield networks Logic programming ~ minimization of “logical inconsistency” A ← B, C. A v ¬B v ¬C D ← B. D v ¬B C ←. C EP = ⅛(1 - SA) (1 + SB) (1 + SC) + ¼ (1 - SD) (1 + SB) + ½ (1 - SC) 3rd order bipolar neural network E = - ⅓ ΣiΣjΣkJijk(3)SiSjSk - ½ Σi Σj Jij(2)SiSj- Σi Ji(1)Si Si := sign(Σj ΣkJijk(3)SjSk + Σj Jij(2)Sj+ Ji(1) )

  11. 3rd MPSGC KUALA LUMPUR 2007 Logic Programming on Little-Hopfield networks translate clauses in the logic program • Boolean algebraic form. • Derive a cost function that is associated with the negation of all the clauses • Obtain the values of connection strengths by comparing the cost function with the energy function • Let the neural networks evolve until minimum energy is reached. • The neural states provide a solution interpretation for the logic program, and the truth of a ground atom in this interpretation may be checked .

  12. 3rd MPSGC KUALA LUMPUR 2007 Logic Programming on Little-Hopfield networks translate clauses in the logic program • how rugged is landscape from logic programming? Computersimulations:… [SS & WATWA] – flat energy landscape  no satisfiability problem (all clauses can be satisfiedi.e. solution always guaranteed)

  13. 3rd MPSGC KUALA LUMPUR 2007 Satisfiability In general, general CNF clauses(conjuctions of disjunctions)not necessarily satisfiable - depends on number of atoms in disjunctions number of disjunctions number of distinct atoms Exist phase transition from easily soluble problems to difficult problems [Zecchina & Monasson]

  14. 3rd MPSGC KUALA LUMPUR 2007 Can 'knowledge' may be associated with the energy landscape? What is knowledge? ? How ingrained are logical rules in neural network – “ingrainedness”: GX←Y,Z = < E({Si} satisfying X←Y,Z) - E({Si} not satisfying X←Y,Z) > Can this be related to landscape ruggedness e.g. correlations?

  15. 3rd MPSGC KUALA LUMPUR 2007 Terima kasih

More Related