190 likes | 519 Views
What is Hopfield's Network?. This is network capable of associating its input with one of patterns stored in network's memoryNext logical questionsHow patterns are stored in memory?How inputs are supplied to the networkWHAT IS THE TOPOLOGY OF THE NETWORK. Inputs. The inputs of the Hopfield netw
E N D
1. Hopfields Network Dragoljub Pokrajac
2003
2. What is Hopfields Network? This is network capable of associating its input with one of patterns stored in networks memory
Next logical questions
How patterns are stored in memory?
How inputs are supplied to the network
WHAT IS THE TOPOLOGY OF THE NETWORK
3. Inputs The inputs of the Hopfield network are values x1,,xN
-1? xi?1
Hence, the vector x=[x1 xN] represents a point from a hyper-cube
4. Topology
5. A Neuron of Hopfields Network
6. How Neuron is Characterized Neuron is characterized by its state si
The output of the neuron is the function of the neurons state: yi=f(si)
The applied function f is soft limiter which effectively limits the output to the [-1,1] range
7. Operations of Neuron Neuron initialization
When an input vector x arrives to the network, the state of i-th neuron, i=1,,N is initialized by the value of the i-th input:
si=xi
8. Updates of Neurons Subsequently
While there is any change:
9. Convergence The subsequent computation of the network will occur until the network does not stabilize
The network will stabilize when all the states of the neurons stay the same
IMPORTANT PROPERTY:
Hopfields network will ALWAYS stabilize after finite time
10. What is Output of the Network Output of the network is vector y=[y1yn] consisting of neuron outputs when the network stabilizes
11. Why Hopfields Network is Important? Assume that we want to memorize M different N-dimensional vectors
What does it mean to memorize?
It means: if a vector similar to one of memorized vectors is brought to the input of the Hopfield network
The stored vector closest to it will appear at the output of the network
12. The following can be proven. If the number M of memorized N-dimensional vectors is smaller than N/4ln(N)
Then we can set the weights of the network as:
Where W contains weights of the network
a symmetric matrix with zeros on main diagonal
NONE of the neurons is connected to itself
Such that the vectors correspond to the stable states of the network
13. In Other Words If vector xm* is on the input of the Hopfields network
the same vector xm* will be on its output
If a vector close to vector xm* is on the input of the Hopfields network
The vector xm* will be on its output
14. Hence The Hopfield network memorizes by embedding knowledge into its weights
15. Why Hopfields Network Works With the N neurons, total number of output vectors is 2N
Only a small fraction (M) of these vectors represent useful information (one of M stored vectors)
This make the Hopfield network highly redundant
NOTE: Hopfield network resembles Error Correcting Codes (ECOC)
16. Main Problems with Hopfield Network What is close
OK, the output associated to input is one of stored vectors closest to the input
However, the notion of closeness is hard encoded in the weight matrix and we cannot have influence on it
17. Main Problems with Hopfield Network Spurious states
Assume that we memorized M different patterns into a Hopfield network
The network may have more than M stable states
Hence (the Pidgeon Hole Principle) the output may be NONE of the vectors that are memorized in the network
In other words: among the offered M choices, we could not decide
18. Main Problems with Hopfield Network What if vectors xm* to be learned are not exact (contain error)?
In other words:
If we had two patterns representing class 1 and class 2, we could assign each pattern to a vector and learn the vectors
However, it we had 100 different patterns representing class 1, and 100 patterns representing class 2, we cannot assign one vector to each pattern
19. Show Matlab demo for Hopfield Networks