Neural network (II) — HNN Hopfield Neural Network

1. Neural network (II) —HNNHopfield Neural Network Date : 2002/09/24 Present by John Chen E-mail : phd9008@cs.nchu.edu.tw

2. Outline • Preliminaries • Introduction • HNN algorithm • Application & Researh Topic • Conclusion

3. Preliminaries • Neural Networks are built of neurons and their connections • The characteristics of Neural Network • Learning capability • The capability of Storage • Fault tolerance • The capability of induce • Pallel processing

4. Preliminaries(cont) • Hebb learning rule • Question : How to learn ? Where to keep memory? • Hebb proposed learning rule in 1949 • Learning is just a local appearance , it is correlated with the excited degrees between connected neurons • It is also called correlated learning rule • dWij/dt = SjXj

5. Preliminaries(cont) Fig Computing Model of Neuron

6. Introduction • Concept ofHopfield Neural Network • Proposed by J. Hopfield in 1982 • Provide the base of research theory • Graph of Hopfield Neural Network

7. Introduction (cont) • The properties of HNN • Parallel input , Parallel output • Operation process divide into two part • Memorizing process • Remembering process • IN memorizing process • Update weights by Hebb learning rule • ∆Wij＝ηXiXj • IN remembring process • Output a result most similar to memorizing example by calculating

8. Introduction (cont) • Two type of HNN • Discrete : (1 or -1) , (0 or 1) • Continuous : real value between 0,1 • The cost function of HNN • Xi：status value of i’th neuron • Xj： status value of j’th neuron • Wij：connection weight between i’th & j’th neuron • θj：bias value of j’th neuron

9. HNN Algorithm • Algorithm of memorizing • Step 1 : set network parameters • Step 2 : read connection weights set Wij＝ XipXjp and Wii＝0

10. HNN Algorithm(cont) • Algorithm of remembering • Step 1 : set network parameters • Step 2 : read connection weights • Step 3 : Input initial vector X • Step 4 : Calculate new vector X neti＝ WijXj 1 if neti＞0 Xinew＝Xiold if neti＝0 －1 if neti＜0 • Step 5 : repeat until network converge

11. Application & Researh Topic • 時空型霍菲爾類神經網路於鼻咽部復發腫瘤之偵測 — TAAI 2001 • 作者：張傳育 樹德科技大學 資訊工程系 • 提出立體時空型霍菲爾類神經網路(SHNC) 來偵測鼻咽部復發腫瘤 • SHNC 結合動態影像的時空資訊及像素點間的結構資訊，對每個像素點作分類；可有效過濾影像中的雜訊 • 採用了競爭式的學習法則，加快了網路收斂的速度 • 霍菲爾類神經網路(HNN)是屬於非監督式學習網路，免除了事先訓練網路的麻煩 • 經實驗可知SHNC所偵測的結果比 K-means , PCA等方法來得正確有效率

12. Application & Researh Topic(cont) • Neural Networks for Visual Cryptography — with Examples for complex Acess Schemes • Author : Suchen Chiang , Tai-Wen Yue Tatung University • Proposed Q’tron NN Model • Derived from HNN • Design Energies function • For Halftoning • For Restoration • For (2,2) Visual Cryptography • Can be extend for (2,2) Visual Cryptography but in another paper

14. Application & Researh Topic(cont) • Reseach Reference • Journal • Neural Network • IEEE Trans. on Neural Network • IEEE Trans. On system, Man , and Cybernetics • IEICE Trans. On Information and system • Conference • ICNN 1987 ~ 1988 (神經網路國際研討會) • IJCNN 1989 ~ (神經網路國際聯合研討會) : IEEE and INNS • TAAI 人工智慧與應用研討會(台灣)

15. Application & Researh Topic(cont) • Research Direction • Read and Reference • Preprocess • Model Modify

16. Conclusion • Research like the learning rule of Neural Network • If I have more time or more resource