1 / 24

Bioinformatics Week 9: Simulations #3: Neural Networks

Bioinformatics Week 9: Simulations #3: Neural Networks. biological neurons natural neural networks artificial neural networks applications. A Biological Neuron has…. soma (the ‘body’ of the neuron) dendrites (for inputs) axon (for output) synapses.

sabo
Download Presentation

Bioinformatics Week 9: Simulations #3: Neural Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bioinformatics Week 9: Simulations #3: Neural Networks • biological neurons • natural neural networks • artificial neural networks • applications JYC: CSM17

  2. A Biological Neuron has… • soma (the ‘body’ of the neuron) • dendrites (for inputs) • axon (for output) • synapses JYC: CSM17

  3. Natural Neural Networks • nerve net • in Coelenterates • e.g. Hydra, sea anemones JYC: CSM17

  4. The Human Brain • ~100 billion neurons • about as many trees in Amazon Rain Forest • the number of connections is about the same as the total number of leaves • up to 100 thousand inputs per cell JYC: CSM17

  5. The Human Brain (from the visible human project) JYC: CSM17

  6. Artificial Neurons • McCulloch & Pitts • single neuron model (1943) • … with weights becomes Hebbian Learning • Rosenblatt’s Perceptron • multi-neuron model (1957) JYC: CSM17

  7. Artificial Neural Networks • supervised • known classes • unsupervised • unknown classes JYC: CSM17

  8. Supervised Neural Networks • multilayer perceptron (MLP) • used where classes are known • trained on known data • tested on unknown data • useful for identification or recognition JYC: CSM17

  9. MLP Architecture • usually 3-layered (I:H:O) • one node for each attribute / character • input layer • one node for each attribute / character • hidden layer • variable number of nodes • output layer • one node for each class JYC: CSM17

  10. JYC: CSM17

  11. MLP Learning Algorithms • summation is carried out by • where wi is the weight and xi is the input value for input i. JYC: CSM17

  12. MLP Learning Algorithms • the non-linear activation function (φ) is given by • where vj is the weighted sum over n inputs for node j JYC: CSM17

  13. MLP Learning Algorithms • backpropagation • (Werbos) Rummelhart & McClelland 1986 • contribution of each weight to the output is calculated • weights are adjusted to be ‘better’ next time…using the delta rule JYC: CSM17

  14. MLP Learning Algorithms • delta rule • … for output nodes • … for hidden nodes JYC: CSM17

  15. Applications • identification / recognition • fault diagnosis e.g. teabag machine • medical diagnosis • decision making JYC: CSM17

  16. Unsupervised NNs • self-organising (feature) maps • ‘Kohonen’ maps • topological maps JYC: CSM17

  17. Kohonen Self-Organising Feature Map (SOM, SOFM) • Teuvo Kohonen (1960s) • input layer • one node for each attribute / character • competitive ‘Kohonen’ layer JYC: CSM17

  18. Kohonen SOM Architecture JYC: CSM17

  19. Kohonen Learning Algorithm • initially random weights between input layer and Kohonen layer • data records (input vectors) presented one at a time • each time there is one ‘winner’ (closest Euclidean distance) • the weights connected to the winner and its neighbours are adjusted so they are closer • learning rate and neighborhood size are reduced JYC: CSM17

  20. SOM Learning Algorithm JYC: CSM17

  21. WebSOM of comp.ai.neuralnets JYC: CSM17

  22. Summary • biological neurons • natural neural networks incl. the brain • artificial neural networks • applications JYC: CSM17

  23. Useful Websites Visible Human Project http://www.nlm.nih.gov/research/visible/ Stuttgart Neural Network Simulator (Unix) http://www-ra.informatik.uni-tuebingen.de/SNNS/ Microsoft’s List of Neural Network Websites http://research.microsoft.com/~jplatt/neural.html Neural Network FAQ ftp://ftp.sas.com/pub/neural/FAQ.html WebSOM http://websom.hut.fi/websom/ JYC: CSM17

  24. References & Bibliography • Greenfield, S. (1998). The human brain : a guided tour. - London : Phoenix, 1998. - 0753801558 • Greenfield, S. (2000)- Brain story. - London : BBC, 2000. - 0563551089 • Haykin, S. (1999). Neural networks : a comprehensive foundation , 2nd ed. – Prentice Hall, Upper Saddle River, N.J., USA. 0139083855, 0132733501 • Dayhoff, Judith E. (1990). Neural network architectures : an introduction. Van Nostrand Reinhold, New York. 0442207441 • Beale, R., Russell & Jackson, T. (1990). Neural computing : an introduction. Hilger, Bristol, UK. 0852742622 • Looney, C.G. (1997). Pattern recognition using neural networks. Oxford University Press, New York, USA. 0195079205 • Aleksander, I, & Morton, H. (1990). An introduction to neural computing. Chapman and Hall, London. - 0412377802 JYC: CSM17

More Related