Week 5

1. Week 5 Artificial Neural Networks (ANNs)

2. Case Scenario ABC Enterprise, as a multinational company has invested in many country sectors for revenues generation. One of the potential revenue for the company comes from the investment in the BSKL shares. The CEO of the company, Mr. Ridzuan intends to invest in selected potential shares available in the current market. He is aware that such investment is a risky business. Unfortunately, he does not have experienced staff to advice on the potential shares investment. Thus, Mr. Ridzuan is thinking whether there is any system that can predict short and long term future of current shares trading in the market with accurately. The system must be able to analyze history of the current shares trading and predicts when the best time to buy the shares is.

3. Human Brain • 50 to 150 billion neurons in brain (estimation) • Neurons grouped into networks • Axons send outputs to cells • Received by dendrites, across synapses

4. Artificial Neural Networks (ANNs) • A model that emulates a biological neural network. • Software simulations of the massively parallel processes that involve processing elements interconnected in a network architecture. • Originally proposed as a model of the human brain’s activities. • The human brain is much more complex.

5. Processing Information in an Artificial Neuron Inputs Weights x1 w1j Output Yj Neuron j Σ wij xi x2 w2j ƒ Summations Transfer function xi wij

6. Processing • Processing elements are neurons • Allows for parallel processing • Each input is single attribute • Connection weight • Adjustable mathematical value of input • Summation function • Weighted sum of input elements • Internal stimulation • Transfer function • Relation between internal activation and output • Sigmoid/transfer function • Threshold value • Outputs are problem solution

7. Learning: Three Tasks The neurons are connected by links, and each link has a numerical weight associated with it. Weights are the basic means of long term memory in ANN. They express the strength or importance of each neuron input. ANN learns through repeated adjustments of these weights. In summary, learning in ANN involves three tasks: 1. Compute Outputs 2. Compare Outputs with Desired Targets 3. Adjust Weights and Repeat the Process

8. Learning Algorithms • Learning is a fundamental characteristic of ANNs. • Two Basic Learning Categories • Supervised Learning • Unsupervised Learning

9. Supervised Learning • For a set of inputs with known (desired) outputs. • Connection weights derived from known cases. • Useful in pattern recognition (character, voice, object etc.) • Examples • Backpropagation network • Hopfield network

10. Supervised Learning : Character Recognition Demonstration of a neural network used within an optical character recognition (OCR) application. The original document is scanned into the computer and saved as an image. The OCR software breaks the image into sub-images, each containing a single character. The sub-images are then translated from an image format into a binary format, where each 0 and 1 represents an individual pixel of the sub-image. The binary data is then fed into a neural network that has been trained to make the association between the character image data and a numeric value that corresponds to the character. The output from the neural network is then translated into ASCII text and saved as a file.

11. Unsupervised Learning • Only input stimuli shown to the network. • Humans assign meanings and determine usefulness. • Useful in clustering (objects) and knowledge discovery. • Examples • Adaptive Resonance Theory (ART) • Kohonen Self-organizing Feature Maps

12. Unsupervised Learning : Data Classification Classification of raw data into two subgroups.

13. Development of Systems • Collect data • The more, the better • Separate data into training set to adjust weights • Divide into test sets for network validation • Select network topology • Determine input, output, and hidden nodes, and hidden layers • Select learning algorithm and connection weights • Iterative training until network achieves preset error level • Black box testing to verify inputs produce appropriate outputs • Contains routine and problematic cases

14. Neural Network Software • Program in: • Programming language (C++, Java, VB) • Neural network package or NN programming tool • Tools (shells) incorporate: (MATLAB) • Training algorithms • Transfer and summation functions • May still need to: • Program the layout of the database • Partition the data (test data, training data) • Transfer the data to files suitable for input to an ANN tool

15. Advantages of ANNs • Pattern recognition, classification, generalization, interpretation of incomplete and noisy inputs. • Character, speech and visual recognition. • Can tackle new kinds of problems. • Robust, flexible and easy to maintain. • Powerful hybrid systems.

16. Limitations of ANNs • Do not do well at tasks that are not done well by people • Lack explanation capabilities • Limitations and expense of hardware technology restrict most applications to software simulations • Training time can be excessive and tedious • Usually requires large amounts of training and test data

17. ANN Examples • NeuroXL Classifier (add-in for Ms Excell) • http://www.neuroxl.com/index.htm • N-OCR (character recognition) • http://www10.brinkster.com/geniusportal/neural/nocr.html • Neural Network Toolbox (MathWorks) • http://www.mathworks.com/products/neuralnet/

18. Intelligence Density Dimension • Accuracy • Flexibility • Embeddedability • Independence from experts