Data Mining and Neural Networks

1. Data Mining and Neural Networks Danny Leung CS157B, Spring 2006 Professor Sin-Min Lee

2. 2 Artificial Intelligence for Data Mining Neural networks are useful for data mining and decision-support applications. People are good at generalizing from experience. Computers excel at following explicit instructions over and over. Neural networks bridge this gap by modeling, on a computer, the neural behavior of human brains.

3. 3 Neural Network Characteristics Neural networks are useful for pattern recognition or data classification, through a learning process. Neural networks simulate biological systems, where learning involves adjustments to the synaptic connections between neurons

4. 4 Anatomy of a Neural Network Neural Networks map a set of input-nodes to a set of output-nodes Number of inputs/outputs is variable The Network itself is composed of an arbitrary number of nodes with an arbitrary topology

5. 5 Biological Background A neuron: many-inputs / one-output unit Output can be excited or not excited Incoming signals from other neurons determine if the neuron shall excite ("fire") Output subject to attenuation in the synapses, which are junction parts of the neuron

6. 6 Basics of a Node A node is an element which performs a function y = fH(?(wixi) + Wb)

7. 7 A Simple Preceptron Binary logic application fH(x) [linear threshold] Wi = random(-1,1) Y = u(W0X0 + W1X1 + Wb)

8. 8 Preceptron Training It�s a single-unit network Adjust weights based on a how well the current weights match an objective Perceptron Learning Rule ? Wi = ? * (D-Y).Ii ? = Learning Rate D = Desired Output

9. 9 Neural Network Learning From experience: examples / training data Strength of connection between the neurons is stored as a weight-value for the specific connection Learning the solution to a problem = changing the connection weights

10. 10 Neural Network Learning Continuous Learning Process Evaluate output Adapt weights Take new inputs Learning causes stable state of the weights

11. 11 Learning Performance Supervised Need to be trained ahead of time with lots of data Unsupervised networks adapt to the input Applications in Clustering and reducing dimensionality Learning may be very slow No help from the outside No training data, no information available on the desired output Learning by doing Used to pick out structure in the input: Clustering Compression

12. 12 Topologies � Back-Propogated Networks Inputs are put through a �Hidden Layer� before the output layer All nodes connected between layers

13. 13 BP Network � Supervised Training Desired output of the training examples Error = difference between actual & desired output Change weight relative to error size Calculate output layer error , then propagate back to previous layer Hidden weights updated Improved performance

14. 14 Neural Network Topology Characteristics Set of inputs Set of hidden nodes Set of outputs Increasing nodes makes network more difficult to train

15. 15 Applications of Neural Networks Prediction � weather, stocks, disease Classification � financial risk assessment, image processing Data association � Text Recognition (OCR) Data conceptualization � Customer purchasing habits Filtering � Normalizing telephone signals (static)

16. 16 Overview Advantages Adapt to unknown situations Robustness: fault tolerance due to network redundancy Autonomous learning and generalization Disadvantages Not exact Large complexity of the network structure

17. 17 Referenced Work Intro to Neural Networks - Computer Vision Applications and Training Techniques. Doug Gray. www.soe.ucsc.edu/~taoswap/ GroupMeeting/NN_Doug_2004_12_1.ppt Introduction to Artificial Neural Networks. Nicolas Galoppo von Borries. www.cs.unc.edu/~nico/courses/ comp290-58/nn-presentation/ann-intro.ppt