1 / 45

Lecture 9 Chapter 6 – Artificial Neural Networks for Data Mining

Business Intelligence. Lecture 9 Chapter 6 – Artificial Neural Networks for Data Mining. Learning Objectives. Understand the concept and definitions of artificial neural networks (ANN) Know the similarities and differences between biological and artificial neural networks

zuwena
Download Presentation

Lecture 9 Chapter 6 – Artificial Neural Networks for Data Mining

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Business Intelligence Lecture 9 Chapter 6 – Artificial Neural Networks for Data Mining Business Intelligence

  2. Learning Objectives • Understand the concept and definitions of artificial neural networks (ANN) • Know the similarities and differences between biological and artificial neural networks • Learn the different types of neural network architectures • Learn the advantages and limitations of ANN • Understand how backpropagation learning works in feedforward neural networks Business Intelligence

  3. Learning Objectives • Understand the step-by-step process of how to use neural networks • Appreciate the wide variety of applications of neural networks; solving problem types of • Classification • Regression • Clustering • Association • Optimization Business Intelligence

  4. Opening Vignette:(Page 242) “Predicting Gambling Referenda with Neural Networks” • Using NeuroSolutions, this study developed and tested models to predict community support for commercial gaming. • The study examined the role of factors that contribute to legalization and/or probation of gambling activities using neural networks. • It attempted to use Neural Network technology to predict various counties voting outcome on this subject. Business Intelligence

  5. Opening Vignette: • On average, the models accurately predicted the voting results for 4 out of every 5 counties (approximately 82% accuracy) on a sample data set. (1287 records of data) • Interestingly, and contrary to popular belief, the counties financial characteristics and age distribution were not found to be significant factors in determining ballot outcome. Dominant factors are identified on Page 244 • The study demonstrates that demographic data can be used to accurately predict voting outcomes on controversial issues. Business Intelligence

  6. Opening Vignette:Predicting Gambling Referenda… Business Intelligence

  7. Opening Vignette: • NeuroSolutions is offered by NeuroDimension and offers algorithms for the in the field of artificial intelligence. • NeuroDimension offers NeuroSolutions, NeuroSolutions for Excel, and a Custom Solution Wizard each of which can be downloaded for a free eval. Business Intelligence

  8. Opening Vignette: • An very good video is offered by the company that explains Neural Network algorithms and the field in general. • Pricing is relatively reasonable for the product. - NS for Excel costs $295 Business Intelligence

  9. Neural Network Concepts(Page 245) • Neural networks (NN): a brain metaphor for information processing– uses artificial neurons (programming constructs that mimic the properties of biological neurons). • Neural computing - pattern recognition methodology for machine learning • Artificial neural network (ANN) – resulting model from neural computing • Many uses for ANN for • pattern recognition, forecasting, prediction, and classification • finance, marketing, manufacturing, operations, information systems, and so on Business Intelligence

  10. ANN Video • Here is an excellent video offered by NeuroSolutions that provides a good overview of ANN Business Intelligence

  11. Biological Neural Networks(Page 246) • Two interconnected brain cells (neurons) • An axon is a long, slender projection of a nerve cell, or neuron, that conducts electrical impulses away from the neuron's cell body or soma. • Dendrites are branched filaments in nerve cells (neurons). The word dendrite derives from the Greek word for tree which describes their branching tree-like structure. Business Intelligence

  12. Biological Neural Networks(Page 246) • Synapse – able to increase or decrease the strength of the connection between neurons and cause excitation or inhibition of a subsequent neuron. • The word "soma" comes from the the Greek word “body”; the soma of a neuron is often called the cell body. Business Intelligence

  13. Processing Information in ANN(Page 247 Figure 6.3) • A single neuron (processing element – PE) with inputs and outputs Business Intelligence

  14. Biology Analogy(Page 247) Business Intelligence

  15. Elements of ANN(Page 248-250) • Processing element (PE) – organized in different ways to form the networks structure. • Network architecture • Hidden layers - takes input from the previous layer and converts into outputs for more processing (used in complex problems) • Parallel processing – resembles the way the brain works – different than serial processing in conventional computing Not this ANN Business Intelligence

  16. Elements of ANN(Page 248-250) • Network information processing • Inputs – single attribute such as age, income level, etc • Outputs – solution to the problem – ie – loan app “yes” or “no” • Connection weights – relative strength of input data (how important) • Summation function – weighted sums of all input elements entering a PE. Business Intelligence

  17. Elements of ANN(Figure 6.4 Page 249) Neural Network with One Hidden Layer Business Intelligence

  18. Elements of ANN Summation Function for a Single Neuron (a) and Several Neurons (b) Business Intelligence

  19. Elements of ANN(Page 251) • Transformation (Transfer) Function – activation level of a neuron (based on this level the neuron may or may not produce an output). • Computed via Sigmoid (logical activation) function – YT=1/(1+e-Y) • Y is computed via weighted summation • Any value less than threshold will not be passed to output (0); anything above does (1) • Threshold value Business Intelligence

  20. Neural Network Architectures(Page 251-252) • Several ANN architectures exist • Feedforward - figure 6.4 page 249 (see previous slide) • Recurrent - - figure 6.7 page 252 (next slide) • Associative memory • Self-organizing feature maps • Hopfield networks, etc Business Intelligence

  21. Neural Network ArchitecturesRecurrent Neural Networks(Page 252, figure 6.7) Business Intelligence

  22. Neural Network Architectures(Page 252) • Architecture of a neural network is driven by the task it is intended to address • Most popular architecture: Feedforward, multi-layered perceptron with backpropagation learning algorithm • Ie – Feedforward Perceptron is the architecture and backpropagation is the learning algorithm. Business Intelligence

  23. Neural Network Architectures • The perceptron is a type of artificial neural network invented in 1957 at the Cornell Aeronautical Laboratory by Frank Rosenblatt. Frank Rosenblatt was a computer scientist born in 1928 in New York City.  He helped to create the Perceptron, a.k.a. the Mark 1, computer in 1960 at Cornell University.  This was the first computer that could learn skills by trial and error in an attempt to mimic human thought processes through the use of a neural network.  (Died 1971) • Backpropagation is a common, supervised method for teaching artificial neural networks how to perform a given task. It was first described by Arthur E. Bryson and Yu-Chi Ho in 1969. Frank Rosenblatt (1957) Business Intelligence

  24. Neural Network Architectures Original Mark 1 (Automatic Sequence Controlled Calculator (ASCC) The building elements of the ASCC were switches, relays, rotating shafts, and clutches. Business Intelligence

  25. Learning in ANN(Page 252) • A process by which a neural network learns the underlying relationship between input and outputs, or just among the inputs • Supervised learning • For prediction type problems • E.g., backpropagation • Unsupervised learning • For clustering type problems • Self-organizing • E.g., adaptive resonance theory Business Intelligence

  26. A Taxonomy of ANN Learning Algorithms(Page 253, Figure 6.8) Most popular Business Intelligence

  27. Read Application Case(Page 254) • Microsoft used BrainMaker Neural Network software from California Scientific to maximize return on direct Mail • Some of the variables considered (25 in total) • Recency (how long since last registration / product purchase) • First date to file – loyal over time? • Number of products bought and filed • Value of products bought and registered • Number of days from product release to purchase • Improved response rate from 4.9% to 8.2% - 35% cost savings on 40 Millions pieces of direct mailings Business Intelligence

  28. A Supervised Learning Process(Pages 255-256, figure 6.9) Three-step process: 1. Compute temporary outputs 2. Compare outputs with desired targets 3. Adjust the weights and repeat the process Business Intelligence

  29. How a Network Learns(Page 256) • Example: single neuron that learns the inclusive OR operation • Learning parameters: • Learning rate • Momentum * See page 257 for step-by-step progression of the learning process Business Intelligence

  30. Backpropagation Learning(Page 258) • Errors are used to correct weights – called Back-error propagation • The (supervised) learning algorithm procedure: • Initialize weights with random values and set other network parameters • Read in the inputs and the desired outputs • Compute the actual output (by working forward through the layers) • Compute the error (difference between the actual and desired output) • Change the weights by working backward through the hidden layers • Repeat steps 2-5 until weights stabilize Business Intelligence

  31. Backpropagation Learning(Figure 6.10 Page 258) • Backpropagation of Error for a Single Neuron Business Intelligence

  32. Development Process of an ANN(Page 259) Similar to structured design for traditional IS, with some new elements See page 253 Business Intelligence

  33. An MLP ANN Structure for the Box-Office Prediction Problem (Page 262, Fig 6.12)This is the vignette at the start of Chapter 5 of page 191 Business Intelligence

  34. Data Collection and Testing(Page 261) • Data is split into three parts • Training (~60%) • Validation (~20%) • Testing (~20%) Business Intelligence

  35. Sensitivity Analysis on ANN Models(Page 264-265) • A common criticism for ANN: The black-box syndrome! • Answer: sensitivity analysis • Conducted on a trained ANN • The inputs are changed while the relative change on the output is measured/recorded • Results illustrates the relative importance of input variables Business Intelligence

  36. Sensitivity Analysis on ANN Models(Page 265, Figure 6.13) • See and read example Application Case 6.5 (Page 266) • Sensitivity analysis reveals the most important injury severity factors in traffic accidents Business Intelligence

  37. Sensitivity Analysis on ANN Models(Page 266) • Application Case 6.5 – see here • 41,000 die in 6M US traffic accidents • Analyze the factors that elevate the risk of severe injury • Factors include behavior, environment, technical, etc. • Used series of ANN models to estimate the significance of the crash factors on the level of severity sustained by the driver. • Two step process used (1) prediction models, (2) sensitivity analysis on trained neural network • Results shows significant differences among models built for different injury severity levels. (The most influential factors HIGHLY depend on the level of injury). Business Intelligence

  38. A Sample Neural Network ProjectBankruptcy Prediction(Pg 267-270) • A comparative analysis of ANN versus logistic regression (LR) (a statistical method) • Inputs • X1: Working capital/total assets • X2: Retained earnings/total assets • X3: Earnings before interest and taxes/total assets • X4: Market value of equity/total debt • X5: Sales/total assets Business Intelligence

  39. A Sample Neural Network ProjectBankruptcy Prediction • Data was obtained from Moody's Industrial Manuals • Time period: 1975 to 1982 • 129 firms (65 of which went bankrupt during the period and 64 nonbankrupt) • Different training and testing propositions are used/compared • 90/10 versus 80/20 versus 50/50 • Resampling is used to create 60 data sets Business Intelligence

  40. A Sample Neural Network ProjectBankruptcy Prediction • Network Specifics • Feedforward MLP • Backpropagation • Varying learning and momentum values • 5 input neurons (1 for each financial ratio), • 10 hidden neurons, • 2 output neurons (1 indicating a bankrupt firm and the other indicating a nonbankrupt firm) Business Intelligence

  41. A Sample Neural Network ProjectBankruptcy Prediction – Results(Page 269 figure 6.2) Business Intelligence

  42. Bottomline - Advantages of ANN(Pages 274-276) • Able to deal with (identify/model) highly nonlinear relationships • Can handle variety of problem types (loan apps, forecast profitability / finances, sports – team success, fraud prevention, time-series forecasting, health care and medicine – diagnose breast cancer – see Case 6.4 on page 276) • Usually provides better results (prediction and/or clustering) compared to its statistical counterparts Business Intelligence

  43. Disadvantages of ANN • They are deemed to be black-box solutions, lacking expandability • It is hard to find optimal values for large number of network parameters • Optimal design is still an art: requires expertise and extensive experimentation • It is hard to handle large number of variables (especially the rich nominal attributes) • Training may take a long time for large datasets; which may require case sampling Business Intelligence

  44. ANN Software(Page 263) • Standalone ANN software tool • NeuroSolutions • BrainMaker • NeuralWare • NeuroShell, … for more (see pcai.com) … • Part of a data mining software suit • PASW (formerly SPSS Clementine) • SAS Enterprise Miner • Statistica Data Miner, … many more … Business Intelligence

  45. Next lecture Chapter 7 - Text and Web Mining Business Intelligence

More Related