1 / 27

LEARNING FROM DATA

LEARNING FROM DATA. Lecture Ten (Chapter 10, Notes; Chapter 11, Textbook). Outline. The Concept of Learning Data Visualization Neural Networks The Basics Supervised and Unsupervised Learning Business Applications Association Rules Implications for Knowledge Management.

lauren
Download Presentation

LEARNING FROM DATA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. LEARNING FROM DATA Lecture Ten (Chapter 10, Notes; Chapter 11, Textbook)

  2. Outline • The Concept of Learning • Data Visualization • Neural Networks The Basics Supervised and Unsupervised Learning Business Applications • Association Rules • Implications for Knowledge Management

  3. The “Context” of Learning • The “value added”collaborative intelligence layer of KM architecture • Relevant technologies are: • Artificial Intelligence • Experts Systems • Case-Based Reasoning • Data Warehousing • Intelligent agents • Neural Networks

  4. The “Process” of Learning • A process of filtering and transforming data into valid and useful knowledge. • Automate via technology tools: • provide a collaborative learning environment for participants • enhance their ability to understand the processes / tasks they are dealing with

  5. The “Goals” of Learning • Final goal is to improve the qualities of communication and decision making • Ways to achieve these goals: • Verify hypotheses (formed from accumulated knowledge) • Discover new patterns in data • Predict future trends and behaviour

  6. Learning from Data • Build learning models that automatically improve with experience. • Top-down approach • Generate ideas • Develop models • Validate models • Bottom-up approach • Discover new (unknown) patterns • Find key relationships in data

  7. Top-down approach (Example) • Start with a hypothesis derived from observation or prior knowledge • “Tourists visiting Egypt earn an annual income of at least $50,000” • Hypothesis tested by querying database followed by analysis • If tests not supportive, hypothesis is revised and test again

  8. Bottom-up approach (Example) • No hypothesis to test • “Find unknown buying patterns by analyzing the shopping basket” • “… showed married males, age 21 to 27, shopped for diapers also brought beer. • “store decided to stack beer cases next to diaper shelf”

  9. Data Visualization • Explore visually for trends in data useful for making decision • Exploring data includes: • Identify key attributes and their distribution • Identify outliers • Extract interesting grouping of data subsets • Identify initial hypothesis

  10. Example of Data Visualization(John Snow and the Cholera outbreak in London, 1845)

  11. Artificial Neural Network as Learning Model • Modeled after human brain’s network • Simulate biological information processing via networks of interconnected neurons • Neural networks are analog and parallel

  12. Neurons – The Basic Elements • The neuron receives inputs, determines their weights (strengths), sums the combined inputs, and compares the total to a threshold(transfer function) • If total is greater than threshold, the neuron fires (sends an output)

  13. A Neuron Model

  14. Learning in Neural Network • Supervised • The NN needs a teacher with a training set of examples of input and output • Unsupervised (or Self-Supervised) • Does not need a teacher

  15. Supervised Learning • Each element in a training set is paired with an acceptable response • Network makes successive passes through the examples • The weights adjust toward the goal state.

  16. A Supervised Neural Network (An Example)

  17. Unsupervised Learning • No external factors can influence adjustment of input’s weights • No advanced indication of correct or incorrect answers • Adjusts through direct confrontation with new experiences

  18. Business Applications (1) • Risk management • Appraise commercial loan applications • NN trained on thousands of applications, half of which were approved and the other half rejected by the bank’s loan officers • Through supervised learning, NN learned to pick risks that constitute a bad loan • Identifies loan applicants who are likely to default on their payments

  19. Business Applications (2) • Predicting Foreign Exchange Fluctuations: • A set of relevant indicators were identified, used as inputs to NN • NN was trained for exchange rates of US dollar against Swiss franc and Japanese yen, using data from first 6 months of 1990. Then it was tested over an 8- to 11-week period • Results revealed return on capital of about 20%

  20. Business Applications (3) • Mortgage Appraisals: • Neural network uses the data in the mortgage loan application • It estimates value of the property based on the immediate neighborhood, the city, and the country • The system comes up with a valuation for the property and a risk analysis for the loan.

  21. Association Rules • A KB tool that generates a set of rules to help understanding relationships that exist in data • Types: • Boolean rule • Quantitative rule • Multi-dimensional rule • Multi-level association rule

  22. Boolean Rule (An Example) • A rule that examines the presence or absence of items • For example, if a customer buys a PC and a 17” monitor, then he will buy a printer. Presence of items (a PC and 17” monitor) implies presence of the printer in the customer’s buying list

  23. Quantitative Rule (An Example) • A rule that considers the quantitative values of items • For example, if a customer earns between $30,000 and $50,000 and owns an apartment worth between $250,000 and $500,000, he will buy a 4-door automobile

  24. Multi-dimensional Rule • A rule that refers to a multitude of dimensions • If a customer lives in a big city and earns more than $35,000, then he will buy a cellular phone • This rule involves 3 attributes: living, earning, and buying. Therefore, it is a multi-dimensional rule

  25. Multi-level Association Rule

  26. Implications for Knowledge Management • Cost / benefit analysis • Tangible costs - user training, hardware + software, backup, support, maintenance • Intangible costs - user resistance and learning curve • Quality Assurance • Adequacy of initial design • Level and frequency of maintenance

  27. . . . . . Layers of KM Architecture User Interface (Web browser software installed on each user’s PC) 1 2 3 4 5 6 7 Authorized access control (e.g., security, passwords, firewalls, authentication) Collaborative intelligence and filtering (intelligent agents, network mining, customization, personalization) Knowledge-enabling applications (customized applications, skills directories, videoconferencing, decision support systems, group decision support systems tools) Transport (e-mail, Internet/Web site, TCP/IP protocol to manage traffic flow) Middleware (specialized software for network management, security, etc.) The Physical Layer (repositories, cables) Data warehousing (data cleansing, data mining) Groupware (document exchange, collaboration) Legacy applications (e.g., payroll) Databases

More Related