Machine Learning: Introduction

1. Machine Learning: Introduction Villanova University Machine Learning Project

2. What is learning? • A broad general definition from psychology: • the process by which a relatively lasting change in behavior or potential behavior occurs as a result of practice or experience • This suggests • there is some entity which is exhibiting behavior • it can somehow perceive and process experiences • its behavior can change • Normally we also assume that the change is somehow an improvement “Learning is making useful changes in our minds.” –Marvin • Minsky Villanova University Machine Learning Project Machine Learning Introduction

3. What Is Machine Learning? • “Learning denotes changes in a system that ... enable a system to do the same task more efficiently the next time.” –Herbert Simon • Similar to psychology definition, but more focused on a good result • “Learning is constructing or modifying representations of what is being experienced.” –Ryszard Michalski • The experiences perceived must be captured or represented in some way; learning modifies that representation. This definition focuses on the process, rather than the result. Villanova University Machine Learning Project Machine Learning Introduction

4. So What is Learning? • For our purposes: • changes in a system that modify its behavior or outputs • based on some experiences or inputs • in a way that moves those behaviors closer to some desired result • relatively permanently Villanova University Machine Learning Project Machine Learning Introduction

5. What Does That Mean?? • In machine learning, the “system” is a computer and its programs; however, this could apply to biological systems as well. • Another way of looking at machine learning is as a way to get a computer to do things without having to explicitly describe what steps to take, by giving it examples or feedback • The computer then looks for patterns which can explain or predict what happens. • The computer is trained through the examples. Villanova University Machine Learning Project Machine Learning Introduction

6. Is This Machine Learning? • You set your preferred language to Spanish in Google. • Amazon recommends books for you based on what you have bought in the past. • Your credit card is approved at lunch. • You tell your GPS to avoid a corner because the radio says there has been a crash. Villanova University Machine Learning Project Machine Learning Introduction

7. Example: Animals Guessing Game • Animals is a simple machine learning program which plays a guessing game. • The machine gradually learns a set of questions and answers • Some of these questions lead to more questions, some to a guess. Villanova University Machine Learning Project Machine Learning Introduction

8. Animals Behind the Scene Computer: Think of an animal and I will guess it. Ready? Human: yes Computer: Is it a frog? Is it a frog? Here the computer has a very simple representation, consisting of a single guess. Villanova University Machine Learning Project 7 Machine Learning Introduction

9. Is it a penguin? Is it a bird? Is it a frog? Animals Behind the Scene Computer: Is it a frog? Human: No Computer: What is it? Human: A penguin Computer: Tell me a question that distinguishes a frog from a penguin. Human: Is it a bird? Computer: What is the answer for a frog? Human: no No Yes Here the computer has been expanded by learning a second guess, a question, and the yes-no choices for each guess. Villanova University Machine Learning Project Machine Learning Introduction 8

10. Is it a frog? Is it a bird? Is it a mouse? Does it have fur? Is it a penguin? Animals Behind the Scene Computer: Is it a bird? Human: No Computer: Is it a frog? Human: No Computer: What is it? Human: A mouse Computer: Tell me a question that distinguishes a frog from a mouse. Human: Does it have fur? Computer: What is the answer for a frog? Human: no Yes No Yes No Another question, another guess, and two more choices. Villanova University Machine Learning Project Machine Learning Introduction 9

11. The Architecture of a Machine Learning System • Every machine learning system has four parts: • a representation or model of what is being learned • an actor. This is the part that uses the representation and actually does something. • a critic. The part that provides feedback. • a learner. The part that modifies the representation or model, using the feedback. Villanova University Machine Learning Project Machine Learning Introduction

12. Animals Guessing Game Architecture • The Animals Guessing Game has all of the parts of a Machine Learning Architecture: • The Representation is a sequence of questions and pairs of yes/no answers (called a binary decision tree). • The Actor “walks” the tree, interacting with a human; at each question it chooses whether to follow the “yes” branch or the “no” branch. • The Critic is the human player telling the game whether it has guessed correctly. • The Learner elicits new questions and adds questions, guesses and branches to the tree. Villanova University Machine Learning Project Machine Learning Introduction

13. Examples: • This would be a good place to get some examples from our advisory board or other domains. Villanova University Machine Learning Project Machine Learning Introduction 11

14. To Review: • We have looked at the four components of a machine learning system for several examples: • Representation • Actor • Critic • Learner • In the following slides we examine each component in more detail. Villanova University Machine Learning Project Machine Learning Introduction

15. Representation • A learning system must have a representation or model of what is being learned. • This is the component that changes based on experience. • In a machine learning system this may be a mathematical model or formula, a set of rules, a decision tree, or some other form of information. Villanova University Machine Learning Project Machine Learning Introduction

16. Representing The Problem • Representing the problem to be solved is the first decision to be made in any machine learning application • It’s also the most important. • And the one that depends most on knowing the domain -- the field in which the problem is set. • There are two aspects of representing a problem: the behavior that we want to learn, and the inputs we will learn it from. Villanova University Machine Learning Project Machine Learning Introduction

17. Representation: Examples • How do we describe our problem? • Guessing an animal: a tree of questions and answers • Playing checkers: the board and piece positions, sets of rules for choosing moves • Deciding whether an email is spam: the frequencies of words used in this email and in our entire mailbox. • Using an optical character recognition (OCR) system to decode the amount on a check; 6x10 matrix of light/dark pixels; % light pixels; # straight lines, # curved lines Villanova University Machine Learning Project Machine Learning Introduction

18. Actor • We are building a machine learning system because we want to do something. • make a prediction • sort into categories • look for similarities • The actor is the part of the system that actually does things. • Once a system has learned, or been trained, this is the component we continue to use. • It may be as simple as a formula to be applied, or it may be a complex program Villanova University Machine Learning Project Machine Learning Introduction

19. Actor • How do we take action? • Guessing an animal: walk the tree and ask associated questions • Playing checkers: look through the rules to identify a move; choose one; make it. • Identifying spam: examine the set of features (word frequencies) to determine whether this is spam • OCRing a check amount: input the features for a digit, output probability for each of 0 through 9. Villanova University Machine Learning Project Machine Learning Introduction

20. Critic • This component provides the experience we learn from. • Typically, it is a set of examples with the decision that should be reached or action that should be taken • But may be any kind of feedback that gives an indication of how close we are to where we want to be. Villanova University Machine Learning Project Machine Learning Introduction

21. Critic • How do we judge correct actions? • Guessing an animal: human feedback • OCRing digits: Human-categorized training set. • Identifying spam: match to a set of human-categorized test documents. • Playing checkers: who won? • Grouping documents: which are most similar in language or content? • Can be generally categorized as supervised, unsupervised, reinforcement. Villanova University Machine Learning Project Machine Learning Introduction

22. Learner • The learner is the core of a machine learning system. It will • examine the information provided by the critic • use it to modify the representation to move toward a more desirable action the next time. • repeat until the performance is satisfactory, or until it stops improving • Normally, this component will be an existing tool which we can use. • Generally, we will focus on which tools to use, what inputs they need, and what the results mean, rather than on how to implement them. Villanova University Machine Learning Project Machine Learning Introduction

23. Learner • What does the learner do? • Guessing an animal: ask the user for a question and add it to the binary tree • OCRing digits: modify the importance of different input features. • Identifying spam: change the set of words likely to be in spam. • Playing checkers: increase the chance of using some rules and decrease the chance for others. • Grouping documents: find clusters of similar documents Villanova University Machine Learning Project Machine Learning Introduction

24. Why Look at Learning?? • Understand and improve efficiency of human learning • Improve methods for teaching and tutoring people • Discover new things or structures that were previously unknown to humans • Example: discover relationships between DNA and diseases • Build software tools that can adapt to their users • Example: learn books we might like based on what we have bought. • Reproduce an important aspect of intelligent behavior • One goal of artificial intelligence research is to reproduce intelligent behavior; learning is an essential component of it. Villanova University Machine Learning Project Machine Learning Introduction