Seminar on Machine Learning Rada Mihalcea

1. Seminar on Machine LearningRada Mihalcea Introduction to Machine Learning Administrivia January 13(!), 2004

2. Machine Learning? Definition: A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience. • Machine Learning has to do with designing computer programs that improve their performance through experience

3. Disciplines relevant to ML • Artificial intelligence • Control theory • Information theory • Computational complexity theory • Psychology and neurobiology • Statistics

4. Applications of ML • Learning to recognize spoken words • SPHINX (Lee 1989) • Learning to drive an autonomous vehicle • ALVINN (Pomerleau 1989) • Learning to classify celestial objects • (Fayyad et al 1995) • Learning to play world-class backgammon • TD-GAMMON (Tesauro 1992) • Learning to translate between languages • Learning to classify texts into categories • Web directories

5. Main directions in ML • Data mining • Finding patterns in data • Use “historical” data to make a decision • Predict weather based on current conditions • Self customization • Automatic feedback integration • Adapt to user “behaviour” • Recommending systems • Writing applications that cannot be programmed by hand • In particular because they involve huge amounts of data • Speech recognition • Hand writing recognition • Text understanding

6. Learning Problem (an example) Learning: improving with experience at some task • Improve over task T • With respect to performance measure P • Based on experience E Example: Learn to play checkers: • T: play checkers • P: percentage of games won in a tournament • E: opportunity to play against itself

7. Learning to play checkers • T: play checkers • P: percentage of games won • What experience? • What exactly should be learned? • How shall it be represented? • What specific algorithm to learn it?

8. Type of Training Experience • Direct or indirect? • Direct: board state  correct move • Indirect: outcome of a complete game • Credit assignment problem • Teacher or not ? • Teacher selects board states • Learner can select board states • Is training experience representative of performance goal? • Training playing against itself • Performance evaluated playing against world champion

9. Choose Target Function • ChooseMove : B  M : board state  move • Maps a legal board state to a legal move • Evaluate : BV : board state  board value • Assigns a numerical score to any given board state, such that better board states obtain a higher score • Select the best move by evaluating all successor states of legal moves and pick the one with the maximal score

10. Possible Definition of Target Function • If b is a final board state that is won then V(b) = 100 • If b is a final board state that is lost then V(b) = -100 • If b is a final board state that is drawn then V(b)=0 • If b is not a final board state, then V(b)=V(b’), where b’ is the best final board state that can be achieved starting from b and playing optimally until the end of the game. • Gives correct values but is not operational

11. m3 : bb3 m2 : bb2 m1 : bb1 State Space Search V(b)= ? V(b)= maxi V(bi)

12. State Space Search V(b1)= ? V(b1)= mini V(bi) m6 : bb6 m5 : bb5 m4 : bb4

13. Final Board States Black wins: V(b)=-100 Blue wins: V(b)=100 draw: V(b)=0

14. 4 x 4 checkers: Number of Board States #board states < 8!*22/(2!*2!*4!) = 1680 Regular checkers (8x8 board, 8 pieces each) #board states < 32!*216/(8! * 8! * 16!)= 5.07*1017

15. Choose Representation of Target Function • Table look-up • Collection of rules • Neural networks • Trade-off in choosing an expressive representation: • Approximation accuracy • Number of training examples to learn the target function

16. Obtaining Training Examples • V(b) : true target function • V’(b) : learned target function • Vtrain(b) : training value • Rule for estimating training values: • Vtrain(b)  V’(Successor(b))

17. Main steps in designing a learning algorithm • Determine training experience: • Historical data • Games against itself / experts • Determine target function • Board move / value • Determine representation of learned function • Linear combination of features • Determine learning algorithm • ANN • Finish design • As with any software product: • Implement • Test • Evaluate performance

18. Why seminar? • A lot of open discussions • Basics in ML, and also state-of-the-art in applying ML to real world applications • Focus (and heavy grade weight) on active participation!

19. General structure for a seminar • Topic announced ahead of time • Check regularly the seminar page • http://www.cs.unt.edu/~rada/CSCE5330 • Presentation of basic material (Rada) • One, two, or more papers presented by you (a different speaker will be assigned each time), and discussed by the entire class • “Reviews” for each of the papers discussed are due before the seminar.

20. Paper discussions • Everybody has to read the papers before the discussion, and fill in a review form • Will be available from seminar webpage • Discussion leader: • Read the paper carefully; read an addition of 1-2 references • Prepare a presentation including: • Research goal • Methods applied • Research results • Comparison with other methods • Deficiencies of the paper • Directions for future research • Formulate questions and issues for class discussions

21. Paper discussions (cont’d) • Discussion “scribe” • Take notes about the discussions made in class • Write a 2-3 page report on that, points made during the presentation, issues raised, questions, etc. • After seminar, discuss with the “discussion leader” to agree upon the content of the report • Submit report to the instructor within one day

22. Conference in ML • The seminar will “culminate” with a class conference • Term project • Topics to be discussed by next time, and decided in two weeks from now • Paper to be reviewed by your fellow colleagues • Conference (with presentations, discussions, etc) to be held during the last class meeting

23. Administrivia • Course requirements: • Active participation • Homework assignments • 3 days late policy • Term project • Seminar webpage • http://www.cs.unt.edu/~rada/CSCE5330

24. Administrivia • Textbook • Machine Learning, by T. Mitchell • Recommended: Data Mining with Java (Weka), Witten & Frank • Software, data: • Weka package • Installed on CSP machines • Various data sets

25. Grades • Assignments: 40% • Project: 40% • Class participation: 20%

26. Course Overview • Introduction to machine learning • Concept learning • Decision tree learning • Rule based learning • Evaluating hypotheses • Instance based learning • Neural networks • Bayesian learning • Hidden Marckov Models • Reinforcement learning • Boosting, bagging • Co-training, Self-training, Counter-training

27. Possible Course Projects • Apply machine learning techniques to your own problem e.g. classification, clustering, data modeling, object recognition • Investigate an ML research problem • Other • For next week: • Think of possible topics for a project. • I will propose several topics Need to decide upon topic in four-six weeks from today.

28. Next time • Concept learning • More on how to make a good presentation • Getting ready for the paper discussions • More on how to make a paper review