机器学习

1. 机器学习 陈昱 北京大学计算机科学技术研究所 信息安全工程研究中心

2. 课程基本信息 • 主讲教师：陈昱 chenyu@icst.pku.edu.cn Tel：82529680 • 助教：程再兴，Tel：62763742 wataloo@hotmail.com • 课程网页：http://www.icst.pku.edu.cn/node/content_102.htm

3. Ch1 Introduction • What is machine learning (ML)? • Design a learning system: an example • ML applications • Miscellaneous issues

4. Ch1 Introduction • What is machine learning (ML)? • Design a learning system: an example • ML applications • Miscellaneous issues

6. Related Problem: Data Deluge

7. Contd too much information

8. A Brief History of Machine Learning • ML as a scientific discipline was born in mid-seventies of last century. • The first ML workshop was held in 1980 at CMU, with some two dozen participants and photocopied preprints. • The first ML publication “Machine Learning” started in 1986.

9. Some Early Seminal Works • Perceptron model proposed by Rosenblatt (1958), so-called “connectionist” approach， a seminal work in neural work. • A system that learns to play checkers (Samuel, 1959 & 1963) • META-DENTRAL program, an example of symbolic learning. (Buchanan, 1978) • Learns rules for predicting how molecules would break into pieces in a mass spectrometer. It was able to generate rules novel enough for a publication in a journal of analytical chemistry.

10. What is Machine Learning? • The central problem it studies: How can we build computer systems that automatically improve with experience, and what are the laws that govern all learning processes? • We state a learning problem as: a machine learns with respect to (w.r.t.) a particular task T, performance metric P, and type of experience E.

11. What is Machine Learning (2) • More precisely, a computer program is said to learn from experience E, w.r.t. to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.

12. Alternative Views • Machine learning as an attempt to automate parts of the scientific method (Wikipedia) • Scientific method refers to a body of techniques for investigating phenomena, acquiring new knowledge, or correcting and integrating previous knowledge. • Machine learning as computational approaches to learning

13. Example of Learning Problem • Handwriting Recognition Task T: recognizing and classifying handwritten words within images Performance measure P: percent of words correctly classified Training experience E: a database of handwritten words with given classification

14. Related Disciplines • Artificial Intelligence • Learning as a way of improving problem solving • Machine learning as a search problem • Computational complexity theory • Control Theory • Information Theory • Philosophy • Psychology and neurobiology • Power law of practice • Statistics • Bayesian statistics

15. Ch1 Introduction • What is machine learning (ML)? • Design a learning system: an example • ML applications • Miscellaneous issues

16. Design a Learning System • Consider the example of learning how to play checkers • T: playing checkers • P: percentage of games it wins in the world tournament • E?

17. starting position of a checkers game, from Wikipedia

18. a checkers board state, from http://www.5025488.net/bbs/thread-49430-1-1.html

19. Choose the Training Experience • Type of feedback provided by training examples (to improve P) • Direct: individual checkers board states plus the correct move for each state • Indirect: move sequences plus final outcome for each game • Need to assign each move a credit/punish for the final outcome

20. Choose the Training Experience • Type of feedback provided by training examples (to improve P) • Direct: individual checkers board states plus the correct move for each state • Indirect: move sequences plus final outcome for each game • Need to assign each move a credit/punish for the final outcome Easy for learning!

21. Choose the Training Experience (2) • How much the learner can control training examples? • Completely rely on a teacher to select board states and provide correct move for each state, • have complete control over board states and final game outcome (indirect feedback), as in the case of playing against itself, or • propose confusing board states to a teacher and ask for correct move.

22. Choose the Training Experience (3) • How well the training examples resemble to the cases in which the final performance P is measured? • Theoretical assumption vs. reality • Related topics: • Concept drifting • Incremental learning • Transfer learning

23. Chose the Training Experience (3) • How well the training examples resemble to the cases in which the final performance P is measured? • Theoretical assumption vs. reality • Related topics: • Concept drifting • Incremental learning • Transfer learning (currently a research hotspot!)

24. Update Summary • A checkers learning problem • T: playing checkers • P: the percent of games it wins in the world tournament • E:games played against itself

25. Remaining Issues • What knowledge to be learned? • How to represent this knowledge? • What algorithm used to learn the knowledge (learning mechanism)?

26. Remaining Issues • What knowledge to be learned? • How to represent this knowledge? • What algorithm used to learn the knowledge (learning mechanism)?

27. Choose the Target Function • Think of a checker learning program as an optimization problem: At every board state the program chooses the best move among all the legal moves. • Reformulate what to be learned as a function ChooseMove: B →M, or a better representation, V: B →R real number set

28. How to Define Target Function V? • If b is a final board state, then it is simple: • If b is won, V(b)=100 (or other big number) • If b is lost, V(b)=-100 • If b is draw, V(b)=0

29. How to Define Target Function V ?(2) • Otherwise, it is tough! We might define V(b)=V(b’), where b’ is the best final state that can be achieved starting from b and playing optimally until end of the program. • However, such definition is not operational!

30. Remaining Issues • What knowledge to be learned? • How to represent this knowledge? • What algorithm used to learn the knowledge (learning mechanism)?

31. Choose a Representation of V • A tradeoff between the expressiveness of V and demand for training data • Let us consider a simple representation Ṽ of V: a linear combination of following board states: • x1: #black pieces on the board • x2: #red pieces • x3: #black kings • x4: #red kings • x5: #black pieces threatened by red (i.e. which can be captured on red’s next move) • x6: #red pieces threatened by black (i.e. which can be captured on black’s next move)

32. A Simple Representation of V

33. Remaining Issues • What knowledge to be learned? • How to represent this knowledge? • What algorithm used to learn the knowledge (learning mechanism)?

34. Choose an Approximation Algorithm • Choose a set of training examples (b, Vtrain(b)) • Estimate Vtrain(b) • For some board state, it is obvious, e.g. Vtrain(b)=100 if x2=0 (assuming the learning program plays black). • Only indirect training examples are available. One common approach is via iteration, such as Vtrain(b) ←Ṽ(Successor(b)).

35. Adjust the Weights • A common approach to obtain the weights is by minimizing the sum of square of error

36. An Algorithm for Finding Weights • Least mean square (LMS) weight update rule: For each training example (b, Vtrain(b)) • Use the current weights to calculate Ṽ(b). • For each weight wi, update it as

37. Summary of the Whole Design Process

38. Issues in Machine Learning • What algorithms exist for learning general target functions from training examples? Convergence of algorithms given sufficient examples? Which algorithms work best for which kind of target functions? • How does number of examples influence accuracy of learned functions? How dose character of hypothesis space impact accuracy? • How can prior knowledge of learner help?

39. Issues in Machine Learning (2) • What specific functions should the learner attempt to learn? Can this process be automated? • How can the learner automatically alter its representation to improve its ability to represent and learn the target function?

40. Ch1 Introduction • What is machine learning (ML)? • Design a learning system: an example • ML applications • Miscellaneous issues

41. Data Mining • Object recognition Machine Learning* • Speech Recognition • Reinforcement learning • Predictive modeling • Pattern discovery • Hidden Markov models • Convex optimization • Explanation-based learning • .... • Automated Control learning • Extracting facts from text

42. Example: Self-Learning Robot iCub • iCub is a humanoid robot the size of a 3.5 year old child. It has been developing for 5-years under the project RobotCub, funded by European Commission for studying human cognition. • RobotCub is an open source project.

43. Some New Frontiers • Mining real-time data that record personal activities, conversations, movements… • Applications: personal health monitoring, traffic guidance… • Industry trends: Twitter, location-based service …become popular • Modeling surprise • By Eric Horvitz of Microsoft Research

44. Modeling Surprise

45. Application Successes • Speech recognition • Two training stages: speaker-independent and speaker-dependent • Computer vision • Face recognition, sorting letters contain hand-written addresses by US postal office • Bio-surveillance • Detecting and tracking outbreak of disease • Robot control • Robots drive autonomously

46. Ch1 Introduction • What is machine learning (ML)? • Design a learning system: an example • ML applications • Miscellaneous issues

47. Research on ML • Current research questions • Long-term questions For the above two items, see “The Discipline of Machine Learning” by Tom Mitchell for a sample of questions. • Machine learning for tough problems: relevant novelty detection, structural learning, active learning.* *: from a slide by Jaime Carbonell et al. in April 2007.

48. Ethical Questions • When and where to apply ML technology? • For example, when collecting data for security or law enforcement, or for marketing purpose, what about our privacy? • Privacy-preserving data mining. Borrow something from Secure Multiparty Computing (SMC)?

49. Major Conference and Journal • International Conference on Machine Learning (ICML) • Conference on Neural Information Processing Systems (NIPS) • Annual Conference on Learning Theory (COLT) • Journal of Machine Learning Research (JMLR) • Machine Learning

50. Some Interesting Ref • Pattern Recognition in industry, by Phiroz Bhagat, Elsevier, 2005. • UCI Machine Learning Repository • “machine learning” entry on Wikipedia • Google offers cloud-based learning engine