Lecture 1: Introduction

1. Lecture 1: Introduction Machine Learning Queens College

2. Today Welcome Overview of Machine Learning Class Mechanics Syllabus Review

3. My research and background • Speech • Analysis of Intonation • Segmentation • Natural Language Processing • Computational Linguistics • Evaluation Measures • All of this research relies heavily on Machine Learning

4. You • Why are you taking this class? • What is your background and comfort with • Calculus • Linear Algebra • Probability and Statistics • What is your programming language of preference? • C++, java, or python are preferred

5. Machine Learning Data Learning Algorithm Behavior ≥ Data Programmer or Expert Behavior Automatically identifying patterns in data Automatically making decisions based on data Hypothesis:

6. Machine Learning in Computer Science Speech/Audio Processing Planning Robotics Natural Language Processing Locomotion Machine Learning Vision/Image Processing Biomedical/Chemedical Informatics Financial Modeling Human Computer Interaction Analytics

7. Major Tasks • Regression • Predict a numerical value from “other information” • Classification • Predict a categorical value • Clustering • Identify groups of similar entities • Evaluation

8. Feature Representations Our Focus Entity in the World Feature Representation Machine Learning Algorithm Feature Extraction Web Page User Behavior Speech or Audio Data Vision Wine People Etc. How do we view data?

9. Feature Representations

10. Classification OR Identify which of N classes a data point, x, belongs to. xis a column vector of features.

11. Target Values Goal of Classification Identify a function y, such that y(x) = t In supervised approaches, in addition to a data point, x, we will also have access to a target value, t.

12. Feature Representations

13. Graphical Example of Classification

14. Graphical Example of Classification ?

15. Graphical Example of Classification ?

16. Graphical Example of Classification

17. Graphical Example of Classification

18. Graphical Example of Classification

19. Decision Boundaries

20. Regression Goal of Classification Identify a function y, such that y(x) = t • Regression is a supervised machine learning task. • So a target value, t, is given. • Classification: nominal t • Regression: continuous t

21. Differences between Classification and Regression • Similar goals: Identify y(x) = t. • What are the differences? • The form of the function, y (naturally). • Evaluation • Root Mean Squared Error • Absolute Value Error • Classification Error • Maximum Likelihood • Evaluation drives the optimization operation that learns the function, y.

22. Graphical Example of Regression ?

23. Graphical Example of Regression

24. Graphical Example of Regression

25. Clustering • Clustering is an unsupervised learning task. • There is no target value to shoot for. • Identify groups of “similar” data points, that are “dissimilar” from others. • Partition the data into groups (clusters) that satisfy these constraints • Points in the same cluster should be similar. • Points in different clusters should be dissimilar.

26. Graphical Example of Clustering

27. Graphical Example of Clustering

28. Graphical Example of Clustering

29. Mechanisms of Machine Learning • Statistical Estimation • Numerical Optimization • Theoretical Optimization • Feature Manipulation • Similarity Measures

30. Mathematical Necessities • Probability • Statistics • Calculus • Vector Calculus • Linear Algebra • Is this a Math course in disguise?

31. Why do we need so much math? • Probability Density Functions allow the evaluation of how likely a data point is under a model. • Want to identify good PDFs. (calculus) • Want to evaluate against a known PDF. (algebra)

32. Gaussian Distributions We use Gaussian Distributions all over the place.

33. Gaussian Distributions We use Gaussian Distributions all over the place.

34. Data Data Data • “There’s no data like more data” • All machine learning techniques rely on the availability of data to learn from. • There is an ever increasing amount of data being generated, but it’s not always easy to process. • UCI • http://archive.ics.uci.edu/ml/ • LDC (Linguistic Data Consortium) • http://www.ldc.upenn.edu/ • Contact me for speech data. • Is all data equal?

35. Class Structure and Policies • Course website: • http://eniac.cs.qc.cuny.edu/andrew/ml/syllabus.html • Email list • CUNY First has an email function – most students do not use the associated email address… • Put your email address on the sign up sheet.