1 / 20

EECS 349 Machine Learning

EECS 349 Machine Learning. Instructor: Doug Downey Note: slides adapted from Pedro Domingos, University of Washington, CSE 546. Logistics. Instructor: Doug Downey Email: ddowney@eecs.northwestern.edu Office: Ford 3-345 Office hours: Wednesdays 3:00-4:30 (or by appt)

cale
Download Presentation

EECS 349 Machine Learning

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. EECS 349Machine Learning Instructor: Doug Downey Note: slides adapted from Pedro Domingos, University of Washington, CSE 546

  2. Logistics • Instructor: Doug Downey • Email: ddowney@eecs.northwestern.edu • Office: Ford 3-345 • Office hours: Wednesdays 3:00-4:30 (or by appt) • TA: Francisco Iacobelli • Email: 'f-iacobelli@northwestern.edu‘ • Office: Ford 2-202 • Office Hours: Tuesdays 2:00-3:00 • Web:www.eecs.northwestern.edu/~downey/courses/349/

  3. Evaluation • Three homeworks (50% of grade) • Assigned Friday of weeks 1, 3, and 5 • Due two weeks later • Via e-mail at 5:00PM Thursday • Late assignments will not be graded (!) • Some programming, some exercises • Final Project (50%) • Teams of 2 or 3 • Proposal due Thursday, October 16th

  4. Source Materials • T. Mitchell, Machine Learning,McGraw-Hill (Required) • Papers

  5. Case Study: Farecast

  6. A Few Quotes • “A breakthrough in machine learning would be worthten Microsofts” (Bill Gates, Chairman, Microsoft) • “Machine learning is the next Internet” (Tony Tether, Director, DARPA) • “Machine learning is the hot new thing” (John Hennessy, President, Stanford) • “Web rankings today are mostly a matter of machine learning” (Prabhakar Raghavan, Dir. Research, Yahoo) • “Machine learning is going to result in a real revolution” (Greg Papadopoulos, CTO, Sun) • “Machine learning is today’s discontinuity” (Jerry Yang, CEO, Yahoo)

  7. So What Is Machine Learning? • Automating automation • Getting computers to program themselves • Writing software is the bottleneck • Let the data do the work instead!

  8. Traditional Programming Machine Learning Computer Data Output Program Computer Data Program Output

  9. Magic? No, more like gardening • Seeds = Algorithms • Nutrients = Data • Gardener = You • Plants = Programs

  10. Sample Applications • Web search • Computational biology • Finance • E-commerce • Space exploration • Robotics • Information extraction • Social networks • Debugging • [Your favorite area]

  11. ML in a Nutshell • Tens of thousands of machine learning algorithms • Hundreds new every year • Every machine learning algorithm has three components: • Representation • Evaluation • Optimization

  12. Representation • Decision trees • Sets of rules / Logic programs • Instances • Graphical models (Bayes/Markov nets) • Neural networks • Support vector machines • Model ensembles • Etc.

  13. Evaluation • Accuracy • Precision and recall • Squared error • Likelihood • Posterior probability • Cost / Utility • Margin • Entropy • K-L divergence • Etc.

  14. Optimization • Combinatorial optimization • E.g.: Greedy search • Convex optimization • E.g.: Gradient descent • Constrained optimization • E.g.: Linear programming

  15. Types of Learning • Supervised (inductive) learning • Training data includes desired outputs • Unsupervised learning • Training data does not include desired outputs • Semi-supervised learning • Training data includes a few desired outputs • Reinforcement learning • Rewards from sequence of actions

  16. Inductive Learning • Given examples of a function (X, F(X)) • Predict function F(X) for new examples X • Discrete F(X): Classification • Continuous F(X): Regression • F(X) = Probability(X): Probability estimation

  17. What We’ll Cover • Supervised learning • Decision tree induction • Rule induction • Instance-based learning • Neural networks • Support vector machines • Bayesian Learning • Learning theory • Unsupervised learning • Clustering • Dimensionality reduction

  18. What You’ll Learn • When can I use ML? • …and when is it doomed to failure? • For a given problem, how do I: • Express as an ML task • Choose the right ML algorithm • Evaluate the results • What are the unsolved problems/new frontiers?

  19. ML in Practice • Understanding domain, prior knowledge, and goals • Data integration, selection, cleaning,pre-processing, etc. • Learning models • Interpreting results • Consolidating and deploying discovered knowledge • Loop

  20. Reading for This Week • Mitchell, Chapters 1 & 2 • Wired data mining article (linked on course Web page)(don’t take it too seriously)

More Related