Machine Learning CS 165B Spring 2012

1. Machine LearningCS 165BSpring 2012

2. Course outline • Introduction (Ch. 1) • Concept learning (Ch. 2) • Decision trees (Ch. 3) • Ensemble learning • Neural Networks (Ch. 4) • Linear classifiers • Support Vector Machines • Bayesian Learning (Ch. 6) • Genetic Algorithms (Ch. 9) • Instance-based Learning (Ch. 8) • Clustering • Computational learning theory (Ch. 7)

3. Eager versus Lazy Learners • Eager learner • Learn the decision boundary at training time • Simple distributions • Lazy learners • Learn the decision boundary at testing time • High cost of classification • Can learn complex local distributions • How to define distance? • Dimensionality reduction techniques

4. K-nearest neighbor • Classify based on the majority label of k-nearest neighbors • Voronoi diagram defines the decision boundary for 1-nearest neighbor • Distance-weighted nearest neighbor • Possible to scale each dimension based on cross-validation

5. Locally weighted regression • Approximate target function by considering weighted contribution of points close to the query point • Generalization of k-nearest neighbor classification • Radial basis functions • Choose a set of kernel functions • Use their weighted combination to learn the target function • Usually Gaussian kernels • Can be achieved by a 2-layer ANN