150 likes | 271 Views
A cursory glance at machine learning. Ashwath Rajan. Overview, in brief. Marriage between statistics, linear algebra, calculus, and computer science Machine Learning: Supervised Learning ex: linear Regression Unsupervised Learning ex: clustering Discriminative Methods
E N D
A cursory glance at machine learning Ashwath Rajan
Overview, in brief • Marriage between statistics, linear algebra, calculus, and computer science • Machine Learning: • Supervised Learning • ex: linear Regression • Unsupervised Learning • ex: clustering • Discriminative Methods • Learns borders in the feature space; border chosen to minimize error functions • Generative Methods • Learns probabilistic distribution of each class, dependent on parameters; gives confidence of classification
Supervised learninglinear regression • Regression can be used to fit a numerical model to a parameter space • To solve, minimize error function • Can solve analytically, or iteratively Test Set Predictions
Iterative linear regression Turns out to be )
Supervised learningsupport vector machines • Mechanism for both pattern recognition and regression • Finds maximum margin n-1 dimensional hyperplanes that split n dimensional parameter spaces • Data can be separable or non-separable Separable vs Non-separable data sets
Unsupervised learningk-means clustering Procedure – 0. Dictate number of clusters (k) Randomly select k starting points. These serve as the initial group centroids. Associate the remaining points with the nearest centroid. Move the centroids to the center of their respective clusters Repeat steps 2 and 3
Unsupervised learning • If number of clusters is unknown, can use different algorithms, where instead of setting # means, we set size of relative neighborhood
Frequentist vs. Bayesian • Maximum Likelihood • Assumes fixed value for parameters • Can be used analytically • Bayesian estimation • Assumes parameter as distribution • Uses evidence to amend prior distribution into posterior
Probability and Bayes rule • Bayesian probability estimates allow prior distributions to be modified with discovered data.
Bayes rule – cancer example Say, probability of rare cancer: • Probability of no cancer: Now say, there is a blood test to detect cancer • Its fairly accurate, as described by the following table: Sensitivity Specificity
Bayes rule – cancer example • So, what happens if you get a positive result? • What is the chance you have cancer? Use Bayes Rule: Sensitivity = .8 * .01 / .594 = .13 Specificity
My research at USC • Machine Learning to help Smart Grid • Take building sensor data, and find models to connect different data streams to kWh usage • Both supervised and unsupervised techniques could be considered – however, supervised learning is often most apt
Online courses • Much of this material has been shamelessly reproduced/copied from online coursework: • Udacity: Statistics 101 and CS 373 AI • Coursera: Great 10 week machine learning course https://www.coursera.org/course/ml
Cited • Regression Example: Andrew Ng, Stanford – Coursera • Cancer Example: Sebastian Thrum, Udacity • “A Tutorial on Support Vector Machines for Pattern Recognition” - CHRISTOPHER J.C. BURGES; Bell Laboratories, Lucent Technologies 1998 • Pattern Recognition Primer – David Doria, 2008