Presentation Transcript

1. MLSIG Seminar Series, Dept. of CSA, IISc Joint work with B. Sriperumbudur, P. Jain, H. Karnick Purushottam Kar MLO Group, Microsoft Research India
2. Outline MLSIG seminar series, Dept. of CSA, IISc
3. Outline MLSIG seminar series, Dept. of CSA, IISc
4. Credit Card Fraud Detection Transaction 2 Guess  Truth  Loss Transaction 3 Guess  Truth  Loss Transaction 4 Guess  Truth  Loss Transaction 1 Guess  Truth  Loss MLSIG seminar series, Dept. of CSA, IISc
5. The Online Learning Process Initialize Receive instance Update Take action Truth revealed Incur loss MLSIG seminar series, Dept. of CSA, IISc
6. Benefits of Online Learning Don’t have to wait for all data to arrive Streaming data, Transactional data Applications to large scale learning Data too large to fit in memory (or even disk) Solution: stream data into memory from disk or network Fast learning Several online learning algorithms have cheap updates Online gradient descent, Mirror descent MLSIG seminar series, Dept. of CSA, IISc
7. Example: Online Classification Instances are vector-label pairs , Actions are classifiers e.g. Loss is the hinge loss function Total loss incurred by adaptive classfn Loss of single best classifier This is what a “batch” learning algorithm would have given The online process suffers Unable to see all data in one go MLSIG seminar series, Dept. of CSA, IISc
8. Regret and Generalization Regret: how much the online process suffers Online learning can compete with batch learning Excess training error if Performance on unseen points: Online-to-batch conversion: For random, convex where MLSIG seminar series, Dept. of CSA, IISc
9. Outline MLSIG seminar series, Dept. of CSA, IISc
10. Pointwise Loss Functions Loss functions for classification, regression … … look at the performance of function at onepoint Examples Hinge loss: Logistic loss: Squared loss: MLSIG seminar series, Dept. of CSA, IISc
11. Metric Learning for Classification Penalize metric for bringing blueand red points close Loss function needs to consider two points at a time! … in other words a pairwise loss function Example: MLSIG seminar series, Dept. of CSA, IISc
12. Bipartite Ranking Want relevant results to be ranked above others Penalize scoring function for each “switch” Chennai Express Search Images taken from cinemahood.com, sify.com, santabanta.com and thehindu.com
13. Pairwise Loss Functions Examples: Mahalanobis metric learning Bipartite ranking Preference learning Two-stage multiple kernel learning Indefinite kernel learning MLSIG seminar series, Dept. of CSA, IISc
14. Learning with Pairwise Loss Functions Algorithmic challenges: Training data available as a set Question: how to create pairs? Solution 1: Expensive for Solution 2: Use online techniques for a batch solver Challenge: Online creation of pairs from a data stream Desirable: Memory efficiency MLSIG seminar series, Dept. of CSA, IISc
15. Learning with Pairwise Loss Functions Learning theoretic challenges: Batch learning methods: learn from pairs Intersection between pairs: training data not i.i.d. Direct application of concentration inequalities not possible Online learning methods: let arrive in a stream Need an appropriate notion of regret Classical OTB proofs require i.i.d. data crucially This talk: mostly algorithmic solutions + hint of theory MLSIG seminar series, Dept. of CSA, IISc
16. Outline MLSIG seminar series, Dept. of CSA, IISc
17. An Online Learning Model for Pairwise Losses At each time step We propose an action (e.g. a scoring function or a metric) We receive a single point We incur loss on action Buffer Pair up with points in buffer Incur loss MLSIG seminar series, Dept. of CSA, IISc
18. An Online Learning Model for Pairwise Losses At each time step We propose an action (e.g. a scoring function or a metric) We receive a single point We incur loss on action Finite Buffer Pair up with points in buffer Incur loss MLSIG seminar series, Dept. of CSA, IISc
19. An Online Learning Model for Pairwise Losses Notions of Regret in this Model How well are we able to do on pairs that we have seen Finite buffer regret How well are we able to do on all possible pairs All pairs regret MLSIG seminar series, Dept. of CSA, IISc
20. An Online Learning Algorithm for Pairwise Losses OLP: Online learning with pairwise losses Simple variant of Zinkevich’s GIGA Start with At each Receive a new point Construct appropriate loss function or If required, update buffer with MLSIG seminar series, Dept. of CSA, IISc
21. An Online Learning Algorithm for Pairwise Losses RS-x: Reservoir sampling with replacement MLSIG seminar series, Dept. of CSA, IISc
22. An Online Learning Algorithm for Pairwise Losses Guarantees for OLP and RS-x Sampling guaranteeAt any time , the contents of buffer are i.i.d. samples from the set Regret guaranteeOLP guarantees** a finite buffer regret Finite-to-all-pairs regret conversion MLSIG seminar series, Dept. of CSA, IISc
23. An Online Learning Algorithm for Pairwise Losses OTB Guarantees for Pairwise loss functions Define For random , convex and unbounded bufferwhere MLSIG seminar series, Dept. of CSA, IISc
24. An Online Learning Algorithm for Pairwise Losses OTB Guarantees for Pairwise loss functions Define For random , convex and finite buffer of size where Corollary: MLSIG seminar series, Dept. of CSA, IISc
25. An Online Learning Algorithm for Pairwise Losses OTB Guarantees for Pairwise loss functions Define For random , strongly convex and unbounded bufferwhere MLSIG seminar series, Dept. of CSA, IISc
26. An Online Learning Algorithm for Pairwise Losses OTB Guarantees for Pairwise loss functions Define For random , strongly convex and finite bufferwhere Corollary: MLSIG seminar series, Dept. of CSA, IISc
27. An Online Learning Algorithm for Pairwise Losses Some other details Our bounds give dimension independent bounds For Hilbertian norm regularizations: no dependence on For sparsity inducing regularizations: dependence Previous work [Wang et al, COLT12]: linear dependence Proofs use (modified notions of) Rademacher averages Trickier symmetrizationstep Previous work: covering number based analysis MLSIG seminar series, Dept. of CSA, IISc
28. Some Open Problems Current all-pairs regret bound for finite buffers Can we get bounds that scale as ? Similar question for OTB conversion bounds OTB bounds require stream-oblivious buffer updates Update algorithm cannot look at just Examples: FIFO, RS, RS-x Guarantees for (suitable) stream-aware policies? MLSIG seminar series, Dept. of CSA, IISc