Robust Optimization and Applications in Machine Learning

1. Robust Optimization andApplications in Machine Learning

2. Time-series prediction via linear least-squares

3. Predicted output

4. Properties of solution

5. Non-linear prediction and kernels

6. Properties of solution

7. What is a kernel, anyway? SVM, LR, LS, MPM, PCA, CCA, FDA…

8. Example: 2nd-order polynomial kernel

9. Example: 2nd-order polynomial kernel

10. A classical way to use kernels

11. Transduction framework

12. Important property of kernel matrices

13. Kernel optimization in least-squares

14. Kernel optimization for least-squares

15. Kernel optimization via SDP or SOCP

16. A non-classical way to use kernels

17. Kernel optimization in other problems

18. Kernel optimization in SVM classifiers

19. Kernel optimization in SVM classifiers (cont’d)

20. Link with robust optimization

21. Kernel optimization and data fusion mRNA expression data hydrophobicity data protein-protein interaction data sequence data (gene, protein) upstream region data (TF binding sites)

22. Challenge

23. Example of a Kernel for Genomic Data: Pairwise Comparison Kernel

24. protein 1 0 0 1 0 1 0 1 1 0 1 0 1 1 0 1 0 0 0 0 1 1 0 0 0 0 1 0 1 1 0 1 0 0 1 0 1 0 0 1 1 0 0 0 0 0 0 1 0 0 1 0 1 0 0 0 protein 2 Example of a Kernel for Genomic Data: Linear Interaction Kernel

25. Exampe of a Kernel for Genomic Data: Diffusion Kernel

26. Learning the Optimal Kernel K

27. Learning the Optimal Kernel Integrate constructed kernels Learn a linear mix Large margin classifier (SVM) Maximize the margin

28. Yeast Protein Function Prediction

29. Yeast Protein Function Prediction

30. Yeast Protein Function Prediction MRF SDP/SVM (binary) SDP/SVM (enriched)

31. Yeast Protein Function Prediction MRF SDP/SVM (binary) SDP/SVM (enriched)

32. Part 3: summary