Support Vector Machines for Multiple-Instance Learning

1. Support Vector Machines for Multiple-Instance Learning Authors: Andrews, S.; Tsochantaridis, I. & Hofmann, T. (Advances in Neural Information Processing Systems, 2002, 15, 577-584) Presentation by BH Shen to Machine Learning Research Lab, ASU 09/19/2006

2. Outline • SVM (Support Vector Machine) • Maximum Pattern Margin Formulation • Maximum Bag Margin Formulation • Heuristics • Simulation results of some other learning algorithms for MIL.

3. Problem Instance • For supervised learning, we are given • For MIL, we are given

4. SVM: To find a Max Margin Classifier Find a classifier that gives the least chance of causing a misclassification if we’ve made a small error in locating the boundary.

5. SVM: To find a Max Margin Classifier The margin of the classifier is the width between the boundary of the distinct classes.

6. SVM: To find a Max Margin Classifier Support vectors are those datapoints on the boundary of the half-spaces. Support vectors

7. SVM The half-spaces define the feasible regions for the data points

8. SVM Soft margin: errors are allowed to solve infeasibility issue for the datapoints that cannot be separated.

9. SVM: Constraints Constraints: are combined into

10. SVM: Objective function Margin:

11. SVM: Objective function Maximizing is the same as Minimizing . We also like to minimize the sum of training set errors, due to the slack variables

12. SVM: Primal Formulation Quadratic minimization problem: Subject to

13. Maximum Pattern Margin Modification to SVM: At least one instance in each positive bag is positive.

14. Maximum Pattern Margin

15. Pattern Margin: Primal Formulation Mixed integer problem: Subject to

16. Heuristics • Idea: Alternate the following TWO steps • For fixed integer variables, solve the associated quadratic problem for optimal discriminate function. • For a given discriminate function, update one, several, or all integer variables that (locally) minimize the objective function.

17. Hueristic for Maximum Pattern Margin

18. Maximum Bag Margin • A bag label is represented by an instance that has maximum distance from a given separating hyperplane. • Select a “witness” data point from each positive bag as the delegant of the bag. • For the given witnesses, apply SVM.

19. Maximum Bag Margin

20. Maximum Bag Margin

21. Maximum Bag Margin

22. Bag Margin: Primal Formulation Mixed integer formulation given in the paper: A (better?) alternative constraint for positive bag might be

23. Hueristic for Maximum Bag Margin

24. Simulation results

25. Accuracy for other MIL Methods “Supervised versus Multiple Instance Learning: An Empirical Comparison” by S Ray, M Craven for the 22nd International Conference on Machine Learning, 2005 Regression Gaussion Quadratic Rule-based Supervised learning algorithm (Non-MIL)

26. Conclusion from the table • Different inductive biases are appropriate for different MI problems. • Ordinary supervised learning algorithms learn accurate models in many MI settings. • Some MI algorithms learn consistently better than their supervised-learning counterparts.

27. The End