1 / 42

Support Vector Machines (part 2)

Support Vector Machines (part 2). Plan of the lecture. SVM – main issues repeated Soft margin Multi-class problems Applications to face recognition Training set optimization. SVM – main issues. Aim: data classification Two stages: learning (training) classification. SVM – main issues.

wilton
Download Presentation

Support Vector Machines (part 2)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Support Vector Machines (part 2) Face Recognition & Biometric Systems

  2. Plan of the lecture • SVM – main issues repeated • Soft margin • Multi-class problems • Applications to face recognition • Training set optimization Face Recognition & Biometric Systems

  3. SVM – main issues • Aim: data classification • Two stages: • learning (training) • classification Face Recognition & Biometric Systems

  4. SVM – main issues • Solves linearly separable problems • Input data are transformed • mapping into higher dimensions • Training: find optimal hyperplane • margin maximisation Face Recognition & Biometric Systems

  5. SVM – main issues • A function: • Data mapping: x(x) • Dot product used in all calculations • Dot product -> kernel of convolution • No need to know the function  Face Recognition & Biometric Systems

  6. Convolution kernels • Linear • Polynomial • RBF (radial basis functions) Face Recognition & Biometric Systems

  7. SVM – main issues • Optimal hyperplane: w0 • x + b0 = 0 • for 2D data it is a line • Optimal margin width: Face Recognition & Biometric Systems

  8. SVM – main issues • Optimal hyperplane: • yi – class label • i – Lagrange multipliers (obtained during optimisation) Face Recognition & Biometric Systems

  9. SVM – main issues • Lagrange coefficients (): • calculated for every vector from the training set • non-zero for support vectors • equal zero for the majority of vectors • Training set after the optimisation: • support vectors •  coefficients for every vector • number of vectors reduced Face Recognition & Biometric Systems

  10. 1 ... n Training Face Recognition & Biometric Systems

  11. SVM – main issues • Classification of a vector: xr, xs – support vectors from opposite classes Face Recognition & Biometric Systems

  12. Soft margin • Error allowed during the training: • Number of errors minimised • Optimised function must be modified Face Recognition & Biometric Systems

  13. Soft margin • Margin maximisation • Minimisation of functional (F – monotonic, convex function): • C – penalty parameter • presentation • Constraints: Face Recognition & Biometric Systems

  14. Soft margin • Optimisation without the soft margin: Face Recognition & Biometric Systems

  15. Soft margin • Optimisation with the soft margin (for F(u) = u2): Face Recognition & Biometric Systems

  16. Multi-class problem • Example Face Recognition & Biometric Systems

  17. Multi-class problem • Based on two-class problem • solved by the SVM • N classes in the training set • Possible solutions: • base-class approach • 1 – N comparisons • 1 – 1 comparisons Face Recognition & Biometric Systems

  18. The base-class approach • The base-class approach • one class selected as a base class • each class compared with the base class • the strongest response decides • Classification of a single vector: • (N – 1) two-class classifications Face Recognition & Biometric Systems

  19. The base-class approach Face Recognition & Biometric Systems

  20. The base-class approach Face Recognition & Biometric Systems

  21. The base-class approach Face Recognition & Biometric Systems

  22. The base-class approach Face Recognition & Biometric Systems

  23. The base-class approach Face Recognition & Biometric Systems

  24. The base-class approach • Advantages: • high speed • effective when non-base classes are easily separable • Disadvantages: • problems with separating non-base classes Face Recognition & Biometric Systems

  25. 1 – N comparisons • Each class compared with the rest • The strongest response decides • Classification of a single vector: • N two-class classifications • Compared to the base-class approach: • more universal (symmetry) • comparable speed Face Recognition & Biometric Systems

  26. 1 – N comparisons Face Recognition & Biometric Systems

  27. 1 – 1 comparisons • Each class compared with each other • The highest precision • Classification of a single vector: • N(N – 1)/2 two-class classifications • Very slow method Face Recognition & Biometric Systems

  28. SVM for face recognition • Detection and verification • Feature vectors comparison • Multi-method fusion • Other applications Face Recognition & Biometric Systems

  29. Face detection • Ellipse detection • Generalised Hough Transform • a set of face candidates • Normalisation of the candidates • Verification • image (as a vector) classified by the SVM • multi-class approach Face Recognition & Biometric Systems

  30. Feature vectors comparison • Aim: measure similarity between feature vectors • Distance-based similarity: • Euclidean distance • Mahalanobis distance • Similarity measured by the SVM: • two vectors subtracted from each other create a difference vector • difference vector classified K11 K21 K12 K22 ... ... K1n K2n Face Recognition & Biometric Systems

  31. Feature vectors comparison K11 - K21 The same class K12 - K22 SVM Different classes ... K1n - K2n Face Recognition & Biometric Systems

  32. Feature vectors comparison • Good improvement for EBGM • Eigenfaces not improved • similar results to other metrics Face Recognition & Biometric Systems

  33. Multi-method fusion • Many feature extraction methods K1 K1 S1 K2 K2 S2 S ... ... ... Kn Kn Sn Two images Feature vectors Similarities Face Recognition & Biometric Systems

  34. Multi-method fusion • Vector of similarities classified • linear kernel • polynomial kernel • time-consuming classification • SVM applied only for the training • linear kernel – weights for the methods (dimensions stand for methods) • average mean based on the weights Face Recognition & Biometric Systems

  35. Other applications • Assessment of recognition accuracy • vector of sorted similarities to the elements in the gallery • can be used for many images of the same person • Image quality estimation • e.g. elimination of blurred images Face Recognition & Biometric Systems

  36. SVM – limitations • Constant and small number of classes • too much time-consuming for many classes • Training set: • must be representative • optimal number of elements • The parameters must be tuned • Relevance Vector Machines Face Recognition & Biometric Systems

  37. Training set optimization • Representative training set: • similarity to the classified data • universal classification rules • difficult to acquire • Real training sets: • data acquired automatically • low quality, faulty data • large number of data Face Recognition & Biometric Systems

  38. Training set optimization • Selection of available data • subset drawn randomly • genetic algorithms • Genetic algorithms • heuristic optimization technique • evolutional strategy • population of individuals • fitness • genetic operators: • selection • mutation • crossover Face Recognition & Biometric Systems

  39. + – Training set optimization Individual Population drawn Class + Class – Population of training sets SVM training N elements N elements Effectiveness test Effectiveness test Individual Fittness Evolutional operations Face Recognition & Biometric Systems

  40. SVM compared to ANN • Support Vector Machines: • more transparent calculations • more controllable than neural networks • implements various types of ANN • useful for image processing • Artificial Neural Networks: • more applications (e.g. compression) • dedicated hardware solutions Face Recognition & Biometric Systems

  41. Summary • Soft margin – automatic selection • Multi-class problems: • can be solved basing on two-class problems • various approaches • Many possible applications in the area of face recognition • Training set optimization Face Recognition & Biometric Systems

  42. Thank you for your attention! • Next time: Elastic Bunch Graph Matching Face Recognition & Biometric Systems

More Related