Introduction to Non-linear Support Vector Machine (SVM)

1. Introduction to Non-linear Support Vector Machine (SVM) Author:Jean-Philippe Vert Bioinformatics Center, Kyoto University, Japan Advisor: Dr.Hsu Graduate: Ching-Wen Hong

2. Outline • 1.Linear SVM • 2.Non-linear SVM • 3.Training a SVM in the feature space • 4.Kernal • 5.Popular kernals • 6.The approach for Non-linear SVM • 7. Classification with a Polynomial kernel • 8. Classification with a Gaussian kernel • 9.Conclusion

3. Linear SVM

4. Linear SVM

5. Linear SVM

6. Linear SVM

7. Linear SVM

8. Linear SVM

9. Linear SVM

10. Linear SVM

11. Non-linear SVM

12. Non-linear SVM

13. Non-linear SVM

14. Training a SVM in the feature space • (1)Input:a training set S={(x1,y1),…,(XN,YN)} is not linearly separable. • (2)A mapping Φ(xi)=( Φ1(xi) , … , ΦM(Xi) ) , i=1,…,N • (3)The training set Φ(S)={ (Φ(x1),y1),…,(Φ(xN), yN) } can be linearly separable in the feature space. • (4)The dual problem is to maximize • Max LD=∑αi-1/2∑αiαjyiyjΦ(xi)․Φ(xj) • S.t. 0 ≤ αi ≤ C , i=1,…,N ,and ∑ αiyi = 0 • (5)We can find the decision function • f(x)=w․Φ(x)+b = ∑αiyiΦ(xi)․Φ(x) + b • K(x,x‘) =Φ(x)․Φ(x') is a Kernel function

15. Kernel • (1).Kernel K(x,x‘)=Φ(x)․Φ(x‘) • (x,x‘) is any two points in the input space • Φ(x) is a mapping to a feature space

16. Popular Kernels

17. Popular Kernels

18. The approach for Non-linear SVM • The following steps: • (1).Input a training set S={(x1,y1),…,(xN,yN)} • (2).Choose a Kernel K(․,․) • (3).Training a SVM in the feature space • i.e.To find the decision function f(x)=∑αiyiK(xi,x) • (4).Classify any new object and to test efficiency on the research of data. • There is usually no automatic way to choose a Kernel and to adjust the corresponding parameters,Therefore we usually has to try different Kernels and paramters.

19. Classification with a Polynomial kernel

20. Classification with a Gaussian kernel

21. Conclusion • Non-linear SVM is a extremely powerful learning algorithm for binary classification. • It is important to find Kernel but it is difficult. • If we can find a way to Kernel,That is a nice thing to develop in the machine learning.