1 / 24

PROJECT

PROJECT. Compare SVM and standard models (neural nets etc) & Bagging+SVM+Data Reduction for fast and efficient learning on large data. By: Nitin Chaudhary Prof: Dr. Vucetic Slobodan. INTRODUCTION. SVM:The goal in training a Support Vector machine is to find the separating hyperplane

jimbo
Download Presentation

PROJECT

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PROJECT • Compare SVM and standard models (neural nets etc) • & • Bagging+SVM+Data Reduction for fast and efficient • learning on large data By: Nitin Chaudhary Prof: Dr. Vucetic Slobodan

  2. INTRODUCTION SVM:The goal in training a Support Vector machine is to find the separating hyperplane with the largest margin; we expect that the larger the margin, the better generalization of the classifier Wow, These are the support vectors y2 y1 Optimal Hyper Plane

  3. The Support Vectors are equally close to the hyperplane The Support Vectors are the training samples that define the optimal separating hyperplane and are the most difficult patterns to classify. Informally speaking, they arethe patterns most informative for the classification task.

  4. FORMULATIONS • C-Support Vector Classification • min (1/2)WT W + C Σ ξi • w,b,ξ 2) nu-Support Vector Classification min (1/2)WT W- pv +(1/L) Σ ξi w,b,ξ,p Above are the primal problems for both classifications

  5. For the first part of the project I have used Pima.txt as my data set Remember the form of polynomial kernel (Gamma*<X(:,i),X(:,j)>+Coefficient)^Degree Wow!I remember these, it was taught in CIS 595 class The form of rbf or Gaussian Kernel exp(-Gamma*|X(:,i)-X(:,j)|^2)

  6. Gamma : If the input value is zero, Gamma will be set defautly as 1/(max_pattern_dimension) in the function. If the input value is non-zero, Gamma will remain unchanged in the function. C- Cost of Constraint Violation

  7. METHODOLGY For nu-SVC and C-SVC using rbf or Gaussian Kernel

  8. For nu-SVC or C-SVC using Polynomial Kernel

  9. For Neural Networks

  10. RESULTS • Maximum Accuracy in case of nu-SVC using rbf or • Gaussian kernel is 79.11% at Gamma = 0.000001 and • C=0.00001 • 2) Maximum Accuracy in case of C-SVC using rbf or • Gaussian kernel is 80.94% at Gamma = 0.000001 and • C=100000 • 3) Maximum Accuracy in case of nu-SVC using Polynomial • kernel is 79.90% at Gamma = 0.00001, C=10, Coeff = 1 • and Degree = 5

  11. RESULTS cont…. 4) Maximum Accuracy in case of C-SVC using Polynomial kernel is 80.68% at Gamma = 0.00001, C=100, Coeff=10 and Degree=3 5)Maximum Accuracy in case of Neural Networks is 81.66% at no of hidden neurons =10, number of training iterations =100, show=10, max_fail=50

  12. Bagging + SVM + Data Reduction for fast and efficient learning on large data Goal: To perform Bagging on very large data sets for SVM. The data set that I have used over here is cover_type.txt

  13. The test_covertype.txt has 20 attributes and 7 classes, so last 7 rows are classes. I have used class 2 ie column 22 as positive class and all other classes as negative class. So it means that I am only dealing with binary classification problem. Same is true for train_covertype.txt But what is Bagging?

  14. Bagging is a “bootstrap” ensemble method that creates individuals for it’s ensemble by training each classifier on a random distribution of the training set. Each classifier’s training set is generated by randomly drawing with replacement, N examples- where N is the size of the training set But how do you actually do it.

  15. Methodology 1) Divide the data set in to train_covertype.txt and test_covertype.txt 2) First take 20% of train_covertype.txt and train SVC using any of the kernels and then use this particular kernel for future experiments too. 3) Then we test our model with test_covertype.txt and get the Predicted Labels or Predictions 4) Records these Predicted Labels as the first column of PreLabel Matrix 5) Again repeat steps 2,3 and 4 for a few number of times, say 3 or 5 6) And record the Predicted Labels as second, third and fourth column respectively 7) Remember that when we take the 20% of train_cover Type.txt after the first time we do it by replacement.

  16. So, now we get a nice PreLabeled Matrix And so on

  17. Once I get the PreLabel Matrix I need to get the Predictions by majority vote

  18. Now, with these Predictions and the True Labels from the test_covertype I calculate the accuracy by using accuracy.m file provided in the class Similarly, I do the above steps exactly the same way by taking 30%, 40%, 50%,….. Of train_covertype.txt and try to calculate the accuracy.

  19. RESULTS

  20. RESULTS cont…

  21. RESULTS cont.. We try to plot a graph between the Percentage of train_cover type taken And accuracy that we got. Accuracy % of Train_covertype.txt

  22. QUESTIONS?????

  23. Good Bye

More Related