1 / 25

Research Presentation:

Research Presentation:. Software Release and Support Vector Machine. Seungchan Lee Intelligent Electronic Systems Human and Systems Engineering Department of Electrical and Computer Engineering. Overview. Software Release Isip_lm_tester Isip_network_builder Debugging utility : Purify

debbie
Download Presentation

Research Presentation:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Research Presentation: Software Release and Support Vector Machine Seungchan Lee Intelligent Electronic Systems Human and Systems Engineering Department of Electrical and Computer Engineering

  2. Overview • Software Release • Isip_lm_tester • Isip_network_builder • Debugging utility : Purify • Verification System • Isip_verify • Support Vector Machine • Audio File Generation • Next Plans

  3. Isip_lm_tester, Isip_network_builder • Dummy Symbol generation problem • Sentence generation terminated when met with dummy symbol at the highest level. • Dummy Symbol should not show at the output sentence.  Include Dummy Symbol check routine • Exclude Symbol generation problem • When turn on exclude symbol flag, lm_tester should not generate exclude symbol. • It need to modify HierarchicalSearch class. • Isip_network_buider • Add save option for ABNF, BNF • Need to correct dummy symbol generation problem • When generating dummy symbol without any subgraph, it generates error message.

  4. Debugger Utility • Purify • What is problem? • Compilation error : When instrumenting purify, it generates error message. • It is not easy to figure out the reason because we have so many linking process when compiling. • How to resolve? • Simple program without IFC classworks fine • Narrow down which classes are problem. • Exclude all linking process, and then add one class repeatedly. • Solution : • After track down the linking process, I can figure out the problem is originated from sphere utility. • How can correct it? • I’m currently doing this.

  5. Isip_verify • When doing HMM training, it generates segmentation fault. • This happens at the end of the program related to HierarchicalDigraph object. • Recently, we have many changes in IFC classes, but this problem might exists sometimes ago. • When doing SVM training, it generates checksum error. • This error did not happen before I was recompiling whole repository.  isip_verify utilty also need to be throughly investigated using purify utility

  6. Audio File Generation • Load one or two SWB CDs • Select 100 conversations • For each conversation, strip the NIST header • Grab every other byte starting with the first byte (first channel) and put that into a raw audio file; PRESERVE the 8-bit ulaw data (do not use or convert to 16-bit) • Convert this file to Sun ".au" using Sox

  7. What to Learn? • Audio File format • .au file format • Widely used in UNIX machine and originated by SUN. • Header + Variable length information + audio data • Support various encoding types • NIST SPHERE file • Raw format • PERL Programming Language • It is simple programming language which performs extracting and printing out information from a text file. • Interpreted Language ( not compiled) • Conversion Utilities • w_decode • Sox • “od” command

  8. Why Support Vector Machine ? • This is new learning technology to be noticed recently. • Even though it has been situated as a subfield of machine learning, it still have many issues about theory and algorithm. • To be more familiar with verification system, it is required to review one field for the next step.

  9. How it works? • Suppose we have low dimensional feature space. It is consist of positive examples and negative examples • How can we classify this? • How about the following case?

  10. How it works? • Simple idea : Low dimensional feature space map into high dimensional feature space using kernel function.

  11. How can we determine maximum margin? • To explain this, we need to know the following concepts. • Margin concepts • Lagrange multiplier • Primal and dual representation • Karush-Kuhn-Tucker Conditions (KKT) • Risk Bounds and Minimization Maximal Margin classifier

  12. Hyperplane • Linear classification Objective Function • Input space X is split into two • parts by the hyperplane defined • by the equation

  13. Margin Geometric margin of two points The margin of training set

  14. Maximal Margin Classifier • The Simplest model, but works only for data which are linearly separable in the feature space. •  easy to understand and main building block for more complex SVMs w H2 H1 Margin • Plus-plane = • Minus-plane = • Separating hyperplane = Classify as.. +1 if -1 if

  15. Maximal Margin Classifier • Computing the margin width w Margin

  16. Maximal Margin Classifier • Computing the margin width w Margin

  17. Maximal Margin Classifier • How to transform this optimization problem into dual problem? w Margin Lagrange Hypothesis can be described as a linear combination of the training points.

  18. Maximal Margin Classifier • How to transform this optimization problem into dual problem? w Margin

  19. Maximal Margin Classifier • How to transform this optimization problem into dual problem? w Margin

  20. Maximal Margin Classifier • How to transform this optimization problem into dual problem? w Margin Only these points are involved for the weight vector.

  21. Maximal Margin Classifier • How to transform this optimization problem into dual problem? w Margin

  22. Maximal Margin Classifier • How to transform this optimization problem into dual problem? w Margin

  23. Review Maximal Margin SVMs • Can be slow in practice • Dose not control the number of support vector (Sparseness) • Only one degree of freedom is the choice of kernel  model selection • Cannot be used non linear separable feature space •  many real world problems deal with nonlinear, noisy data. • However, it is a starting point for the more sophisticated SVMs.

  24. Next Plan • Software Release • Resolve purify compilation problem • Examine memory leak problem using purify utility • Track down remaining bugs • Test several cases • Verification System • Do NIST 2003 Experiment using new isip_verify • Implemenation techniques of support vector machine • Algorithm comparison between several SVM softwares • Resolve memory leak problems

  25. Reference • An introduction to Support Vector Machines and other kernel-based learning methods by “Nello Cristianini and John Shawe-Taylor”, 2000, Cambridge Press • Support Vector Machines Tutorial Slides by Andrew W. Moore • http://www.autonlab.org/tutorials/svm15.pdf • Practical Perl Programming • http://www.cs.cf.ac.uk/Dave/PERL/

More Related