Research Presentation:

Research Presentation: Software Release and Support Vector Machine Seungchan Lee Intelligent Electronic Systems Human and Systems Engineering Department of Electrical and Computer Engineering

Overview • Software Release • Isip_lm_tester • Isip_network_builder • Debugging utility : Purify • Verification System • Isip_verify • Support Vector Machine • Audio File Generation • Next Plans

Isip_lm_tester, Isip_network_builder • Dummy Symbol generation problem • Sentence generation terminated when met with dummy symbol at the highest level. • Dummy Symbol should not show at the output sentence.  Include Dummy Symbol check routine • Exclude Symbol generation problem • When turn on exclude symbol flag, lm_tester should not generate exclude symbol. • It need to modify HierarchicalSearch class. • Isip_network_buider • Add save option for ABNF, BNF • Need to correct dummy symbol generation problem • When generating dummy symbol without any subgraph, it generates error message.

Debugger Utility • Purify • What is problem? • Compilation error : When instrumenting purify, it generates error message. • It is not easy to figure out the reason because we have so many linking process when compiling. • How to resolve? • Simple program without IFC classworks fine • Narrow down which classes are problem. • Exclude all linking process, and then add one class repeatedly. • Solution : • After track down the linking process, I can figure out the problem is originated from sphere utility. • How can correct it? • I’m currently doing this.

Isip_verify • When doing HMM training, it generates segmentation fault. • This happens at the end of the program related to HierarchicalDigraph object. • Recently, we have many changes in IFC classes, but this problem might exists sometimes ago. • When doing SVM training, it generates checksum error. • This error did not happen before I was recompiling whole repository.  isip_verify utilty also need to be throughly investigated using purify utility

Audio File Generation • Load one or two SWB CDs • Select 100 conversations • For each conversation, strip the NIST header • Grab every other byte starting with the first byte (first channel) and put that into a raw audio file; PRESERVE the 8-bit ulaw data (do not use or convert to 16-bit) • Convert this file to Sun ".au" using Sox

What to Learn? • Audio File format • .au file format • Widely used in UNIX machine and originated by SUN. • Header + Variable length information + audio data • Support various encoding types • NIST SPHERE file • Raw format • PERL Programming Language • It is simple programming language which performs extracting and printing out information from a text file. • Interpreted Language ( not compiled) • Conversion Utilities • w_decode • Sox • “od” command

Why Support Vector Machine ? • This is new learning technology to be noticed recently. • Even though it has been situated as a subfield of machine learning, it still have many issues about theory and algorithm. • To be more familiar with verification system, it is required to review one field for the next step.

How it works? • Suppose we have low dimensional feature space. It is consist of positive examples and negative examples • How can we classify this? • How about the following case?

How it works? • Simple idea : Low dimensional feature space map into high dimensional feature space using kernel function.

How can we determine maximum margin? • To explain this, we need to know the following concepts. • Margin concepts • Lagrange multiplier • Primal and dual representation • Karush-Kuhn-Tucker Conditions (KKT) • Risk Bounds and Minimization Maximal Margin classifier

Hyperplane • Linear classification Objective Function • Input space X is split into two • parts by the hyperplane defined • by the equation

Margin Geometric margin of two points The margin of training set

Maximal Margin Classifier • The Simplest model, but works only for data which are linearly separable in the feature space. •  easy to understand and main building block for more complex SVMs w H2 H1 Margin • Plus-plane = • Minus-plane = • Separating hyperplane = Classify as.. +1 if -1 if

Maximal Margin Classifier • Computing the margin width w Margin

Maximal Margin Classifier • How to transform this optimization problem into dual problem? w Margin Lagrange Hypothesis can be described as a linear combination of the training points.

Maximal Margin Classifier • How to transform this optimization problem into dual problem? w Margin

Maximal Margin Classifier • How to transform this optimization problem into dual problem? w Margin Only these points are involved for the weight vector.

Maximal Margin Classifier • How to transform this optimization problem into dual problem? w Margin

Review Maximal Margin SVMs • Can be slow in practice • Dose not control the number of support vector (Sparseness) • Only one degree of freedom is the choice of kernel  model selection • Cannot be used non linear separable feature space •  many real world problems deal with nonlinear, noisy data. • However, it is a starting point for the more sophisticated SVMs.

Next Plan • Software Release • Resolve purify compilation problem • Examine memory leak problem using purify utility • Track down remaining bugs • Test several cases • Verification System • Do NIST 2003 Experiment using new isip_verify • Implemenation techniques of support vector machine • Algorithm comparison between several SVM softwares • Resolve memory leak problems

Reference • An introduction to Support Vector Machines and other kernel-based learning methods by “Nello Cristianini and John Shawe-Taylor”, 2000, Cambridge Press • Support Vector Machines Tutorial Slides by Andrew W. Moore • http://www.autonlab.org/tutorials/svm15.pdf • Practical Perl Programming • http://www.cs.cf.ac.uk/Dave/PERL/

Research Presentation: