Development o f Kernel Fisher Discriminant Model Using The Cross-Entropy Method

1. Development of Kernel Fisher Discriminant Model Using The Cross-Entropy Method Budi Santosa Andiek Sunarto Industrial Engineering Institut Teknologi Sepuluh Nopember (ITS) Indonesia

2. Introduction Fisher linear discriminant analysis (LDA) aims to separatetwo classes of data linearly through discriminant function. Using eigen decomposition we can find the parameter for discriminant function f(x)=wx+b

3. Problem • Fisher Discriminant (FDA) analysis often fails when the matrix of sum square within (SSW) is singular • FDA is good only for linear cases, for nonlinear the results are not good • Kernel FDA handle both problems

4. How to Solve KFD • This paper proposed a new approach Cross Entropy (CE)

5. Cross Entropy • Generate N vector solutions which is the coefficient of discriminat function , , using certain procedure • Generate  with the following formula =+r , r; random number which follos normal distribution • Pu  to the objetive function: • Choose the  100 %  (out of N) that produce best objetive function • Utilize these best sample (elite sample) to update parameter  and  using

6. Data

7. Results

8. Conclusions • In this paper we showed how KFD can be solved not only by analytical or deterministic method (eigendecomposition method) but also stochastic method (cross entropy method). • Some advantages of applying the CE method to solve KFD algorithm is : • 1) it gives the new alternative method to optimize the classifier function of KFD; • 2) CE-KFD does not face with the problem of postive definiteness of the covariance matrix as in FDA; • 3) For the most of the experimented data sets, the accuracy of CE-KFD is promising.