1 / 21

Analysis of Robust Soft Learning Vector Quantization

Analysis of Robust Soft Learning Vector Quantization. Gert-Jan de Vries (gj.de.vries@philips.com). February 2009. Outline. Learning Vector Quantization Robust Soft LVQ Analyses in controlled environment Simulations Mathematical Application on real world data Summary. N-dim.

Download Presentation

Analysis of Robust Soft Learning Vector Quantization

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Analysis of Robust Soft Learning VectorQuantization Gert-Jan de Vries (gj.de.vries@philips.com) February 2009

  2. Outline • Learning Vector Quantization • Robust Soft LVQ • Analyses in controlled environment • Simulations • Mathematical • Application on real world data • Summary

  3. N-dim Learning Vector Quantization (LVQ) • Training: • Initialize prototypes • Present data sample • Select closest prototype(s) • Update prototype(s) • Attract • Repel • Classification: • Voronoi tessellation

  4. LVQ update scheme • Prototype w of class S • Update at time step μ • Present sample ξ of class σ • Learning rate η • Update strength hS • Distances {dT = dist(ξ,wT)} • Data class label σ • … • - e.g. LVQ2.1: • Along line through ξ, wS

  5. Robust Soft LVQ (RSLVQ)* • Prototype update • Closest correct • Closest incorrect • Assignment probabilities: * [Seo & Obermayer. 2003]

  6. Robust Soft LVQ (RSLVQ) • Assumption on distribution around prototypes (Gaussian) • Controlled environment* with 2 prototypes (2 classes) • Distance-based • Further away, larger update (basic LVQ) • Relative distance (information gain) • Takes care of stability issues * [Biehl et al. 2007]

  7. Mathematical Analysis • Controlled environment • 7 characteristic quantities • Describe development of characteristic quantities in terms of recurrence relations • Use prototype update formula • Rewrite recurrence relations as differential equations (ODE) • Coupled system of 7 ODE’s • Perform averages on the ODE’s • Use characteristics of data model

  8. Analysis (2): Φ-correction • Update function: • Replace by: • Slope correction • Equal slope at d=0 • Analytical solution ODEs in limit N∞, η0

  9. Comparison of ODE’s and simulation Original Φ-corrected • Evolution of characteristic quantities • Simulations and ODE solutions coincide for both ‘original’ and ‘Φ-corrected’ RSLVQ

  10. Comparison of ODE’s and simulation (2) • Slight differences between ‘original’ and ‘Φ-corrected’, however similar tendencies • Equal generalization error for ‘original’ and ‘Φ-corrected’ RSLVQ.

  11. Analysis: Effect of softness parameter Eg (large #epochs) Eg (asymptotic) vsoft vsoft Analytical Simulations

  12. Facial Expressions (real world data set) * [Kanade et al. 2000] 108x147 pixel images*

  13. Facial Expressions: Feature Extraction (LBP) * [Ojala et al. 2002] • Per grey valued (ic) pixel, calculate Local Binary Pattern (LBP): • Only consider LBP with at most 2 bitwise transitions* (yields L=59) • These ‘uniform patterns’ represent local primitives such as edges and corners

  14. Facial Expressions: Feature Extraction (LBP) • Divide image into (42) regions • Per region Rj obtain histogram of LBP values: • Combine histograms into a single vector of length N = 42*59 = 2478 • Using overlapping regions (overlap=1/2): 143 regions  N = 8437

  15. Facial Expressions: Classification • Facial expressions of 95 university students • 7 classes: Anger, Disgust, Fear, Joy, Sadness, Surprise, Neutral • 1214 instances • 10-fold cross validation • Testing with unseen people

  16. Facial Expressions: Training 92.2% 93.2% Correct rate Correct rate vsoft vsoft LBP LBP with overlap Similar influence of softness on performance as in mathematical analysis (with limited learning time) Sometimes numerical problems

  17. Facial Expressions: Training 93.6% <Images to be included> Correct rate All Single All double Number of prototypes per class Number of prototypes per class Using optimal parameter settings as single prototype case

  18. Facial Expressions: Training Mean and standard deviation over 10 fold cross validation * [Gritti et al. 2008] Results comparable with SVM

  19. Summary • Analytical description of learning dynamics of RSLVQ • Within controlled environment • Analysis for N∞ • (Asymptotic) generalization error studied • Application to facial expression data • Similar effects on softness parameter • No large effect on number of prototypes per class • RSLVQ performs well

  20. Acknowledgements References M. Biehl, A. Gosh, and B. Hammer. Dynamics and generalization ability of LVQ algorithms. In Journal of Machine Learning Research, 8 (Feb):323-360, 2007. T. Kanade, J. Cohn, and Y. Tian. Comprehensive database for facial expression analysis. In IEEE International Conference on Automatic Face & Gesture Recognition (FG), 46-53, 2000. T. Ojala, M. Pietikäinen, and T. Mäenpää. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(7):971-987, 2002. S. Seo, and K. Obermayer, Soft learning vector quantization, Neural computation, 15:1589-1603, 2003 C. Shan, S. Gong, and P.W. McOwan. Robust facial expression recognition using local binary patterns. In IEEE International Conference on Image Processing (ICIP), volume 2:370-373, Genoa, Italy, September 2005. T. Gritti, C. Shan, V. Jeanne, R. Braspenning, Local Features based Facial Expression Recognition with Face Registration Errors, FG 2008. • Philips Research – Video Processing & Analysis: • Vincent Jeanne • Caifeng Shan • Rijksuniversiteit Groningen: • Michael Biehl

More Related