1 / 12

Introduction to Whitening Transformations in Pattern Recognition

Understand the power of normal distributions to model continuous feature vectors, the impact of mean and variance on normal distributions, and the transformation process for multivariate distributions. Learn about support regions, derivation techniques, and the importance of identity covariance. Explore whitening transformations for unequal variances, off-diagonal elements, and coordinate transformations in pattern recognition.

alejandrov
Download Presentation

Introduction to Whitening Transformations in Pattern Recognition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. • Objectives: Normal Random Variables Support Regions Whitening Transformations Resources: DHS – Chap. 2 (Part 2) K.F. – Intro to PR X. Z. – PR Course S.B. – Statistical Norm. Java Applet ECE 8443 – Pattern Recognition LECTURE 07: WHITENING TRANSFORMATIONS • URL: .../publications/courses/ece_8443/lectures/current/lecture_07.ppt

  2. 07: WHITENING TRANSFORMATIONS UNIVARIATE NORMAL DISTRIBUTION • A normal or Gaussian density is a powerful model for modeling continuous-valued feature vectors corrupted by noise due to its analytical tractability. • Univariate normal distribution: where the mean and covariance are defined by: and the entropy is given by:

  3. The peak is at: • 66% of the area is within one ; 95% is within two ; 99% is within three . 07: WHITENING TRANSFORMATIONS MEAN AND VARIANCE • A normal distribution is completely specified by its mean and variance: • A normal distribution achieves the maximum entropy of all distributions having a given mean and variance. • Central Limit Theorem: The sum of a large number of small, independent random variables will lead to a Gaussian distribution.

  4. where  represents the mean (vector) and  represents the covariance (matrix). • Note the exponent term is really a dot product or weighted Euclidean distance. 07: WHITENING TRANSFORMATIONS MULTIVARIATE NORMAL DISTRIBUTIONS • A multivariate distribution is defined as: • The covariance is always symmetric and positive semidefinite. • How does the shape vary as a function of the covariance?

  5. 07: WHITENING TRANSFORMATIONS SUPPORT REGIONS • A support region is the obtained by the intersection of a Gaussian distribution with a plane. • For a horizontal plane, this generates an ellipse whose points are of equal probability density. • The shape of the support region is defined by the covariance matrix.

  6. 07: WHITENING TRANSFORMATIONS DERIVATION

  7. 07: WHITENING TRANSFORMATIONS IDENTITY COVARIANCE

  8. 07: WHITENING TRANSFORMATIONS UNEQUAL VARIANCES

  9. 07: WHITENING TRANSFORMATIONS OFF-DIAGONAL ELEMENTS

  10. 07: WHITENING TRANSFORMATIONS UNCONSTRAINED ELEMENTS

  11. 07: WHITENING TRANSFORMATIONS COORDINATE TRANSFORMATIONS • Why is it convenient to convert an arbitrary distribution into a spherical one? (Hint: Euclidean distance) • Consider the transformation: Aw=  -1/2 where  is the matrix whose columns are the orthonormal eigenvectors of  and  is a diagonal matrix of eigenvalues (=   t). Note that  is unitary. • What is the covariance of y=Awx? E[yyt] = (Awx)(Awx)t =( -1/2x)( -1/2x)t =  -1/2xxt-1/2 t =  -1/2 -1/2 t =  -1/2   t -1/2 t =  t -1/2  -1/2 (t) = I

  12. 07: WHITENING TRANSFORMATIONS MAHALANOBIS DISTANCE • The weighted Euclidean distance: is known as the Mahalanobis distance, and represents a statistically normalized distance calculation that results from our whitening transformation. • Consider an example using our Java Applet.

More Related