Multi -Attribute Spaces: Calibration for Attribute Fusion and Similarity Search

1. Multi-Attribute Spaces: Calibration for Attribute Fusion and Similarity Search Walter Scheirer, Neeraj Kumar, Peter N. Belhumeur, Terrance E. Boult, CVPR 2012 5th December 2012 University of Oxford

2. Attributes based image description 4-Legged White Male Orange Symmetric Asian Striped Ionic columns Beard Furry Classical Smiling Slide Courtesy: Neeraj Kumar

3. Attribute Classifiers Attribute and Simile Classifiers for Face Verification N. Kumar, A. C. Berg, P. N. Belhumeur, and S. K. Nayar ICCV 2009 FaceTracer: A Search Engine for Large Collections of Images with Faces N. Kumar, P. N. Belhumeur, and S. K. Nayar ICCV 2009

4. Attributes Fusion FaceTracer: “smiling asian men with glasses” Slide Courtesy: Neeraj Kumar

5. Score Normalization: Problem • Necessary to prevent high confidence for one attribute from dominating the results. • Ideal normalization technique should, • Normalize scores to a uniform range say, [0,1] • Assign perceptual quality to the scores. • Positive and negative distributions of different classifiers do not necessarily follow same distribution. • Fitting a Gaussian or any other distribution to scores satisfies condition 1 but doesn’t satisfy condition 2. Negative Scores Distributions Positive Scores Distributions

6. Score Normalization: Solution • Model distance between positive scores and the negative scores . • If we knew distribution of negative scores, we could do a • hypothesis test for each positive score using that distribution. • Unfortunately, we don’t know anything about overall negative distribution. But, we know something about tail of the negative score distribution.

7. Extreme Value Theory • Central Limit Theorem: • The “mean” of a sufficiently large iid random variables will be distributedaccording to Normal distribution • Extreme Value Theory: • The maximum of a sufficiently large iid random variable will be distributed according to Gumbell, Frechet or Weibull distribution. • If the values are bounded from above and below, the the values are distributed according to “Weibull” distribution.

8. Weibull Distribution • Weibull Distribution • PDF • CDF • k and λ are shape and location parameters respectively. PDF CDF

9. Extreme Value Theory: Application Overall Negative Score Distribution Maximum values of random variables Tail • Tail of negative scores can be seen as a collection of maxima of some random • variables. • Hence it follows Weibull distribution according to Extreme Value Theory.

10. W-score normalization: Procedure • For any classifier, • Fix the decision boundary on the scores • (Ideally this should be at score = 0 ) • Select maximum N (tail size) samples from • negative side of the boundary. • Fit a Weibull Distribution to these tail scores. • Renormalize scores using Cumulative Density Function (CDF) of this Weibull distribution.

11. Results: Dataset • “Labeled Faces In The Wild” dataset. • About 13,000 images of 5000 celebrities. • 75 different attribute classification scores available from • “Attribute and Simile Classifiers for Face Verification”. Kumar et al. ICCV 09. • Labeled Faces in the Wild: A Database for StudyingFace Recognition in Unconstrained Environments.

12. Results

13. Multi Attribute Fusion: • Joint score can be computed as multiplication of individual attribute probabilities. • Attributes may not be independent. • Low probability due to: • bad classifier • absence of images belonging to an attribute. • Instead of product, authors propose use l1 norm of probabilities as a fusion score.

14. Results

15. Similarity Search: • Given an image and a set of attributes, find nearest images. • Perceived difference between images in different ranges might be similar. • Distances between query attribute and its nearest neighbor needs to be normalized. • Normalize query attribute scores on query image. • Get nearest neighbor distances. • Fit Weibull distribution to distances.

16. Summary • Provides way of normalizing scores intuitively. • Provides way for combining attributes. • Relies on finding the right threshold and tail size. Requires fair bit of tuning.

17. Questions?