520 likes | 691 Views
Window-based models for generic object detection. Mei-Chen Yeh 04/24/2012. Object Detection. Find the location of an object if it appear in an image Does the object appear? Where is it?. Viola-Jones face detector. P. Viola and M. J. Jones. Robust Real-Time Face Detection. IJCV 2004.
E N D
Window-based models for generic object detection Mei-Chen Yeh 04/24/2012
Object Detection • Find the location of an object if it appear in an image • Does the object appear? • Where is it?
Viola-Jones face detector P. Viola and M. J. Jones. Robust Real-Time Face Detection. IJCV 2004.
Viola-Jones Face Detector: Results Paul Viola, ICCV tutorial
Consumer application: iPhoto 2009 http://www.apple.com/ilife/iphoto/ Slide credit: Lana Lazebnik
Consumer application: iPhoto 2009 • Things iPhoto thinks are faces Slide credit: Lana Lazebnik
Consumer application: iPhoto 2009 • Can be trained to recognize pets! http://www.maclife.com/article/news/iphotos_faces_recognizes_cats Slide credit: Lana Lazebnik
Challenges scale illumination view point variation Michelangelo 1475-1564 occlusion Magritte, 1957 Slide credit: Fei-Fei Li
Challenges deformation background clutter Xu, Beihong 1943 Klimt, 1913 Slide credit: Fei-Fei Li
Basic framework • Build/train object model • Choose a representation • Learn or fit parameters of model / classifier • Generate candidates in new image • Score the candidates
Basic framework • Build/train object model • Choose a representation • Learn or fit parameters of model / classifier • Generate candidates in new image • Score the candidates
Window-based modelsBuilding an object model Given the representation, train a binary classifier Face/non-face Classifier Yes, face. No, not a face.
Basic framework • Build/train object model • Choose a representation • Learn or fit parameters of model / classifier • Generate candidates in new image • Score the candidates
Window-based modelsGenerate and score candidates • Scans the detector at multiple locations and scales face/non-face Classifier
Car/non-car Classifier Window-based object detection • Training: • Obtain training data • Define features • Define classifier • Given new image: • Slide window • Score by classifier Training examples Feature extraction
Viola-Jones detection approach • Viola and Jones’ face detection algorithm • The first object detection framework to provide competitive object detection rates in real-time • Implemented in OpenCV • Components • Features • Haar-features • Integral image • Learning • Boosting algorithm • Cascade method
Haar-features (1) • The difference between pixels’ sum of the white and black areas
Haar-features (2) • Capture the face symmetry
Haar-features (3) Type A Four types of haar features Can be extracted at any location with any scale! A 24x24 detection window
24 24 Haar-features (4) • Too many features! • location, scale, type • 180,000+ possible features associated with each 24 x 24 window • Not all of them are useful! • Speed-up strategy • Fast calculation of haar-features • Selection of good features AdaBoost
3 5 8 12 15 • 5 8 11 16 22 28 • 9 14 18 24 31 39 • 1 2 3 4 3 • 2 1 2 2 3 • 2 1 1 1 2 Image Integral image Integral image (1) Sum of pixel values in the blue area Example: Time complexity?
A, B: 2 rectangles => C: 3 rectangles => D: 4 rectangles => Integral image (2) a = sum(1) b = sum(1+2) c = sum(1+3) d = sum(1+2+3+4) 1 2 a b 3 4 c d Sum(4) = ? d + a – b – c Four-point calculation! 6-point 8-point 9-point
Feature selection • A very small number of features can be combined to from an effective classifier! • Example: The 1st and 2nd features selected by AdaBoost
Feature selection • A weak classifier h f1 f2 f1 > θ (a threshold) => Face! f2 ≤ θ (a threshold) => Not a Face! 1 if fi > θ h = 0 otherwise
Feature selection • Idea: Combining several weak classifiers to generate a strong classifier …… α1 α3 α2 αT α1h1+ α2h2 + α3h3 + … + αThT Tthresold >< ~performance of the weak classifier on the training set weak classifier (feature, threshold) h1 = 1 or 0
Feature selection • Training Dataset • 4916 face images • non-face images cropped from 9500 images non-face images positive samples negative samples
AdaBoost • Each training sample may have different importance! • Focuses more on previously misclassified samples • Initially, all samples are assigned equal weights • Weights may change at each boosting round • misclassified samples => increase their weights • correctly classified samples => decrease their weights
Boosting illustration: 2D case Weak Classifier 1 Slide credit: Paul Viola
Boosting illustration Weights Increased
Boosting illustration Weak Classifier 2
Boosting illustration Weights Increased
Boosting illustration Weak Classifier 3
Boosting illustration Final classifier is a combination of weak classifiers
Viola-Jones detector: AdaBoost • Want to select the single rectangle feature and threshold that best separates positive (faces) and negative (non-faces) training examples, in terms of weighted error. Resulting weak classifier: For next round, reweight the examples according to errors, choose another filter/threshold combo. … Outputs of a possible rectangle feature on faces and non-faces. Kristen Grauman
Initial weights for each data point AdaBoost -∞ ∞ decreased decreased increased fi ~ error rate error↘ α↗ misclassified
AdaBoost ……
Learning the classifier • Initialize equal weights to training samples • For T rounds • normalize the weights • select the best weak classifier in terms of the weighted error • update the weights(raise weights to misclassified samples) • Linearly combine these T weak classifiers to form a strong classifier
AdaBoost Algorithm Start with uniform weights on training examples {x1,…xn} For T rounds Evaluate weighted error for each feature, pick best. Re-weight the examples: Incorrectly classified -> more weight Correctly classified -> less weight Final classifier is combination of the weak ones, weighted according to error they had. Freund & Schapire 1995
Feature Selection: Results First two features selected
Boosting: pros and cons • Advantages of boosting • Integrates classification with feature selection • Complexity of training is linear in the number of training examples • Flexibility in the choice of weak learners, boosting scheme • Testing is fast • Easy to implement • Disadvantages • Needs many training examples • Often found not to work as well as an alternative discriminative classifier, support vector machine (SVM) • especially for many-class problems Slide credit: Lana Lazebnik
Viola-Jones detection approach • Viola and Jones’ face detection algorithm • The first object detection framework to provide competitive object detection rates in real-time • Implemented in OpenCV • Components • Features • Haar-features • Integral image • Learning • Boosting algorithm • Cascade method
Even if the filters are fast to compute, each new image has a lot of possible windows to search. • How to make the detection more efficient?
1 2 3 Cascade method Strong Classifier = (α1h1+ α2h2) + (…)+ (…+ αThT) Tthresold >< Most windows contain no face! Rejects negative windows in an early stage!