1 / 48

Features-based Object Recognition

This thesis defense presentation by Pierre Moreels from the California Institute of Technology focuses on the recognition of individual objects in the category of cars using feature-based methods. The presentation covers problem setup, features, a coarse-to-fine algorithm, probabilistic modeling, experiments, and conclusions.

damore
Download Presentation

Features-based Object Recognition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Features-based Object Recognition Pierre Moreels California Institute of Technology Thesis defense, Sept. 24, 2007

  2. The recognition continuum Categories cars means of transportation Individual objects BMW logo variability

  3. Applications Help Daiki find his toys ! Identification, Security. Autonomous navigation

  4. Outline • Problem setup • Features • Coarse-to-fine algorithm • Probabilistic model • Experiments • Conclusion

  5. The detection problem New scene (test image) … Models from database Find models and their pose (location, orientation…)

  6. Hypotheses – models + positions 1 2 New scene (test image) … Models from database Θ = affine transformation

  7. Matching features New scene (test image) … Models from database • Set of correspondences = assignment vector

  8. Features detection

  9. Image characterization by features • Features = high information content ‘locations in the image where the signal changes two-dimensionally’ C.Schmid • Reduce the volume of information • [Sobel 68] • Diff of Gaussians [Crowley84] • [Harris 88] • [Foerstner94] • Entropy [Kadir&Brady01] edge strength map features

  10. Correct vs incorrect descriptors matches 5 6 2 1 Mutual Euclidean distances in appearance space of descriptors 7 4 8 3 • - Pixels intensity within a patch • - Steerable filters [Freeman1991] • SIFT [Lowe1999,2004] • Shape context [Belongie2002] • - Spin [Johnson1999] • - HOG [Dalal2005]

  11. Stability with respect to nuisances • Which detector / descriptor combination is best for recognition ?

  12. Past work on evaluation of features • Use of flat surfaces, ground truth easily established • In 3D images appearance changes more ! [Schmid&Mohr00] [Mikolajczyk&Schmid 03,05,05]

  13. Database : 100 3D objects

  14. Testing setup [Moreels&Perona ICCV05, IJCV07] Used by [Winder, CVPR07]

  15. Results – viewpoint change No ‘background’ images Mahalanobis distance

  16. 2D vs. 3D Ranking of detectors/descriptors combinations are modified when switching from 2D to 3D objects

  17. Features matching algorithm

  18. Features assignments New scene (test image) Interpretation models from database . . . . . .

  19. Coarse-to-fine strategy my car • We do it every day ! Search for my place : Los Angeles area – Pasadena – Loma Vista - 1351

  20. Coarse-to-fine example [Fleuret & Geman 2001,2002] Face identification in complex scenes Coarse resolution Intermediate resolution Fine resolution

  21. Coarse-to-Fine detection • Progressively narrow down focus on correct region of hypothesis space • Reject with little computation cost irrelevant regions of search space • Use first information that is easy to obtain • Simple building blocks organized in a cascade • Probabilistic interpretation of each step

  22. Coarse data : prior knowledge • Which objects are likely to be there, which pose are they likely to have ? unlikely situations

  23. Model voting Search tree (appearance space – leaves = database features) 4 votes 2 votes 0 vote New scene (test image) … Models from database

  24. Use of rich geometric information (x1,y1,s1,1) (x2,y2,s2,2) • Transform predicted by this match: • x = x2-x1 • y = y2-y1 • s = s2 / s1 •  = 2 - 1 Each match is represented by a dot in the space of 2D similarities (Hough space) s  y x [Lowe1999,2004]

  25. Coarse Hough transform • Prediction of position of model center after transform • The space of transform parameters is discretized into ‘bins’ • Coarse bins to limit boundary issues and have a low false-alarm rate for this stage • We count the number of votes collected by each bin. Model correct transformation Test scene

  26. d d d Correspondence or clutter ? PROSAC • Similar to RANSAC – robust statistic for parameter estimation • Priority to candidates with good quality of appearance match • 2D affine transform : 6 parameters  each sample contains 3 candidate correspondences. Output of PROSAC : pose transformation + set of features correspondences [Fischler 1973] [Chum&Matas 2005]

  27. Probabilistic model

  28. Generative model

  29. Recognition steps

  30. Score of an extended hypothesis Features assignments observed features geometry + appearance Hypothesis: model + position database of models constant Votes per model pose bin (Hough transform) Votes per model Consistency (after PROSAC) Prior on model and poses Prior on assignments (before actual observations)

  31. Consistency Consistency between observations and predictions from hypothesis Common-frame approximation : parts are conditionally independent once reference position of the object is fixed. [Lowe1999,Huttenlocher90,Moreels04] model m Constellation model Common-frame position of model m

  32. Consistency Consistency between observations and predictions from hypothesis foreground features ‘null’ assignments appearance geometry appearance geometry Consistency - appearance Consistency - geometry

  33. Learning foreground & background densities • Ground truth pairs of matches are collected • Gaussian densities, centered on the nomimal value that appearance / pose should have according to H • Learning background densities is easy: match to random images. [Moreels&Perona, IJCV, 2007]

  34. Experiments

  35. An example Hough bins Model voting

  36. An example Probabilistic scores After PROSAC

  37. Efficiency of coarse-to-fine processing

  38. Giuseppe Toys database – Models 61 objects, 1-2 views/object

  39. Giuseppe Toys database – Test scenes 141 test scenes

  40. Home objects database – Models 49 objects, 1-2 views/object

  41. Home objects database – Test scenes 141 test scenes

  42. Results – Giuseppe Toys database - undetected objects: features with poor appearance distinctiveness index to incorrect models + • Lower false alarm • rate • more systematic • verification of • geometry consistency • - more consistent • verification of • geometric consistency Lowe’99,’04

  43. Results – Home objects database

  44. Failure mode Test image hand-labeled before the experiments

  45. Test – Text and graphics

  46. Test – no texture

  47. Test – Clutter

  48. Conclusions • Coarse-to-fine strategy prunes irrelevant search branches at early stages. • Probabilistic interpretation of each step. • Higher performance than Lowe, especially in cluttered environment. • Front end (features) needs more work for smooth or shiny surfaces.

More Related