1 / 52

Pictorial Structures and Distance Transforms

03/31/11. Pictorial Structures and Distance Transforms. Computer Vision CS 543 / ECE 549 University of Illinois Ian Endres. Goal: Detect all instances of objects. Cars. Faces. Cats. Object model: last class. Statistical Template in Bounding Box Object is some (x,y,w,h) in image

deirdre
Download Presentation

Pictorial Structures and Distance Transforms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 03/31/11 Pictorial Structures and Distance Transforms Computer Vision CS 543 / ECE 549 University of Illinois Ian Endres

  2. Goal: Detect all instances of objects Cars Faces Cats

  3. Object model: last class • Statistical Template in Bounding Box • Object is some (x,y,w,h) in image • Features defined wrt bounding box coordinates Image Template Visualization Images from Felzenszwalb

  4. Last class: sliding window detection … …

  5. Last class: statistical template • Object model = log linear model of parts at fixed positions ? +3 +2 -2 -1 -2.5 = -0.5 > 7.5 Non-object ? +4 +1 +0.5 +3 +0.5 = 10.5 > 7.5 Object

  6. When are statistical templates useful? Caltech 101 Average Object Images

  7. Deformable objects Images from Caltech-256 Slide Credit: Duan Tran

  8. Deformable objects Images from D. Ramanan’s dataset Slide Credit: Duan Tran

  9. Object models: this class • Articulated parts model • Object is configuration of parts • Each part is detectable Images from Felzenszwalb

  10. Parts-based Models Define object by collection of parts modeled by • Appearance • Spatial configuration Slide credit: Rob Fergus

  11. How to model spatial relations? • Star-shaped model Part Part Part Root Part Part

  12. How to model spatial relations? • Star-shaped model Part = Part X Part X Root ≈ Part Part X

  13. How to model spatial relations? • Tree-shaped model

  14. How to model spatial relations? • Many others... O(N6) O(N2) O(N3) O(N2) Fergus et al. ’03 Fei-Fei et al. ‘03 Leibe et al. ’04, ‘08Crandall et al. ‘05 Fergus et al. ’05 Crandall et al. ‘05 Felzenszwalb & Huttenlocher ‘05 Csurka ’04 Vasconcelos ‘00 Bouchard & Triggs ‘05 Carneiro & Lowe ‘06 from [Carneiro & Lowe, ECCV’06]

  15. Today’s class • Tree-shaped model • Example: Pictorial structures • FelzenszwalbHuttenlocher 2005 • Optimization with Dynamic Programming • Distance Transforms

  16. Pictorial Structures Model Part = oriented rectangle Spatial model = relative size/orientation Felzenszwalb and Huttenlocher 2005

  17. Part representation • Background subtraction

  18. Pictorial Structures Model Geometry likelihood Appearance likelihood

  19. Modeling the Appearance • Any appearance model could be used • HOG Templates, etc. • Here: rectangles fit to background subtracted binary map • Train a detector for each part independently Geometry likelihood Appearance likelihood

  20. Pictorial Structures Model Minimize Energy (-log of the likelihood): Appearance Cost Geometry Cost

  21. Optimization – Brute Force Appearance Cost Deformation Cost H T F

  22. Optimization H T F

  23. Optimization - Dynamic Programming H T F

  24. Optimization - Dynamic Programming H T F

  25. Optimization - Dynamic Programming H T F

  26. Optimization - Dynamic Programming H T F

  27. Optimization - Complexity Each part can take N locations ? For all lT? H Overall? (for K parts) T F

  28. Distance Transform • For each pixel p, how far away is the nearest pixel q of set S • G is often the set of edge pixels G: black pixels p1 q1 q2 p2

  29. Computing the Distance Transform • 1-Dimension

  30. Computing the Distance Transform • Extending to N-Dimensions • Savings can be extended from 1-D to N-D cases if deformation cost is separable:

  31. Distance Transform - Applications • Set distances – e.g. Hausdorff Distance • Image processing – e.g. Blurring • Robotics – Motion Planning • Alignment • Edge images • Motion tracks • Audio warping • Deformable Part Models (of course!)

  32. Generalized Distance Transform • Original form: • General form: • m(q): arbitrary function sampled at discrete points • For each p, find a nearby q with small m(q) • For original DT: • For part models: • For some deformation costs,

  33. How do we do it? • Key idea: Construct lower “envelope” of data • Given envelope, compute f(p) in O(N) time • Goal: Compute envelope in O(N) time

  34. Computing the Lower Envelope • Key idea: Keep track of intersection points • fi(x), fj(x) only intersect once (let i<j) Start(1) = -Inf End(1) = Inf

  35. Computing the Lower Envelope • Key idea: Keep track of intersection points • fi(x), fj(x) only intersect once (let i<j) Start(1) = -Inf End(1) = 3 Start(2) = 3 End(2) = Inf X

  36. Computing the Lower Envelope • Key idea: Keep track of intersection points • fi(x), fj(x) only intersect once (let i<j) Start(1) = -Inf End(1) = 3 Start(2) = 3 End(2) = 4 Start(3) = 4 End(3) = Inf X

  37. Computing the Lower Envelope • Key idea: Keep track of intersection points • fi(x), fj(x) only intersect once (let i<j) Start(1) = -Inf End(1) = 3 Start(2) = 3 End(2) = 4 Start(3) = 4 End(3) = Inf X

  38. Computing the Lower Envelope • Key idea: Keep track of intersection points • fi(x), fj(x) only intersect once (let i<j) Start(1) = -Inf End(1) = 3 Start(2) = 3 End(2) = 4 Start(3) = [] End(3) = [] Start(4) = 4 End(4) = Inf X

  39. Computing the Lower Envelope • Key idea: Keep track of intersection points • fi(x), fj(x) only intersect once (let i<j) Start(1) = -Inf End(1) = 3 Start(2) = 3 End(2) = 4 Start(3) = [] End(3) = [] Start(4) = 4 End(4) = Inf X

  40. Computing the Lower Envelope • Key idea: Keep track of intersection points • fi(x), fj(x) only intersect once (let i<j) Start(1) = -Inf End(1) = 3 Start(2) = 3 End(2) = 4 Start(3) = [] End(3) = [] Start(4) = 4 End(4) = Inf X

  41. Computing the Lower Envelope • Key idea: Keep track of intersection points • fi(x), fj(x) only intersect once (let i<j) Start(1) = -Inf End(1) = 3 Start(2) = [] End(2) = [] Start(3) = [] End(3) = [] Start(4) = 3 End(4) = Inf X

  42. Computing the Lower Envelope • Key idea: Keep track of intersection points • fi(x), fj(x) only intersect once (let i<j) Start(1) = -Inf End(1) = 2.6 Start(2) = [] End(2) = [] Start(3) = [] End(3) = [] Start(4) = 2.6 End(4) = Inf X

  43. Computing the Lower Envelope • Key idea: Keep track of intersection points • fi(x), fj(x) only intersect once (let i<j) Start(1) = -Inf End(1) = 2.6 Start(2) = [] End(2) = [] Start(3) = [] End(3) = [] Start(4) = 2.6 End(4) = 3.5 Start(5) = 3.5 End(5) = Inf X

  44. Is This O(N)? • Yes! • N parabolas are added • For each addition, may remove up to O(N) parabolas • But, each parabola can only be deleted once • Thus, O(N) additions, and at most O(N) deletions

  45. What distances can we use? • Quadratic (with linear shift) • Abs. diff • Min-composition • Bounded (Requires extra bookkeeping)

  46. Back to the Pictorial structures model Problem: May need to infer more than one configuration • May be more than one object • Report scores for every root position, apply NMS • Optimal solution may be incorrect • Sampling • Sample root node, then each node given parent, until all parts are sampled

  47. Sample poses from likelihood and choose best match with Chamfer distance to map

  48. Results for person matching

  49. Results for person matching - Mistakes • Ambiguous Background subtracted image

  50. Recently enhanced pictorial structures BMVC 2009

More Related