1 / 48

Finding Glass

Kenton McHenry Jean Ponce David Forsyth. Finding Glass. Background. Layer Seperation (Szleski, Avidan, and Aniandan, CVPR'00), (Levin, Zomet, and Weiss, CVPR'04).

harley
Download Presentation

Finding Glass

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Kenton McHenry Jean Ponce David Forsyth Finding Glass

  2. Background • Layer Seperation (Szleski, Avidan, and Aniandan, CVPR'00), (Levin, Zomet, and Weiss, CVPR'04) • 3D Structure (Hata, Saitoh, Kumamura and Kaida, ICPR'96) (Ben-Ezra and Nayar, ICCV'03) (Miyazaki, Kagesawa and Ikeuchi, ICCV'03) (Murase, ICCV'90) • Recognition (Osadchy, Jacobs, and Ramamoorthi, ICCV'03) • Segmentation (Singh and Huang, CVPR'03)

  3. (Adelson and Anandan, AAAI'90) • 0 < a≤ 1 • e ≥ 0 I = aIB+ e

  4. Classifying Junctions Non-Reversing: transparency, ambiguous depth ordering Single-Reversing: transparency Double-Reversing: no transparency

  5. (Singh and Huang, CVPR'03)

  6. (Singh and Huang, CVPR'03)

  7. Our Goal

  8. The Background • The appearance of a glass object changes with the background (i.e. the scene w/o any transparent objects) • We have seen how knowledge of the background can be extremeley useful in reconstructing transparent surfaces • Ideal situation: know the background, use background subtraction

  9. Glass Objects and their Edges Why? • Highlights • Mirrors • Hysteresis

  10. Adelson et al Revisited • Though they focus on junctions they are classifying edges • The proposed rules are binary cues between a transparent object and its background

  11. Proposed Method • Break edges into small segments and classify them based on the information from the two sides • Properties of glass: transparency, refraction and reflection

  12. Cues • Transparency • Color Similarity • Overlay Consistency • Refraction • Texture Distortion • Blurring • Reflection • Highlights

  13. Color Similarity • (HSV) Hue • (HSV) Saturation

  14. Overlay Consistency

  15. Texture Distortion • Filer Bank: 2 scales, 6 orientations (0,p)

  16. Blurring • DCT • Shift in mean in frequency space

  17. Highlights • Highlights on smooth shiny surfaces tend to have a profile with a sharp spike (Healey and Binford, '87), (Nayar, Ikeuchi and Kanade, '91)

  18. Highlights • Iteratively fit a line to perimeter (starting from threshold of 1.0) • Plot line fit errors

  19. Highlights

  20. Single Classifier • 5 cues provide 6 values • SVM with Gaussian kernel • Must be conservative with false positives • Classifier can achieve high accuracy on training data • Move hyperplane until true positives < 30%

  21. Multiple Classifiers • If we were to consider the 6 values as logical propositions we could write: glass ⇐ similar_color ∧ high_alpha ∧ (low_emmission ∨ highlight ∨ smoother ∨ distortion)

  22. Multiple Classifiers • We can re-write the previous statement as four different statements of three propositions: glass ⇐ similar_color ∧ high_alpha ∧ low_emmission glass ⇐ similar_color ∧ high_alpha ∧ highlight glass ⇐ similar_color ∧ high_alpha ∧ smoother glass ⇐ similar_color ∧ high_alpha ∧ distortion

  23. Multiple Classifiers • Each proposition is a seperatley trained classifier of lower dimension • Combining the sub-classifiers • Logical OR • Weighted Sum • Exponential Model

  24. Global Integration • Due to conservativeley built classifiers we will have few positives • Hysteresis: connect positves along a common edge • Snakes (Kass, Witkin, Terzopoulos, '87)

  25. Experiments • Training Set: 15 images, 6 with glass objects in front of various backgrounds, 9 with no glass objects • 333 positive examples • 4581 negative examples • Test Set: 50 images, 35 with glass objects, 15 with no glass objects at all

  26. Experiments Precision 68.76% 56.04% 58.78% 56.04% 73.70% Single SVM Multiple SVM's + OR Multiple SVM's + Weighted Sum Multiple SVM's + Exponential Model Multiple SVM's + Weighted Sum (sampled)

  27. Results

  28. Results

  29. Results

  30. Results

  31. Classifying Regions as Glass • We need not restrict ourselves to regions around edges • Given two regions we ask the question “is one region a glass covered version of the other?”

  32. Over Segmentation • We want regions of similar material (Felzenszwalb and Huttenlocher, '04) • Can adjust size of super-pixels (degree of over-segmentation) with smaller k values • Use color, texture, and edgels to set weights

  33. Discrepency • We use our previous classifier as a measure of how much two regions don't belong two the same material (i.e. glass and not glass) • Use distance from seperating hyperplane (Platt, '00) • Large values: far on the postive glass side • Small values (negative): far on the not glass side • Reasonable if data takes a normal distribution • Drop blur cue since DCT can't be done on non-rectangular regions.

  34. Ambiguities • Discrepency is high for a material and a glass covered version of that material, but also for two completley different materials • Above example has two possible segmentations

  35. Affinity Aij = 1 – aij / p

  36. Affinity • Because of refraction most straight background edges that pass through the glass will appear broken • Edges from glass contour ussually the longest smoothest edges in the area

  37. Affinity

  38. Certainty of Discrepency/Affinity • High discrepency: likely different materials • Low discrepency: cannot ascertain whether one regions is glass and the other is background • High affinity: likely same material • Low affinity: not very informative, edge path may just have been broken

  39. Objective Function • We wish to maximize our measures • First term: maximize discrepency between glass and other stuff • Second term: maximize affinity in the glass • Third term: minimize affinities between glass and other • Combinatorial problem!

  40. Relaxed Objective Function • Relax region constraints • Treat pixels as a sampling of an underlying continuous function

  41. Geodesic Active Contours

  42. Curve Evolution

  43. Experiments Precision 68.76% 56.04% 58.78% 56.04% 73.70% 77.03% Single SVM Multiple SVM's + OR Multiple SVM's + Weighted Sum Multiple SVM's + Exponential Model Multiple SVM's + Weighted Sum (sampled) Proposed Method

  44. Results

  45. Results

  46. Results

  47. Results

  48. Results

More Related