1 / 98

My Group’s Current Research on Image Understanding

My Group’s Current Research on Image Understanding. An image-understanding task. Low -level vision. Color, Shape, Texture. Low-level vision. Simple Segmentation. Color, Shape, Texture. Low-level vision. Simple Segmentation. Color, Shape, Texture. Object recognition. Low-level vision.

erelah
Download Presentation

My Group’s Current Research on Image Understanding

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. My Group’s Current Research on Image Understanding

  2. An image-understanding task

  3. Low-level vision

  4. Color, Shape, Texture Low-level vision

  5. Simple Segmentation Color, Shape, Texture Low-level vision

  6. Simple Segmentation Color, Shape, Texture Object recognition Low-level vision

  7. High-level perception Simple Segmentation Color, Shape, Texture Object recognition Low-level vision

  8. High-level perception Simple Segmentation Color, Shape, Texture Object recognition Pattern recognition Low-level vision

  9. High-level perception Simple Segmentation Color, Shape, Texture Object recognition Pattern recognition Analogy-making Low-level vision

  10. High-level perception “Meaning” Simple Segmentation Color, Shape, Texture Object recognition Pattern recognition Analogy-making Low-level vision

  11. High-level perception “Meaning” ??? Simple Segmentation Color, Shape, Texture Object recognition Pattern recognition Analogy-making Low-level vision

  12. High-level perception “Meaning” Simple Segmentation Color, Shape, Texture Object recognition Pattern recognition Analogy-making Low-level vision The “SEMANTIC GAP’

  13. High-level perception “Meaning” Simple Segmentation Color, Shape, Texture Object recognition Pattern recognition Analogy-making Low-level vision HMAX model of visual cortex Riesenhuber, Poggio, et al. The “SEMANTIC GAP’

  14. High-level perception “Meaning” Active Symbol Architecture for high-level perception Hofstadter et al. Simple Segmentation Color, Shape, Texture Object recognition Pattern recognition Analogy-making Low-level vision HMAX model of visual cortex Riesenhuber, Poggio, et al. The “SEMANTIC GAP’

  15. High-level perception “Meaning” Active Symbol Architecture for high-level perception Hofstadter et al. Simple Segmentation Color, Shape, Texture Object recognition Pattern recognition Analogy-making Low-level vision HMAX model of visual cortex Riesenhuber, Poggio, et al. The “SEMANTIC GAP’

  16. The HMAX model for object recognition (Riesenhuber, Poggio, Serre, et al.)

  17. Streetscenes “scene understanding” system(Bileschi, 2006) Recognition Phase … 1. Densely tile the image with windows of different sizes. 2. HMAX features are computed in each window. 3. The features in each window are given as input to the trained support vector machine. 4. If the SVM returns a score above a learned threshold, then the object is said to be “detected” . …

  18. Object detection (here, “car”) with HMAX model (Bileschi, 2006)

  19. Some limitations of the Streetscenes approach to scene understanding

  20. Some limitations of the Streetscenes approach to scene understanding • Requires exhaustive search for object identification and localization

  21. Some limitations of the Streetscenes approach to scene understanding • Requires exhaustive search for object identification and localization Exhaustive search over:

  22. Some limitations of the Streetscenes approach to scene understanding • Requires exhaustive search for object identification and localization Exhaustive search over: • Window size and location in the image

  23. Some limitations of the Streetscenes approach to scene understanding • Requires exhaustive search for object identification and localization Exhaustive search over: • Window size and location in the image • Object categories (e.g., car, pedestrian, tree, etc.)

  24. Some limitations of the Streetscenes approach to scene understanding • Requires exhaustive search for object identification and localization Exhaustive search over: • Window size and location in the image • Object categories (e.g., car, pedestrian, tree, etc.) Exhaustive use of HMAX features in each window

  25. Does not recognize spatial and abstract relationships among objects for whole scene understanding

  26. Does not recognize spatial and abstract relationships among objects for whole scene understanding • Has no prior knowledge about object categories and their place in “conceptual space”

  27. Does not recognize spatial and abstract relationships among objects for whole scene understanding • Has no prior knowledge about object categories and their place in “conceptual space” • HMAX model is completely feed-forward; no feedback to allow context to aid in scene understanding.

  28. Goal of our project • Perform whole-scene interpretation without exhaustive search. • Incorporate conceptual knowledge • Allow feedforward and feedback modes to interact

  29. A Simple Semantic Network (or “Ontology”) “Dog walking” Person Dog leash holds attached to action action walking

  30. But... http://www.dogasaur.com/blog/wp-content/uploads/2011/04/dogwalker.jpg

  31. But... http://www.vet.k-state.edu/depts/development/lifelines/images/dog_jog_1435.jpg

  32. “Dog walking” Person Dog leash holds attached to Dog Group action action running walking

  33. “Dog walking” Person Dog leash holds attached to Dog Group action action Allowing “conceptual slippage” running walking

  34. But... http://3.bp.blogspot.com/_1YuoCTv4oKQ/S71jUDm7kOI/AAAAAAAAAak/jz4Pg7zzzQ8/s1600/23743577.JPG

  35. http://lh3.ggpht.com/-ZZrYWeBFTjo/SFQH_0ijwaI/AAAAAAAABjA/8nwryW2BmEw/IMG_0356.JPGhttp://lh3.ggpht.com/-ZZrYWeBFTjo/SFQH_0ijwaI/AAAAAAAABjA/8nwryW2BmEw/IMG_0356.JPG

  36. “Dog walking” Tail holds attached to leash Dog Group Person Dog action Cat action walking running Iguana

  37. But... http://www.mileanhour.com/post/Dog-walking-bike.aspx

  38. http://cl.jroo.me/z3/Z/e/C/d/a.aaa-Thus-walking-dog.png

  39. ttp://thedaemon.com/images/DARPA_Segue_Dog.jpg

  40. http://www.bikeforest.com/product45422.jpg

  41. http://www.k9ring.com/blog/image.axd?picture=2010%2F3%2Fwalking_dog_from_car.jpghttp://www.k9ring.com/blog/image.axd?picture=2010%2F3%2Fwalking_dog_from_car.jpg

  42. http://www.guy-sports.com/fun_pictures/dog_walking_helicopter.jpghttp://www.guy-sports.com/fun_pictures/dog_walking_helicopter.jpg

  43. http://static.themetapicture.com/media/funny-dog-walking-horse-leash.jpghttp://static.themetapicture.com/media/funny-dog-walking-horse-leash.jpg

  44. http://macwetblog.files.wordpress.com/2012/05/dog-walking.jpghttp://macwetblog.files.wordpress.com/2012/05/dog-walking.jpg

  45. “Dog walking” Tail Person Dog leash Helicopter Segue-ing Biking Driving Car holds attached to Dog Group action action Cat running Iguana walking Horse Treadmill-ing

  46. Active Symbol Architecture(Hofstadter et al., 1995)

  47. Active Symbol Architecture(Hofstadter et al., 1995) • Basis for • Copycat (analogy-making), Hofstadter & Mitchell • Tabletop (anlaogy-making), Hofstadter & French • Metacat(analogy-making and self-awareness), Hofstadter & Marshall and many others…

  48. Semantic network Active Symbol Architecture(Hofstadter et al., 1995) Perceptual agents (codelets) are “active symbols” Workspace Temperature

  49. Petacat:(Descendant of Copycat, part of the PetaVisionproject)Integration of Active Symbol Architecture and HMAX Initial task: Decide if image is an instance of “taking a dog for a walk”, and if so, how good an instance it is.

More Related