1 / 91

Object Recognition Through Reasoning About Functionality:

Object Recognition Through Reasoning About Functionality: A Survey of Related Work and Open Problems. Melanie Sutton University of the West Florida Pensacola, Florida. Louise Stark University of the Pacific Stockton, California. Function-Based Research. Dr. Louise Stark

denim
Download Presentation

Object Recognition Through Reasoning About Functionality:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Object Recognition Through Reasoning About Functionality: A Survey of Related Work and Open Problems Melanie Sutton University of the West Florida Pensacola, Florida Louise Stark University of the Pacific Stockton, California Dagstuhl Oct09

  2. Function-Based Research • Dr. Louise Stark University of the Pacific Stockton, CA Dagstuhl Oct09

  3. University of the Pacific Dagstuhl Oct09

  4. University of the Pacific Dagstuhl Oct09

  5. University of West Florida Pre/post-hurricane season…

  6. Seminar Goals • This seminar brings together scientists from disciplines such as computer science, neuroscience, robotics, developmental psychology, and cognitive science Dagstuhl Oct09

  7. Seminar Goals • Hope to further the knowledge • how the perception of form relates to object function • how intention and task knowledge (and hence function) aids in the recognition of relevant objects Dagstuhl Oct09

  8. Overview • Recognition based on functionality • Overview of GRUFF approach • Functionality in Related Disciplines • Open Problem Areas Dagstuhl Oct09

  9. Cognitive Psychology/Human Perception Artificial Intelligence Computer Vision Robotics Function-based Approaches Representations of object categories Human-robot interaction strategies Wayfinding Document/aerial image analysis Interpreting human motion Object recognition/categorization Formal representations of knowledge Machine learning techniques to automate reasoning Mapping of indoor environments Object detection Navigation/interaction plans Formalisms for autonomous robot control Dagstuhl Oct09

  10. Computer Vision? • Deriving meaningful descriptions of the • environment from images • Descriptions needed for • Recognition • Manipulation • Reasoning about objects Dagstuhl Oct09

  11. Generic Object Recognition • Minsky (1991) • Argued for the necessity of representing knowledge about functionality • “… rarely use a representation in an intentional vacuum, but we always have goals…” • “… we must classify things… according to what they can be used for.” Dagstuhl Oct09

  12. Motivation Parameterized Model Structural Model Could these be recognized? Dagstuhl Oct09

  13. GRUFF Generic Recognition Using Form and Function chair(cher) n. - a piece of furniture for one person to sit on Dagstuhl Oct09

  14. What is the goal? Develop alternative approaches to generic object recognition & manipulation - concentrate on man made objects (artifacts) Human artifacts – existence or non/existence of properties can be deduced by analyzing the shape of an object For any particular object category – there is some set of functional properties shared by ALL objects in that category. Dagstuhl Oct09

  15. Approach to the Problem • Derive the format of my function-based representation • Confirm feasibility of appoach test domain- • perfect input - planar face models • Expand the domains • Test real data • Interact to confirm functionality • Exploit contextual information Dagstuhl Oct09

  16. Knowledge in GRUFF is of three types: A category hierarchy which specifies superordinate / basic / subordinate categories furniture  chair  arm chair Functional properties that define each catgory (provides_sittable_surface, provides_stability,...) Knowledge primitives used to reason about shape (dimensions, relative orientation, ...) All organized into a "category definition tree" which is GRUFF's knowldge about the world. Dagstuhl Oct09

  17. Category Representation Tree Conventional Chair Provides Sittable Surface Provides Stable Support Dagstuhl Oct09

  18. We imagine the definition of a generic object category to be something like... straight_back_chair ::= provides_seating_surface + provides_stability + provides_back_support_surface and recognition is conceptualized as ... Provides_back_support provides_arm_support Provides_sittable_surface provides_stable_support Dagstuhl Oct09

  19. Shape-based Knowledge Primitives A functional requirement such as : provides_sittable_surface is implemented as a sequence of calls to shape-based operators. dimensions(shape_element, dimensions_type, range_parameters) relative_orientation(normal 1,normal 2, range_parameters) clearance(shape_element clearance_volume) Dagstuhl Oct09

  20. Knowledge Primitives Abstract shape reasoning • Metric dimensions (width, depth, height, area, contiguous surface, volume • Proximity • Relative orientation • Clearance • Stability • Enclosure Dagstuhl Oct09

  21. Knowledge Primitives Physical interaction reasoning • Change orientation • Apply force • Observe deformation Dagstuhl Oct09

  22. Evaluation Measures Value returned from knowledge primitive invocation 1.0 Evaluation Measure 0.0 least low high greatest ideal ideal Values of Shape Property Dagstuhl Oct09

  23. Combining Evidence • Combine required measurements using probabilistic AND (0-1) • Combine descendent subcategory node measure using probabilistic OR Dagstuhl Oct09

  24. Recognition Process • Category representation graph is control structure • Structural Constraint Propagation – subcategory nodes constrained by what was found for the parent Dagstuhl Oct09

  25. Recognition Stage 2 approaches 1. Check all known categories in the knowledge base 2. Confirm/deny object can/cannot function as a specified (sub)category Dagstuhl Oct09

  26. Valid Chairs Recognized by GRUFF Dagstuhl Oct09

  27. History of GRUFF Project Dagstuhl Oct09

  28. Context-based Reasoning GRUFF - Generic object recognition system Reasons about and generates plans for understanding 3D scenes of objects Extension to Context-based Reasoning - Determine significance of accumulated functional evidence to infer the existence of scene concepts Dagstuhl Oct09

  29. Functionality in the Large What makes an 'office' an office? A desk with at least one chair in close proximity. You categorize areas or workspaces by the functional configuration of the objects in the area. Dagstuhl Oct09

  30. Context-based Reasoning Name: Office Type: Category Function Verification Plan Realized by Potential Results Name: Infer Seating Areas Name: Infer Back Support Name: Infer worksurfaces Name: Provides potential seating Name: Provides potential worksurfaces Context-based Reasoning Shape-based Reasoning Dagstuhl Oct09

  31. What Did Change? • Multiple objects in scene • Relax functional requirements • Allow partial evidence Dagstuhl Oct09

  32. What Did Not Change? • Basic set of functional primitives • Organization of the representation • Categorization, not identification Dagstuhl Oct09

  33. Test Data Simulated data - Complete 3D models evaluated no occlusion surfaces - Partial 3D models derived from laser range finder simulation tool Real data - Stereo camera system generating range data (SRI's Small Vision System software) Dagstuhl Oct09

  34. Test Scenes Used in Context-based Reasoning Dagstuhl Oct09

  35. Test Scenes Used in Context-based Reasoning Dagstuhl Oct09

  36. Context-based Reasoning System Infer contextual relationships from accumulated functional evidence Provides potential worksurfaces Provides potential seating (back support and/or seating area) Dagstuhl Oct09

  37. What is the goal? Question – How do we recognize objects we have never previously encountered? - we don'thave a model (or do we?) Essentially- We categorize objects using some type of "model" Dagstuhl Oct09

  38. Earlier Work • Roberts • “Machine perception of three dimensional solids” 1965 • Analyze intensity image • Extract edge information • Match against library of geometric models • - “Model-based vision” paradigm • - “Single arbitrary view 3-D object recognition” paradigm Dagstuhl Oct09

  39. Earlier Work Binford “Survey of model-based image analysis systems” 1982 “The essential definition of object class is functional. … Object classes have an associated 3-D form: form equals function. … Dagstuhl Oct09

  40. Earlier Work Binford “Survey of model-based image analysis systems” 1982 “An object’s function is often a geometric function. The function of a room is to be an enclosing volume. … The function of a chair… is to be a flat surface at a comfortable height for sitting….” Dagstuhl Oct09

  41. Earlier Work • Winston, Binford, Katz and Lowry • “Learning physical descriptions from functional definitions, examples and precedents” 1984 • Discussed used of function-based definitions of object categories • Infinity of individual physical descriptions of objects in a category… • Single functional description to represent all (cup example) Dagstuhl Oct09

  42. Earlier Work • Brady, Agre, Braunegg and Connell • “The mechanics mate” 1985 • Connell and Brady • “Generating and generalizing models of visual objects” 1987 • Discussed relation between geometric structure and functional significance • Generalized structural description learned from sequence of examples Dagstuhl Oct09

  43. Earlier Work Minsky “The Society of Mind”, 1985 “… The solution is that we need to combine at least two different kinds of descriptions. On one side, we need structural descriptions for recognizing chairs when we see them. ” Dagstuhl Oct09

  44. Earlier Work Minsky “The Society of Mind”, 1985 “… On the other side we need functional descriptions in order to know what we can do with them… we need connections between parts of the chair structure and the requirements of the human body that those parts are supposed to serve. “ Dagstuhl Oct09

  45. Background • DiManzo, Trucco, Giunchiglia, Ricci • “FUR: Understanding Functional Reasoning”, 1989 • Utilized functional knowledge within an expert system framework • Primitives defined as individual expert systems that evaluate 3D information Dagstuhl Oct09

  46. Background • Rivlin and Rosenfeld • “Navigational Functionalities”, 1995 • Explored functionality as it relates to mobile robots • Navigating agent may classify objects in its environment in functional terms as “threat,” “landmark” and so on. Dagstuhl Oct09

  47. Cognitive Psychology/Human Perception Artificial Intelligence Computer Vision Robotics Function-based Approaches Representations of object categories Human-robot interaction strategies Wayfinding Document/aerial image analysis Interpreting human motion Object recognition/categorization Formal representations of knowledge Machine learning techniques to automate reasoning Mapping of indoor environments Object detection Navigation/interaction plans Formalisms for autonomous robot control Dagstuhl Oct09

  48. Artificial Intelligence • Two areas within AI that impact function-based research • Work on formal representations of knowledge about functionality • Application of machine learning techniques to automate the process of constructing function-based systems Dagstuhl Oct09

  49. Artificial Intelligence • AI approach developed greater formalism and depth than that in computer vision • Advantage as complexity of system requirements increases Dagstuhl Oct09

  50. Robotics • Incorporate best practices from other fields • Evolution • Service robots (controlled environment) • Interaction to confirm function • General navigational systems Dagstuhl Oct09

More Related