An Ontology for Qualitative Description of Images

1. An Ontology for Qualitative Description of Images Zoe Falomir, Ernesto Jiménez-Ruiz, Lledó Museros, M. Teresa Escrig Cognition for Robotics Research (C4R2) Temporal Knowledge Base Group (TKBG) University Jaume I, Castellón (SPAIN)

2. Motivation (I) • Our group is applying Freksa’s Double Cross Orientation model to robotic navigation indoors. • Our robots use a laser sensor to find the landmarks of a room which are its corners and the corners of the obstacles inside the room. • Problem: sometimes a robot tries to localize itself inside a room and the geometry of the detected landmarks and its relative situation wrt the other landmarks is not enough to solve ambiguous situations. • Solution: to describe visually the landmarks of the room in order to differentiate easily between them. C1 C2 Spatial and Temporal Reasoning for Ambient Intelligence Systems at COSIT 2009

3. Qualitative Image Description Ontology Motivation (II) • Our approach describes qualitatively any image, by describing: • the visual features (shape and colour) and • the spatial features (orientation and topology) of the objects contained in an image. • An ontology provides our qualitative description: • A formal representation of the knowledge inside the robot • A standard language to exchange information between agents • New information inferred by the reasoners Spatial and Temporal Reasoning for Ambient Intelligence Systems at COSIT 2009

4. Index 1. Qualitative Description of Images 1.1. Approach 1.2. Models of Shape, Colour, Topology and Orientation 1.3. Structure of the Description 1.4. A Case of Study 2. Ontology 2.1. Terminological Knowlege Box (T-Box) 2.2. Assertional Knowledge Box (A-Box) 3. Results 3.1. Approach 3.2. New Knowledge Inferred from the Case of Study 4. Conclusion and Future Work Spatial and Temporal Reasoning for Ambient Intelligence Systems at COSIT 2009

5. 1. Qualitative Description of Images: 1.1. Approach Colour graph-based segmentation Qualitative Models of Shape, Colour, Topology and Orientation Image Processing Algorithms Qualitative Image Description Spatial and Temporal Reasoning for Ambient Intelligence Systems at COSIT 2009

6. 1. Qualitative Description of Images: 1.2.Models of Shape, Colour, Topology and Orientation Qualitative Shape of relevant point j: <KEC(j), A(j) or TC(j), L(j), C(j)> KEC: {line-line, line-curve, curve-line, curve-curve, curvature-point} A: {very-acute, acute, right, obtuse, very-obtuse} TC: {very-acute, acute, semicircular, plane, very-plane} L: {much-shorter (msh), half-lenght (hl), quite-shorter (qsh), similar-lenght (sl), quite-longer (ql), double-lenght (dl), much-longer (ml)} C: {convex, concave} Fixed Orientation Qualitative ColourTags: {black, dark-grey, grey, light-grey, white, red, yellow, green, turquoise, blue, violet} Relative Orientation Topology Model: • Disjoint (x,y): • Touching (x, y): • Completedly_inside (x, y): • Container (x, y): • Neighbours: Objects with the same container Spatial and Temporal Reasoning for Ambient Intelligence Systems at COSIT 2009

7. 1. Qualitative Description of Images: 1.3. Structure of the Description Qualitative Image Description Spatial Description (1 .. nRegions) Visual Description (1 .. nRegions) Shape (Region) Colour (Region) Topology (Region) Fixed Orientation (Region) Relative Orientation (Region) Containers Reference Systems Neighbours Spatial and Temporal Reasoning for Ambient Intelligence Systems at COSIT 2009

8. 1. Qualitative Description of Images: 1.4. A Case of Study Spatial and Temporal Reasoning for Ambient Intelligence Systems at COSIT 2009

9. Index 1. Qualitative Description of Images 1.1. Approach 1.2. Models of Shape, Colour, Topology and Orientation 1.3. Structure of the Description 1.4. A Case of Study 2. Ontology 2.1. Terminological Knowlege Box (T-Box) 2.2. Assertional Knowledge Box (A-Box) 3. Results 3.1. Approach 3.2. New Knowledge Inferred from the Case of Study 4. Conclusion and Future Work Spatial and Temporal Reasoning for Ambient Intelligence Systems at COSIT 2009

10. 2. Ontology • Provides our qualitative description with: • A formal and explicit meaning to the qualitative labels. • A standard language to share information between agents. • New information inferred by the reasoners • Tools: • Ontology language: OWL3 • Editor: Protégé 4 • Reasoners: FacT++ and Pellet • Knowledge layers: • Reference Conceptualization • Contextualized Descriptions • Ontology Facts  Assertional Knowledge Box (A-Box) Terminological Knowlege Box (T-Box) Spatial and Temporal Reasoning for Ambient Intelligence Systems at COSIT 2009

11. 2. Ontology: 2.1. Terminological Knowlege Box (T-Box) Reference Conceptualization represents knowledge which is supposed to be valid for any application. Spatial and Temporal Reasoning for Ambient Intelligence Systems at COSIT 2009

12. 2. Ontology: 2.1. Terminological Knowlege Box (T-Box) Contextualized Knowledge represents a concrete domain which is application oriented. Spatial and Temporal Reasoning for Ambient Intelligence Systems at COSIT 2009

13. 2. Ontology: 2.2. Assertional Knowledge Box (A-Box) Ontology facts represent the individuals extracted from the description of the image. Spatial and Temporal Reasoning for Ambient Intelligence Systems at COSIT 2009

14. Index 1. Qualitative Description of Images 1.1. Approach 1.2. Models of Shape, Colour, Topology and Orientation 1.3. Structure of the Description 1.4. A Case of Study 2. Ontology 2.1. Terminological Knowlege Box (T-Box) 2.2. Assertional Knowledge Box (A-Box) 3. Results 3.1. Approach 3.2. New Knowledge Inferred from the Case of Study 4. Conclusion and Future Work Spatial and Temporal Reasoning for Ambient Intelligence Systems at COSIT 2009

15. 3. Results 3.1. Approach Spatial and Temporal Reasoning for Ambient Intelligence Systems at COSIT 2009

16. 3. Results 3.2. New Knowledge Inferred • Inferences: • Object 0  UJI_Lab_Wall • Objects 4, 6  UJI_Lab_Door Spatial and Temporal Reasoning for Ambient Intelligence Systems at COSIT 2009

17. Index 1. Qualitative Description of Images 1.1. Approach 1.2. Models of Shape, Colour, Topology and Orientation 1.3. Structure of the Description 1.4. A Case of Study 2. Ontology 2.1. Terminological Knowlege Box (T-Box) 2.2. Assertional Knowledge Box (A-Box) 3. Results 3.1. Approach 3.2. New Knowledge Inferred from the Case of Study 4. Conclusion and Future Work Spatial and Temporal Reasoning for Ambient Intelligence Systems at COSIT 2009

18. 4. Conclusions and Future Work • Our approach describes qualitatively any image using qualitative models of shape, colour, topology and orientation. • The qualitative description obtained is represented by an ontology, which provides our system with: • A formal representation of the knowledge inside the robot • A standard language to exchange information between agents • New knowledge inferred by the reasoners. • As future work, we intend to: • Extend our approach to integrate the reasoner inside the robot system. • Extend our ontology to characterize and classify more landmarks of the robot environment. Spatial and Temporal Reasoning for Ambient Intelligence Systems at COSIT 2009

19. 1 October 2009 is the first birthday of… A spin-off business of The mission of C-Robots is to provide a 100% fully autonomous solution to existing machinery automation. Our product is an intelligent and artificial brain for service robotics. It is composed of specific hardware and an intelligent software which combines traditional robotic solutions with the more advanced cognitive solutions to address specific requirements. We are looking for partners to apply to European projects

20. Thank you for your attention Suggestions to improve our work?