以空間關係相鄰圖為基礎之空間關係相似性量測方法 Retrieval by spatial similarity based on interval neighbor group

1. 以空間關係相鄰圖為基礎之空間關係相似性量測方法Retrieval by spatial similarity based on interval neighbor group 研究生：黃彥人 指導教授：蔣依吾博士 中山大學資訊工程學系

2. Outline • Introduction • Retrieval by Spatial Similarity (RSS) • Single-Instance • Between two pairwise spatial relations • Between a set of pairwise spatial relations and a pairwise spatial relation • Between two sets of pairwise spatial relations • Multiple-Instance • Experimental study • Conclusions and future work

3. Image retrieval • Symbolic manipulation • Google image search • Drawback: ambiguity • Content-Based Image Retrieval (CBIR) • Visual features • Color [S. Berretti ,2002] • Shape[W. C. Lin, 2007] • Texture[P. W. Huang, 2003] • Relationships features

4. Motive • Retrieval by Spatial Similarity (RSS) • The fine granularity of spatial similarity measure • Similarity ranking Query image

5. Flowchart Image Spatial relation feature retrieval Spatial relation feature retrieval Spatial relation feature Image Database Image Database Image Indexing SRS Image Retrieval Query image Spatial relation feature retrieval Spatial relation feature retrieval Spatial relation feature Similarity measure Similarity measure Similarity ranking • One v.s One • Many v.s One • Many v.s Many Results

6. Image Indexing True image picture Symbolic picture[1991] MBR (Minimum Bounding Rectangle ) Spatial Reasoning and Indexing Image Database

7. Spatial Reasoning and Indexing SMR SRC [2001] [97] 9DLT Matrix SRR [2001] [91] GPN Matrix PN Matrix [97] [2001] 2d G-string 2d C-string 2d C+ string [94] 2d string 2d string [88] [92] 2d B-string 2d Bε-string SRR MSR [87] [92] [92] [2005] [2004] WeiRe’s method 2d H-string [88] [2001] 2d+ string 2d PIR-string [99] SBA [95] Pointset [91] [2005]

8. RSS (Retrieval by Spatial Similarity) RSS (Retrieval by Spatial Similarity) Ranking Yes No Image Database Query image

9. 13 1D spatial relations • Interval Neighbor Group • Pairwise Node • Neighbor • Path • The shortest path [Lee and Hsu, 1992] • Distance • The shortest distance • Similarity 1 [C. Freska, 1994]

10. One v.s One , 1 1: exact match , 0: total irrelevance

11. Query image Q Database image D 4+1=5

12. Many v.s One spatial relations • n objects : Query image Q Database image D

14. Many v.s Many • Frequently encountered in the database retrieval applications Query image Q Database image D

16. Image Spatial relation feature retrieval Spatial relation feature Image Database Image Indexing SRS Image Retrieval Query image Spatial relation feature retrieval Spatial relation feature Similarity measure Similarity ranking Results Spatial relation bit sequence coding

17. SuperNode v.s SuperNode SimpleNode v.s. SimpleNode 1101 OR 0011 1111 1101 Exclusive-OR 0001 1100 SuperNode v.s SimpleNode SuperNode SimpleNode SimpleNode

18. Image transformation • Rotated by [2007] A before B A ~overlap B A ~overlap B A after B

20. Database simulation • 1000 fundamental pictures • MBR-covered object random dimension and positioning • 2 to 5 MBR-covered objects • 7000 pictures in a database • 2 to 7 MBR-covered objects

21. Experimental study • Comparison:RSS-ING v.s 2D Be-string[2003] • 13 1D spatial relations longest common subsequence (LCS) Query image Q Database image D 12 12

22. Single-Instance retrieval results based on similarity measure Query image 2D Be-string RSS-ING

23. RSS-ING v.s 2D Be-string Single-Instance retrieval results based on similarity measure RSS-ING Single-Instance Image transformation (Rotated by ) Query image 2D Be-string Single-Instance

24. Flowchart Image Spatial relation feature retrieval Spatial relation feature Image Database Image Indexing SRS Image Retrieval Query image Spatial relation feature retrieval Spatial relation feature Similarity measure Spatial relation feature retrieval Multiple query images MIL Similarity ranking Results

25. MIL (Multiple-Instance Learning) • Using a single image to query a database might employ spatial relations that do not match user’s expectation ∩ ━ ∩ ∩ ∩ ━ ： Concept = {Stone, Waterfall}– {Sky, Cloud, Grass, Stone} ={Waterfall} = ??

26. Two Positive images Common spatial feature Diverse Density [Maron, 1998] Ideal spatial feature Retrieval by Spatial Similarity MIL

27. Single-Instance v.s. Multiple-Instance retrieval results based on similarity measure by two Positive images Single query image RSS-ING Single-Instance Ideal spatial feature RSS-ING Multiple-Instance

28. Multiple-Instance v.s. Multiple-Instance retrieval results based on similarity measure by two Positive images and one negative image RSS-ING Multiple-Instance 2 example images one negative image Ideal spatial feature RSS-ING Multiple-Instance 3 example images

29. RSS-ING v.s 2D Be-string Multiple-Instance retrieval results based on similarity measure RSS-ING Multiple-Instance 3 example images Ideal spatial feature 2D Be-string Multiple-Instance 3 example images

30. Execution time and memory space

31. Conclusions and future work • The degree of similarity between two spatial relations is linked to the distance between the associated nodes in an ING • Quantitatively ranked according to the degree of similarity with the query • Fine granularity • MIL procedure identifies common positive features and excludes negative ones to further clarify the user’s searching criteria • Future work • Video employment • Other visual features