1 / 25

Barcelona Forum on Ph.D. Research in Communications, Electronics and Signal Processing

Barcelona Forum on Ph.D. Research in Communications, Electronics and Signal Processing 21 st October 2010. OP2: Information Theoretical Region Merging Approaches and Fusion of Hierarchical Image Segmentation Results Felipe Calderero, Thesis Advisor: Ferran Marqués. Outline. Introduction

prue
Download Presentation

Barcelona Forum on Ph.D. Research in Communications, Electronics and Signal Processing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Barcelona Forum on Ph.D. Research in Communications, Electronics and Signal Processing 21st October 2010 OP2: Information Theoretical Region Merging Approaches and Fusion of Hierarchical Image Segmentation ResultsFelipe Calderero, Thesis Advisor: Ferran Marqués

  2. Outline • Introduction • Information Theoretical Region Merging • Cooperative Region Merging • Conclusions

  3. 1. What is image segmentation? • Image Segmentation • Partition of the image into regions (disjoint sets of spatially contiguous pixels) • Key step in image analysis • Semantically, first level of abstraction • Practically, reduction of primitives • But image segmentation is a difficult task…

  4. 1. An ill-posed problem… • Image Segmentation is an ill-posed problem • A unique solution may not exist • Different levels of detail • Same level of detail

  5. 1. An ill-posed problem… • Image Segmentation is an ill-posed problem • A unique solution may not exist • Different levels of detail • Same level of detail Hierarchical Segmentation Approaches Fusion of (Hierarchical) Segmentation Results

  6. 1. PhD Thesis Objectives • Objective 1: Hierarchical Segmentation Approaches • Objective 2: Fusion of (Hierarchical) Segmentation Results Provide an unsupervised hierarchical solution to the segmentation of generic images Design a generic and scalable segmentation scheme to fuse in an unsupervised manner hierarchical segmentation results

  7. 1. PhD Thesis Approach • Solutions to Objective 1 and Objective 2 • Generic • No a priori information • Hierarchical solution • Unsupervised Bottom-up hierarchy [Marr82] Region Merging Techniques Hierarchy of most representative partitions at different levels of detail

  8. 1. PhD Thesis Contributions • Contribution 1: Hierarchical Segmentation Approaches • Contribution 2: Fusion of (Hierarchical) Segmentation Results • Information Theoretical Region Merging Techniques (IT-RM) • Cooperative Region Merging Scheme (CRM)

  9. 2. Region Merging Techniques Binary Partition Tree (BPT) [Garrido99] Efficiency of computation and representation • Region Merging • Hierarchical bottom-up segmentation approaches • Specified by • Region Model • Merging Criterion • Merging Order • Partition Selection Criterion G G Selection Criterion F F E Hierarchy creation A E E B Unsupervised mode A Relevant partition extraction A B C D D C B

  10. 2. Information Theory Region Merging • Information Theoretical Region Merging (IT-RM) • Statistical and information theoretical framework • Region Model • i.i.d / Markov region model • Merging Criteria • Kullback-Leibler / Bhattacharyya Criteria • Merging Order • Classical / Scale-based • Unsupervised mode: • Multiple partition selection criterion (statistically relevant)

  11. 2. IT-RM Applications • Semantic image analysis 1st Significant Partition 2nd Significant Partition Original

  12. 2. IT-RM Applications • Semantic image analysis (textures) 1st Significant Partition 2nd Significant Partition Original

  13. 2. IT-RM Applications • Object-based representation and analysis

  14. 3. Cooperative Region Merging • Motivation: Objective 2 • Most IT-RM techniques have similar and accurate performance… • Instead of selecting, why not combining the set of techniques? • Cooperative Region Merging (CRM) • Similar to a negotiation process in decision making

  15. 3. Cooperative Region Merging • Cooperative Region Merging • Segmentation results are computed independently by each technique (RM step) • A basic consensus or agreement is established between the set of techniques (FUSION step) • Steps 1 and 2 are repeated while further consensus is possible • Characteristics: parallel, scalable, hierarchical, unsupervised, flexible…

  16. 3. CRM Applications • Accuracy and robustness improvement • Combining different segmentation techniques Human Partition 3rd Median Partition 2nd Median Partition 1st Median Partition Original

  17. 3. CRM Applications • Fusion of heterogeneous information channels • Combining color and depth for object-based segmentation Disparity map 3rd Sign. Partition 2nd Sign. Partition 1st Sign. Partition Color image

  18. 3. CRM Applications • Scalability and flexibility of the fusion scheme • Fusion of multispectral band and vegetation classification RGB composition NDVI Vegetation extraction using bands: B, G, R, IR Vegetation extraction using bands: B, G, R, IR + PAN

  19. 4. Conclusions • Unsupervised segmentation of generic images: a challenge • Hierarchical Region Merging approach as a possible solution • Info. Theoretical Region Merging Segmentation • State-of-the-art results in unsupervised manner • Relevant image explanations at different levels of detail • Application independent object-based semantic tool • Cooperative Region Merging Information Fusion • Accuracy and robustness improvement in unsupervised manner • Scalable, flexible and generic scheme • Fusion of homogeneous/heterogeneous information channels

  20. Information Theoretical Region Merging Approaches and Fusion of Hierarchical Image Segmentation ResultsFelipe Caldererofelipe.calderero@upf.eduImage Processing Group Pompeu Fabra University (UPF) Barcelona, SpainFerran Marquésferran.marques@upc.eduImage Processing Group Universitat Politècnica de Catalunya Barcelona, Spain

  21. 4. Applications • CRM: Fusion of color and depth information

  22. 4. Applications • IT-RM: Semantic image analysis 1st Significant Partition 2nd Significant Partition Original

  23. 5. Conclusions • Information Theoretical Region Merging • State-of-the-art segmentation results without any assumption about the nature of the region • Unsupervised extraction of most relevant image explanations at different levels of detail • Application independent accurate tool for object-based representation and semantic analysis of generic images

  24. 5. Conclusions • Cooperative region merging • Global improvement of the accuracy and the stability of the segmentation results by combining different segmentation approaches • Parameter removal solution • Fusion of heterogeneous information channels • Flexibility to incorporate specificities of the fusion problem • A priori information about the fused sources (e.g. channel priority) • Joint segmentation and classification stages

More Related