1 / 47

ISTORAMA: A Content-Based Image Search Engine and Hierarchical Triangulation of 3D Surfaces.

ISTORAMA: A Content-Based Image Search Engine and Hierarchical Triangulation of 3D Surfaces. Dr. Ioannis Kompatsiaris Centre for Research and Technology Hellas Informatics and Telematics Institute Thermi-Thessaloniki, Greece ikom@iti.gr. Outline. Introduction Istorama architecture

vea
Download Presentation

ISTORAMA: A Content-Based Image Search Engine and Hierarchical Triangulation of 3D Surfaces.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ISTORAMA: A Content-Based Image Search Engine andHierarchical Triangulation of 3D Surfaces. Dr. Ioannis Kompatsiaris Centre for Research and Technology Hellas Informatics and Telematics Institute Thermi-Thessaloniki, Greece ikom@iti.gr

  2. Outline • Introduction • Istorama architecture • K-Means with Connectivity Constraint Algorithm (KMCC) • Demo • Object/model based coding • Adaptive Triangulation and Progressive transmission • Reduced pyramid - quincunx sampling • Experimental results • Conclusions

  3. Need for efficient image search • Huge number of images or databases of images • Highly visual and graphical nature of the Web • Text descriptors are not always efficient • Greater flexibility with “content-based” access • Queries which are more natural to humans

  4. Proposed approach • Usually a description, a “signature” or a set indexes is created for the whole image • Images usually contain different objects • Proposed approach: the image is first separated into objects (segmentation) • Descriptors are created for each object • The user can search for a specific object contained in images

  5. ISTORAMA architecture World Wide Web User PHP Crawler - Spider Indexing - Retrieval Algorithms Data Base JDBC Java Data Base Connection Server

  6. The K-Means with Connectivity Constraint Algorithm (KMCC) I • Based on K-Means algorithm • K-Means does not take into account spatial information • In KMCC, the spatial proximity of each region is also taken into account by defining a new spatial center and by integrating the K-Means with a component labeling procedure • Automatic correction of the number of regions K

  7. The K-Means with Connectivity Constraint Algorithm (KMCC) II • Step1 K-Means is performed  • Step2Spatial centers are calculated • Step3 Generalised distance • Step 4 Component labeling  L connected regions

  8. The K-Means with Connectivity Constraint Algorithm (KMCC) III

  9. Object descriptors • Color, texture and spatial characteristics • Color: histogram, 8 bins • Spatial: (centroid), • Shape: area, eccentricity • where λ1, λ2are the two first eigenvalues

  10. Experimental Results (Synthetic)

  11. Experimental Results (Synthetic)

  12. Experimental Results

  13. Experimental Results (Claire) Original sequence Frames 1-10 Segmentation Moving object Facial region

  14. Experimental results (Claire)

  15. Experimental results (table-tennis) Original sequence Frames 1-10

  16. Experimental results (table-tennis) Moving objects Segmentation

  17. Experimental results (Akiyo+Foreman) Original sequence Frames 1-10 Facial region Original sequence Frames 1-10 Facial region

  18. Conclusions • K-means with spatial proximity algorithm • Multiple features segmentation • Higher order segmentation • Correspondence of objects between consequent frames • Max-min criterion for automatic regularisation parameters

  19. Future work • Use of texture • Indexing of video • Integration with text descriptors

  20. Introduction • Triangular meshes of high quality are used in: • Computer Aided Design • 3D representation of objects (e.g. archaeological artifacts) • Animation and visual simulation • Entertainment (computer games) • Digital Terrain Modelling

  21. Object/model-based coding

  22. Object/model-based coding

  23. Object/model-based coding

  24. Adaptive triangulation Compressionof finely detailed surfaces is necessary for: • computation • storage • transmission • display efficiency

  25. Progressive transmission • Early, coarse approximations are refined though additional bits

  26. Background • Vertices removal and retriangulation [Schroeder] [Cohen] • General mesh optimization process/function [Hoppe] • Multiresolution analysis (MRA) [Lounsbery] • Wavelets [Schroeder] [Gross] • Progressive transmission [Schroeder] [Hoppe] • Generalized triangle mesh representation [Deering]

  27. Properties of the algorithm • Efficient compression of the wireframe information • Simplification of the wireframe by adaptive triangulation • Progressive transmission of the wireframe information • Prioritised transmission of the wireframe • Straightforward correspondence between successive scales

  28. Input surfaces • Surface represented as a parametric function in the parametric space • determined by the position of a set of control points or nodes • It allows for arbitrary, possibly closed wire-frame surfaces to be defined.

  29. Input surfaces • The filters are applied to the 2D parametric representation of the surface as though it were a 2D image with intensity equal to • Such surfaces include also: • depth images estimated from stereo pairs and • every surface that is homomorphic to a plane, cylinder or torus

  30. Block diagram of the proposed procedure

  31. Reduced pyramid with quincunx sampling matrix

  32. Corresponding triangulation

  33. Optimal filters • Optimal filters are determined by their Fourier transform: • where is the power spectral density. • Alternatively may be determined by the equation:

  34. Optimum bit allocation • bits/vertex is assumed to be transmitted • bits/vertex are allocated to each level using • is the sum of error variances

  35. Error prioritization • The prediction errors corresponding to all predicted vertices are calculated and sorted with the vertices corresponding to higher errors being put first on the list Lower Errors Higher Errors

  36. Entropy estimation • Entropy coding is used • The number of bits needed for error transmission is the entropy of the errors • Using the quincunx sampling geometry at the receiver, there is no need to transmit the exact co-ordinates of the position of each transmitted vertex • The final cost of the transmission is the sum of the error entropy and the position entropy

  37. Adaptive Triangulation Procedure • Synthesis stage of the QMVINT pyramid • The vertex along with the vertices used to predict it are added to the mesh • Handling of cracks • Triangulation of the next vertex

  38. Adaptive triangulation procedure

  39. Adaptive triangulation procedure

  40. Experimental results • Original dense depth map and surface of the “Venus” data

  41. Experimental results • 2569 vertices and 4006 triangles at level 2 MSE = 1.30

  42. Experimental results • 7661 vertices and 11135 triangles at level 1 MSE = 1.30

  43. Experimental results • 11416 vertices and 15827 triangles at level 0 MSE = 0.12

  44. Experimental results

  45. Conclusions • Hierarchical representation of 3D surfaces using 3D adaptive triangular wireframes • The variance of the error transmitted is minimised and therefore results to optimal compression of the wireframe information • It produces a hierarchy where coarse meshes are as similar to their finer versions as is possible

  46. Conclusions • The triangulation algorithm is integrated with a bit allocation procedure • The number of nodes and triangles of the wireframe as well as the information needed for the transmission or storage of the wireframe are reduced simultaneously using a unified approach (QMVINT filtering) • Precise correspondence between triangles at each level is achieved

  47. Future work • Expansion and application directly to 3D surfaces • Estimation of filters

More Related