1 / 20

SWE 423: Multimedia Systems

SWE 423: Multimedia Systems. Chapter 4: Graphics and Images (3). Outline. Image Processing Basics Image Features Image Segmentation Textbook Section 4.3

unity
Download Presentation

SWE 423: Multimedia Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SWE 423: Multimedia Systems Chapter 4: Graphics and Images (3)

  2. Outline • Image Processing Basics • Image Features • Image Segmentation • Textbook Section 4.3 • Additional Reference: Wasfi Al-Khatib, Y. Francis Day, Arif Ghafoor, and P. Bruce Berra. Semantic modeling and knowledge representation in multimedia databases. IEEE Transactions on Knowledge and Data Engineering, 11(1):64-80, 1999.

  3. Image Processing • Image processing involves the analysis of scenes or the reconstruction of models from images representing 2D or 3D objects. • Image Analysis • Identifying Image Properties (Image Features) • Image Segmentation • Image Recognition • We will look at image processing from a database perspective. • Objective: Design of robust image processing and recognition techniques to support semantic modeling, knowledge representation, and querying of images.

  4. Semantic Modeling and Knowledge Representation in Image Databases • Feature Extraction. • Salient Object Identification. • Content-Based Indexing and Retrieval. • Query Formulation and Processing.

  5. Semantic Specification Knowledge Base Semantic Identification Process Object Recognition Process Object Models Feature Extraction Process Feature Specification Image Data Still Video Frames Multi-Level Abstraction Semantic Modeling And Knowledge Representation Layer Object Recognition Layer Feature Extraction Layer Multimedia Data

  6. Feature Extraction Layer • Image features: Colors, Textures, Shapes, Edges, ...etc. • Features are mapped into a multi-dimensional feature space allowing similarity-based retrieval. • Features can be classified into two types: Global and Local.

  7. Global Features • Generally emphasize coarse-grained pattern matching techniques. • Transform the whole image into a functional representation. • Finer details within individual parts of the image are ignored. • Examples: Color histograms and coherence vectors, Texture, Fast Fourier Transform, Hough Transform, and Eigenvalues. • What are some of the example queries?

  8. Color Histogram • How many pixels of the image take a specific color • In order to control the number of colors, the domain is discretized • E.g. consider the value of the two leftmost bits in each color channel (RGB). • In this case , the number of different colors is equal to __________ • How can we determine whether two images are similar using the color histogram?

  9. Color Coherence Vector • Based on the color histogram • Each pixel is checked as to whether it is within a sufficiently large one-color environment or not. • i.e. in a region related by a path of pixels of the same color • If so, the pixel is called coherent, otherwise incoherent • For each color j, compute the number of coherent and incoherent pixels (j , j), j = 1, ..., J • When comparing two images with color coherence vectors (j , j) and (j , j), j = 1, ..., J, we may use the expression

  10. Texture • Texture is a small surface structure • Natural or artificial • Regular or irregular • Examples include • Wood barks • Knitting patterns • The surface of a sponge

  11. Texture Examples • Artificial/periodic • Artificial/non-periodic • Photographic/pseudo-periodic • Photographic/random • Photographic/structured • Inhomogeneous (non-texture)

  12. Texture • Two basic approaches to study texture • Structural analysis searches for small basic components and an arrangement rule • Statistical analysis describes the texture as a whole based on specific attributes (local gray-level variance, regularity, coarseness, orientation, and contrast. • Either done in the spatial domain or the spatial frequency domain

  13. Global Features • Advantages: • Simple. • Low computational complexity. • Disadvantages: • Low accuracy

  14. Local Features • Images are segmented into a collection of smaller regions, with each region representing a potential object of interest (fine-grained). • An object of interest may represent a simple semantic object (e.g. a round object). • Choice of features is domain specific: • X-ray imaging, GIS, ...etc require spatial features (e.g. shapes [may be calculated through edges] and dimensions.) • Paintings, MMR imaging, ...etc may use color features in specific regions of the image.

  15. Edge Detection • A given input image E is used to gradually compute a (zero-initialized) output image A. • A convolution mask runs across E pixel by pixel and links the entries in the mask at each position that M occupies in E with the gray value of the underlying image dots. • The result of the linkage (and the subsequent sum across all products from the mask entry and the gray value of the underlying image pixel) is written to the output image A.

  16. Convolution • Convolution is a simple mathematical operation which is fundamental to many common image processing operators. • Convolution provides a way of `multiplying together' two arrays of numbers, generally of different sizes, but of the same dimensionality, to produce a third array of numbers of the same dimensionality. • This can be used in image processing to implement operators whose output pixel values are simple linear combinations of certain input pixel values. • The convolution is performed by sliding the kernel over the image, generally starting at the top left corner, so as to move the kernel through all the positions where the kernel fits entirely within the boundaries of the image.

  17. Convolution Computation • If the image E has M rows and N columns, and the kernel K has m rows and n columns, then the size of the output image A will have M - m + 1 rows, and N - n + 1 columns and is given by: • Example page 60. • http://homepages.inf.ed.ac.uk/rbf/HIPR2/sobel.htm

  18. Similarity Metrics • Minkowski Distance • Weighted Distance • Average Distance • Color Histogram Intersection

  19. Prototype Systems • QBIC (http://www.hermitagemuseum.org) • Uses color, shape, and texture features • Allows queries by sketching features and providing color information • Chabot (Cypress) • Uses color and textual annotation. • Improved performance due to textual annotation (Concept Query) • KMeD • Uses shapes and contours as features. • Features are extracted automatically in some cases and manually in other cases.

  20. Demo (Andrew Berman & Linda G. Shapiro ) • http://www.cs.washington.edu/research/imagedatabase/demo/seg/ • http://www.cs.washington.edu/research/imagedatabase/demo/edge/ • http://www.cs.washington.edu/research/imagedatabase/demo/fids/

More Related