Outline

1. Fast Image Retrieval Using Low Frequency DCT CoefficientsDept. of Computer EngineeringTatung University Presenter: Yo-Ping Huang (黃有評)

2. Outline • Introduction • Problem Description • System Architecture • Feature Extraction Module • Color Space Transformation • Discrete Cosine Transform (DCT) • Similarity Measurement Module • Experimental Results • Conclusions

3. Introduction (1/4) • The content-based image retrieval (CBIR) has received a great attention in the last decades. • CBIR systems use features that can be extracted from the image files themselves in searching a collection of images. • Fast content-based multimedia retrieval is becoming more important as the number of compressed multimedia images and video data increases.

4. Introduction (2/4) • CBIR comprises both indexing and retrieval. • Many techniques have been proposed to index the images or find the feature vectors based on their contents, such as color, texture and shape attributes. • The techniques in transform domain are also been used in CBIR, such as • Fourier Transform • Discrete Cosine Transform (DCT) • Wavelet Transform • Gabor Transform

5. Introduction (3/4) • A retrieval task is performed by matching the feature vector of the query image with those of the database images. • Fewer items in a feature vector will lead to a faster matching at the expense of accuracy. • The feature vector that we propose is extracted from the low frequency DCT coefficients which represent the texture features of an image.

6. Introduction (4/4) • We have implemented an example system employing the proposed feature extraction technology. • The experimental results showed that the proposed feature is sufficient for performing a high efficiency of retrieval with acceptable accuracy. • The retrieval quality can be further improved by introducing users’ opinions on the query images. • Multiple passes of retrieval can be of great help for novel or inexperienced users.

7. Problem Description (1/4) • CBIR often comprises both indexing and retrieval. • Indexing is a process that performs data reduction of images into mathematical features. • Indexing may be subdivided into the following steps: • Feature extraction • Feature vector organization • Classification

8. Problem Description (2/4) • Feature extraction finds out the suitable properties of interest and converting them into mathematical “feature vectors”. • Feature vector organization intends to organize the feature vectors in the database into a structure optimized for searching efficiency. • Classification labels the images into categories of interest to narrow down the search space.

9. Problem Description (3/4) • Retrieval is a process that supports the user interaction to retrieve desired images from the database. • Retrieval comprises the following issues: • User query formulation • User query feature extraction • Query search space strategy • Similarity matching method

10. Problem Description (4/4) • User query formulation: • QBT (Query by text) • QBE (Query by example) • QBS (Query by sketch) • A search strategy is often defined by the choice of organization of the feature vectors. • Similarity matching is typically defined by a distance function, e.g. Euclidean distance. • Fewer items in a feature vector will lead to a faster matching, which is the very problem we try to tackle in this paper.

11. System Architecture • Two major modules in the system • Feature extraction module • Similarity measurement module • Feature extraction module • Compute image features automatically from the given images to archive and/or retrieve images. • Similarity measurement module • Compute the distance between the query image and those images in the database.

12. Figure 1. The architecture of the proposed system.

13. Feature Extraction Module (1/4) • Feature extraction is the most important step in image retrieval. • DCT is one of the best filters for feature extraction in the spatial frequency domain. • The steps of the proposed feature extraction • Transform a RGB image into the YUV color space. • Calculate the low frequency DCT coefficients for Y, U, and components. • The feature vectors are stored into the database along with original images.

14. Feature Extraction Module (2/4) • YUV color space • Y: luminance (or brightness) Y(x, y) = 0.299 R(x, y)+0.587 G(x, y)+0.114 B(x, y) • U: blue chrominance U(x, y) = 0.492(B(x, y)-Y(x, y)) • V: red chrominance V(x, y) = 0.877(R(x, y)-Y(x, y)) • Psycho-perceptual studies have shown that the human brain perceives images largely based on their luminance value, and only secondarily based on their color information

15. Feature Extraction Module (3/4) • Discrete Cosine Transform • DCT coefficients are generated for a K×K image on a pixel by pixel basis. The K×K DCT coefficients thus give the nature of textual energy for each pixel. where f(x, y) is the pixel value at the (x, y) coordinate position in the image, F(u, v) is DCT domain representation of f(x, y), and

16. Feature Extraction Module (4/4) • The coefficients with small u and v correspond to low frequency components. • For most images, much of the signal energy lies at low frequencies. • We use the size of 5×5 low frequency DCT coefficients. The experimental results show that it is sufficient for performing a high efficiency of retrieval. • Users could try to conduct the retrieval based on the DCT coefficients of Y, U or V component.

17. Similarity Measurement Module • For moderate computational complexity, the low frequency block size was chosen to be 5×5. • Distance function where and represent the DCT coefficients of Y-, U- or V-component of the query image q and a database image x, respectively.

18. Experimental Results (1/2) • We have tested an image database of 1000 color images downloaded from the WBIIS database. • It mainly consists of scenes of natural, animals, insects, building, people, and so on. • No pre-processing was done on the images. • We present two different query experiments, each of which uses one of the YUV components as the feature vector.

19. Figure 2. The main screen of the proposed system

20. The first query Figure 3. Retrieved results using Y-components. Figure 4. Retrieved results using U-components.

21. The second query Figure 5. Retrieved results using Y-components. Figure 6. Retrieved results using V-components.

22. Experimental Results (2/2) • For the first query • Five right answers (related to outer space, spaceman, or spaceflight) in the top 10 using Y-component, as shown in Fig. 3. • Eight correct answers if we use U-component, as shown in Fig. 4. • For the second query • Seven correct answers (related to flowers) in the top 10 using Y-component, as shown in Fig. 5. • Ten correct answers if we use V-component as the feature vector instead, as shown in Fig. 6.

23. Conclusions (1/2) • We mainly focus on the texture features of images in YUV color space, which are formed by low frequency DCT coefficients. • The second query has a better result than the first one because the second query example has common textures. • Since only a total of 25 DCT coefficients are calculated as the complete set of features, it leads to fast matching with an acceptable retrieval quality.

24. Conclusions (2/2) • Users can choose any one of the components as its main feature for comparison. • They can choose another component instead if they are not satisfied with the current retrieval results. • Such multiple passes of retrieval can be of great help for novel or inexperienced users. • In the future, we will integrate multiple feature attributes to improve the retrieval results.