1 / 17

Image Search

Image Search. Using Color and Texture. Project: NETRA. http://www-iplab.ece.ucsb.edu/netra/Netra.html Examples: edge flow: http://www-iplab.ece.ucsb.edu/netra/examples/Netra.html. Efficient Color Representation.

jodie
Download Presentation

Image Search

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Image Search Using Color and Texture

  2. Project: NETRA • http://www-iplab.ece.ucsb.edu/netra/Netra.html • Examples: edge flow: http://www-iplab.ece.ucsb.edu/netra/examples/Netra.html

  3. Efficient Color Representation • For efficiency, Yining Deng,et.al.1 proposed using a dominant color descriptor, which is the peak histogram value for each of three colors RGB. • Previous implementations used colors from 10s to 100s of bins, time and storage were excessive • Singular Value Decomposition used to reduce computational complexity, however the reduction is not related to features and will produce errors • Hilbert curve fitting- features close together in the original may be far apart on the curve. • Color Moment Descriptors (mean, variance, and third order moment) perform slightly worse than a high level histogram. The mean colors may be quite different from any of the original colors • Since they perform satisfactorily even with this disadvantage, I decided to implement this • The Dominant color descriptor takes the dominant colors from a histogram and searches for colors within a distance of these colors (each of which have a percentage of each of RGB)

  4. Efficient Color Representation • Image is first segmented using the edgeflow algorithm • Color clustering is performed on each segment to obtain representative colors • The dominant color descriptors is then: • F = {{ci, pi}, i = 1, …, N} • N is the number of color clusters • ci is the color vector • pi is its percentage where pi = 1 • Three to four colors are sufficient to describe an image region • Complexity of a search is O(m n) where n is the number of colors in a query, m is the number of nodes accessed per query color

  5. Texture Features • B.S Manjunath, W.Y. Ma2,3 suggest using a Gabor function and its frequency components, the Fourier transform • Higher frequency components may be sufficient to retrieve images • Calculating the entire Gabor function for a 512 x 512 image takes 9.3 seconds on a SUN Spark 20, search and retrieval takes 1.02 seconds. • Using the adaptive filter that they suggest using four feature components takes 2.3 seconds for feature extraction and 0.7 seconds for adaptive filter selection and 0.1 seconds for retrieval • This was too long • Used edge flow to segment images\ • Mean and variance, presentation by Dr Lu • Using the Mean, Standard Deviation, Third and Forth Moment • Compatible with the color calculations, efficient, I have adopted this method

  6. Implementation Parameters • Web based so that anyone can upload pictures and try it out • Use of a robust language • Java: • Easy interface with web pages • Capable of performing appropriate calcs on the server • Used several Java Servlets • ImageInfo http://www.geocities.com/SiliconValley/Lakes/6686/, gets width, height, etc of an image • MultipartRequest http://www.geocities.com/jasonpell/programs.html, parses a multipart form and writes out any files found • ServletMultipartRequest http://www.millstone.org/documentation/apidocs/org/millstone/webadapter/ServletMultipartRequest.html, wrapper for MultipartRequest for use in Servlets

  7. Implementation Parameters • JSP • Java compiled on the fly • Java, SQL, and HTML mixed on the same page • SQL • Used Hypersonic SQL, an all Java implementation • Find a server that has all of these • Although I already have a web server (and FTP) on 2 home machines, determining how to set up a JSP and HSQL server would be too time consuming (plus Oracle SQL uses busy-waiting and takes up all your clock cycles) • http://www.mycgiserver.com/ • Free, although they email requests for donations for an upgrade • Sometimes slow, they are planning an upgrade to speed things up • http://www.webappcabaret.com/ only 1 month free • http://www.brinkster.com/ • Moved the start page here because it is faster (and also free)

  8. Implementation • I didn’t get as far as I had expected, but have images being uploaded and added to a DB • http://www25.brinkster.com/kenwschmidt/search/index.html

  9. Future Implementation • Texture • By Mean and variance • Mean: x = 1/n  xi • Standard Deviation (second moment) • = 1/n  (xi – x )2 • Third Moment = 1/n  (xi – x )3 • Fourth Moment = 1/n  (xi – x )4 • Search the DB for images within Y% of these four texture parameters • Color • Record the mean of each of the three colors • Search for images within Y% of these

  10. How to Implement? • Found some implementations which would be easy but they were not standard SUN Java, therefore the server that I had chosen probably doesn’t have them available • Searching SUN’s documentation, I found several ways to access pixels: • DataBuffer • SampleModel • BandedSampleModel • ColorModel

  11. How to Implementfootnote Java has some built in functions that are interesting float[] elements = { 0.0f, -1.0f, 0.0f,-1.0f, 4.f, -1.0f, 0.0f, -1.0f, 0.0f}; BufferedImage bimg = new BufferedImage(bw,bh,BufferedImage.TYPE_INT_RGB); Kernel kernel = new Kernel(3, 3, elements); ConvolveOp cop = new ConvolveOp(kernel, ConvolveOp.EDGE_NO_OP,null); cop.filter(bi,bimg);

  12. How to Implementfootnote byte reverse[] = new byte[256]; for (int j=0; j<200; j++){ reverse[j]=(byte)(256-j); } ByteLookupTable blut=new ByteLookupTable(0, reverse); LookupOp lop = new LookupOp(blut, null); lop.filter(bi,bimg);

  13. How to Implementfootnote RescaleOp rop = new RescaleOp(1.5f, 1.0f, null); rop.filter(bi,bimg);

  14. How to Implementfootnote Sharpening with Convolution: float[] elements = { 0.0f, -1.0f, 0.0f, -1.0f, 5.f, -1.0f, 0.0f, -1.0f, 0.0f};   Kernel kernel = new Kernel(3,3,elements); ConvolveOp cop = new ConvolveOp(kernel, ConvolveOp.EDGE_NO_OP, null); cop.filter(bi,bimg);

  15. How to Implement • Each of these did too much, I just wanted access to the raw pixels • java.awt.image.PixelGrabber

  16. for (int j = 0; j < h; j++) { for (int i = 0; i < w; i++) { pixel = pixels [ j * w + i ]; red = (pixel >> 16) & 0xff; green = (pixel >> 8) & 0xff; blue = (pixel ) & 0xff; redSum2 += pow ( ( red - redMean ), 2 ); greenSum2 += pow ( ( green- greenMean ), 2 ); blueSum2 += pow ( ( blue- blueMean ), 2 ); redSum3 += pow ( ( red - redMean ), 3 ); greenSum3 += pow ( ( green- greenMean ), 3 ); blueSum3 += pow ( ( blue- blueMean ), 3 ); redSum4 += pow ( ( red - redMean ), 4 ); greenSum4 += pow ( ( green- greenMean ), 4 ); blueSum4 += pow ( ( blue- blueMean ), 4 ); } } redMoment2 = redSum2 / noOfPixels; greenMoment2 = greenSum2 / noOfPixels; blueMoment2 = blueSum2 / noOfPixels; redMoment3 = redSum3 / noOfPixels; greenMoment3 = greenSum3 / noOfPixels; blueMoment3 = blueSum3 / noOfPixels; redMoment4 = redSum4 / noOfPixels; greenMoment4 = greenSum4 / noOfPixels; blueMoment4 = blueSum4 / noOfPixels; overallMoment2 = ( redMoment2 + greenMoment2 + blueMoment2 ) / 3; overallMoment3 = ( redMoment3 + greenMoment3 + blueMoment3 ) / 3; overallMoment4 = ( redMoment4 + greenMoment4 + blueMoment4 ) / 3; Pseudo Code for (int j = 0; j < h; j++) { for (int i = 0; i < w; i++) { pixel = pixels [ j * w + i ]; red = (pixel >> 16) & 0xff; green = (pixel >> 8) & 0xff; blue = (pixel ) & 0xff; redSum += red; greenSum += green; blueSum += blue; } } redMean = redSum / noOfPixels; greenMean = greenSum / noOfPixels; blueMean = blueSum / noOfPixels; overallSum = redSum + greenSum + blueSum; overallMean = overallSum / noOfPixels;

  17. References • An Efficient Color Representation for Image Retrieval, Yining Deng, B. S. Manjunath, Charles Kenney, Michael S. Moore, Hyundoo Shin, IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 10, NO. 1, JANUARY 2001 • Texture Features for Browsing and Retrieval of Image Data, B.S Manjunath, W.Y. Ma, IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 18, NO. 8, JANUARY 1996 • A Texture Thesaurus for Browsing Large Aerial Photographs, Wei-Ying Ma and B. S. Manjunath, JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE. 49(7):633–648, 1998 • Texture Features and Learning Similarity, Wei-Ying Ma and B. S. Manjunath, 1063-6919/96IEEE

More Related