1 / 36

Using HTML Textual and Structural Data for Web Image Search Cheng Thao, Ethan Munson, Jim Dabrowski, Nikolas D. Bohne Un

Using HTML Textual and Structural Data for Web Image Search Cheng Thao, Ethan Munson, Jim Dabrowski, Nikolas D. Bohne University of Wisconsin-Milwaukee. Which image is George Bush or has George Bush?. Which images are similar to this image?.

michon
Download Presentation

Using HTML Textual and Structural Data for Web Image Search Cheng Thao, Ethan Munson, Jim Dabrowski, Nikolas D. Bohne Un

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Using HTML Textual and Structural Data for Web Image Search Cheng Thao, Ethan Munson, Jim Dabrowski, Nikolas D. Bohne University of Wisconsin-Milwaukee

  2. Which image is George Bush or has George Bush?

  3. Which images are similar to this image?

  4. Does the HTML source tell which images is George Bush? <tr> <td width=400 bgcolor=ffffff><center><FONT FACE"Arial, Helvetica" SIZE=-1> <IMG SRC="http://www.gopbi.com/community/groups/bigband/images/bill%20cosby.jpg" alt="" BORDER=0 VSPACE=3 HSPACE=3> </FONT></center><em><center><FONT FACE="Arial, Helvetica" SIZE=-1>Bill Cosby</center></FONT></em></td> <td width=210 bgcolor=ffffff><center><FONT FACE="Arial, Helvetica" SIZE=-1> <IMG SRC="http://www.gopbi.com/community/groups/bigband/images/betty%20white.jpg" alt="" BORDER=0 VSPACE=3 HSPACE=3> </FONT></center><em><center><FONT FACE="Arial, Helvetica" SIZE=-1>Betty White</center></FONT></em></td> </tr> <tr> <td width=400 bgcolor=ffffff><center><FONT FACE"Arial, Helvetica" SIZE=-1> <IMG SRC="http://www.gopbi.com/community/groups/bigband/images/tom%20brokaw.jpg" alt="" BORDER=0 VSPACE=3 HSPACE=3> </FONT></center><em><center><FONT FACE="Arial, Helvetica" SIZE=-1>Tom Brokaw</center></FONT></em></td> <td width=210 bgcolor=ffffff><center><FONT FACE="Arial, Helvetica" SIZE=-1> <IMG SRC="http://www.gopbi.com/community/groups/bigband/images/george%20bush.jpg" alt="" BORDER=0 VSPACE=3 HSPACE=3> </FONT></center><em><center><FONT FACE="Arial, Helvetica" SIZE=-1>Pres. George Bush</center></FONT></em></td> </tr> <tr> <td width=400 bgcolor=ffffff><center><FONT FACE"Arial, Helvetica" SIZE=-1> <IMG SRC="http://www.gopbi.com/community/groups/bigband/images/ed%20mcmahon.jpg" alt="" BORDER=0 VSPACE=3 HSPACE=3> </FONT></center><em><center><FONT FACE="Arial, Helvetica" SIZE=-1>Ed McMahon</center></FONT></em></td> <td width=210 bgcolor=ffffff><center><FONT FACE="Arial, Helvetica" SIZE=-1> <IMG SRC="http://www.gopbi.com/community/groups/bigband/images/bob%20barker.jpg" alt="" BORDER=0 VSPACE=3 HSPACE=3> </FONT></center><em><center><FONT FACE="Arial, Helvetica" SIZE=-1>Bob Barker</center></FONT></em></td> </tr>

  5. Introduction -image search is difficult - performance is slow - image identification is a complex, inaccurate task -most research on image search has emphasized analysis of image content -few Web image search engines - commercial: Alta Vista, Google - research: WebSeek -little research in textual image search

  6. HTML overview -HTML document composed of: -head -title -meta -body -paragraph, -table, -text, -link, -image, …

  7. <html> <head> <title>Sample HTML</title> <meta keywords="html, html elements"> <meta description="showing a simple html and some html elements"> </head> <body> <h1>HTML overiew</h1> <p> first paragraph</p> <table border="1"> <caption> Simple Table</caption> <tr><td>1<td>2<td>3 <tr><td>4<td>5<td>6 <tr><td>7<td>8<td>9 </table> Here is a photo of George Bush. <br> <img src="g-bush.jpg"> </body> </html>

  8. Previous work - Yelena Tsymbalenko -studied HTML constructs and determine what can be used in image search. -found the following to be effective - title of the page - image filename - image alt attribute

  9. Research Goals • What HTML features make good clues to the content of images? • Structural features (document, table) • File names or URLs • Formatting of material (bold, heading) • - How can clues be combined into a single relevance rating?

  10. Image Search Study Process -Downloading pages with matching text - Use existing search engine to identify matches - These pages provide a corpus of images - We download pages so that our corpus remains static -Download acts as a snap shot -Clue extraction -Analyze each page in corpus for all possible clues to image content -Human relevance ratings -human rates if an image is relevant to the query -Statistical analysis to find clue-based relevance functions

  11. Process: Downloading Web Pages query queries Downloading Software <query> Search Engine URLs Web Pages images Web pages and images are saved to local disk.

  12. Design: Queries in XML Multiple queries are stored in an XML file Engine: 1=Altavista, 2=Excite, 3=Hotbot, 4=Google Method: 1 =or, 2 = and, 3 = expression Search for George Bush using Alta Vista and must have all the words Search for Bill Clinton using Hotbot and search for exact expression <query> <engine> 1 </engine> <method> 2 </method> <word> George </word> <word> Bush </word> </query> <query> <engine> 2 </engine> <method> 3 </method> <word> Bill </word> <word> Clinton </word> </query>

  13. Process: Clue Extraction queries Extraction Software clues Clues Extraction Software <query> clues

  14. Data to be analyzed • For each page • Query used to find page • Source URL • For each image • Source URL • Attributes • Position in document • For each clue • Whether clue feature occurs in document at all • If feature occurs with text matching the query • Position in document for each occurrence

  15. Process: Relevance Rating Query & image queries Relevant Rating Software <query> Human Relevant/not Presents images from each query to the user from the database, and record the human relevance rating back to the database.

  16. Clues : global Global Clues - clues that apply every image on the page - filename of page - path of page - host of page - title element of the web page - keywords found in meta element - description found in meta element Why do we break the URL into three clues? Different parts of the URL contributes different relevance factor to the overall relevance of the image in that page.

  17. Clues : global <HEAD> <TITLE>Apple</TITLE> <META NAME="keywords" CONTENT="Apple Computer, Power Macintosh, PowerBook, AppleWorks, WebObjects, iMovie, QuickTime, Desktop Movies, Software, Operating Systems, Mac OS, iMac, iBook"> <META NAME="Description" CONTENT="Visit www.apple.com for the latest news, the hottest products, and technical support resources from Apple Computer, Inc."> <META HTTP-EQUIV="Expires" CONTENT="Fri, 26 Mar 1999 23:59:59 GMT"> <META NAME="Date.Modified" CONTENT="19992109"> </HEAD>

  18. Clues : image file Image file properties - external properties - filename - path - host An image can be from another host, and have different paths.

  19. Clues : common attributes Elements have common attributes -title - describe what the element is -id - used in identifying the element -name - same as id, older HTML Clues that use these attributes: link, image, object, table, cell, row

  20. Clues: Image Container Link to an image <a> - text enclosed within the link element Embed image element <img> -alt attribute (usually describes what the image is) Object element <object> -text that enclosed within the object element

  21. Clues: table Table (<table>): - summary attribute - describes the table content - caption - describes table content - row heading - row - column heading - column - cell - neighboring cells (above, below, right, left)

  22. Clues: table

  23. Clues: table

  24. Clues: table

  25. Clues: table

  26. Clues: table

  27. Clues: table

  28. Clues: table

  29. Clues: headings Heading elements(h1, h2, ..h6) - headings above image - headings below image <h1>header above image</h1> <img src=“sample.jpg”> <h2>header below image</h2> Heading can indicate a topic and images below the heading maybe relate to the heading. Some use headings as caption above images, and sometimes below images. Some headers are used where fonts should be used or bold should be used.

  30. In this photo, the heading comes after the image. Often if it is used as a topic, it usually comes before the image. But some images have heading as caption below the images.

  31. Clues : text Emphasized text elements - bold - italic - underline - strong - emphasis - big Body text - text that surrounds the image - distance

  32. Current Project Status • Prototype download and clue extraction software nearly complete • now testing implementation • - data (without human relevance ratings) in early November • Recruiting students to build on-line relevance rating system • hope to get students outside lab to help with ratings via Web interface

  33. Challenges for image search systems • - computing word distance from image • - Stylesheet used for presentation • - table pattern • - pattern of HTML elements usage • - CGI returned images • structural boundaries • patterns in Web page design • -HTML generators

  34. Cheng Thao, chengt@uwm.edu

More Related