1 / 28

Enhancing Internet Search Engines to Achieve Concept-based Retrieval

Enhancing Internet Search Engines to Achieve Concept-based Retrieval. F. Lu, T. Johnsten, V. Raghavan, and D. Traylor. Agenda. Information on the Internet. Boolean Retrieval Model and the Internet. Personalized Search. Concept-Based Retrieval (RUBRIC / CS 3 ).

alicehubbard
Download Presentation

Enhancing Internet Search Engines to Achieve Concept-based Retrieval

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Enhancing Internet Search Engines to Achieve Concept-based Retrieval F. Lu, T. Johnsten, V. Raghavan, and D. Traylor

  2. Agenda • Information on the Internet. • Boolean Retrieval Model and the Internet. • Personalized Search. • Concept-Based Retrieval (RUBRIC / CS3). • CS3 and Boolean Search Engines. • Deep Web Sources. • Current & Future Work.

  3. Information on the Internet • Large volume. • Rapid growth rate. • Wide variations in quality and type.

  4. Boolean Retrieval Model and the Internet • Most Internet search engines are based on the Boolean Retrieval Model. • Boolean Retrieval Model is relatively easy to implement. • Limitations: • Inability to assign weights to query or document terms. • Inability to rank retrieved documents. • Naïve users have difficulty in using

  5. Personalized Search Personalized Results User Query Personalized Engine Query Processor User Profile General Profile Result Processor Query Augmentation Search Results Search Engine

  6. Concept-Based Retrieval • Address shortcomings of Boolean Retrieval Model. • Search Requests specified in terms of concepts structured as rule-base trees.

  7. Development of Rule-Base Trees (General) • Top-down refinement strategy. • Support for AND / OR relationships. • Support for user-defined weights.

  8. Development of Rule-Base Trees (CS3) • Concept-Set Structuring System (CS3) • CS3 supports the creation, storage and modification of user-defined concepts • Post-processing of results of sub-queries • CS3 user-interface.

  9. CS3 User Interface

  10. Evaluation of Rule-Base Trees (RUBRIC) • Run-time, bottom-up analysis. • Propagation of weight values (MIN / MAX). • Disadvantage of run-time analysis.

  11. Evaluation of Rule-Base Trees (CS3) • Static, bottom-up analysis. • Construct Minimal Term Set (MTS). • Propagation of terms. • CS3 user-interface.

  12. MTS-Minimal Term Set • A MTS for a topic is a set of terms such that if each term in the set appears in the document, the document would get a RSV larger than 0. If not, the RSV would be 0. • A topic could have more than one MTSs. • A user can choose from those MTSs to perform a search to his needs.

  13. CS3 and Boolean Search Engines • CS3 is designed to interface with existing Boolean search engines. • U.S. Department of Energy’s “Information-Bridge” search engine. • U.S. Department of Transportation’s “National Transportation Library” search engine.

  14. System Architecture Client (Java/ Applet ) CORBA CGI Server (JAVA) Server (JAVA/C++) JDBC DOE InfoBridge etc. … ORACLE

  15. Information-Bridge and CS3 • Search request: Boolean Vs. Concept • Output: Non-Ranked Vs. Ranked. • Calculation of RSV: • Given a document D and a set S of MTS expressions satisfied by D, the RSV of D is equal to the sum of all the weights of S plus the maximum weight in S.

  16. Information-Bridge and CS3 (Example) • Boolean search request (“Environmental Science Network” Form): • (“Hydrogeology” OR “Dnapl” OR (“Colloid*” AND “Environmental Transport”)). • Concept (CS3): • “Hydrogeology”. • Rule-Base Tree.

  17. CS3 Hydrogeology Rule Base

  18. CS3 search results

  19. Deep Web Sources • Also referred to as hidden Web or invisible Web • Resides behind search forms in databases e.g. monster.com, louisiana1st.com, PubMed. • Web pages in deep Web are generated dynamically based on the submitted queries. • Not indexed by current search engines. Search engines index content on the surface Web.

  20. Deep Web Sources and Concept-based Retrieval • Deep Web in terms of size and quality: Size (Deep Web) = 500 * Size (Surface Web) Quality (Deep Web) = 1000 * Quality (Surface Web) • Queries submitted at deep Web sources are more stable compared to queries submitted to search engines • So, naturally concept-based retrieval is more suitable for deep Web sources

  21. Current and Future Work • Conduct experiments to evaluate effectiveness (future). • Investigate alternative methods to compute RSVs [KADR00, KDR01*]. • Learning edge weights through relevance feedback [KR00]. • Thesaurii based rulebase generation [KLR00].

  22. Relevant URLs [LJRT99*] RaghavanHome  Publications since 1991 www.allinonenews.com

More Related