1 / 20

Internet Resources Discovery (IRD)

Internet Resources Discovery (IRD). Intelligent IRD. Motivation for Intelligence. “ We are drowning in information but starved of knowledge “ John Naisbit. Content. Classical IRD characteristics and the Information food chain Agents - Softbots family Meta SE - Metacrawler

claire
Download Presentation

Internet Resources Discovery (IRD)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Internet Resources Discovery (IRD) Intelligent IRD T.Sharon-A.Frank

  2. Motivation for Intelligence “We are drowning in information but starved of knowledge“ John Naisbit T.Sharon-A.Frank

  3. Content • Classical IRD characteristics and the Information food chain • Agents - Softbots family • Meta SE - Metacrawler • Homepage finder - Ahoy! • ILA – Internet Learning Agent • Shopbot – Jango et al. • See Oren Ezioni’s Web site at:http://www.cs.washington.edu/research/projects/WebWare1/www/softbots/softbots.html T.Sharon-A.Frank

  4. Classical IRD Characteristics • Massive memory and network resources required. • Amortized over millions of queries per day. • Minimal cycles devoted to each individual. • No memory of previous requests. • Least common denominator service. No Time for Intelligence! T.Sharon-A.Frank

  5. Classical Information Food Chain T.Sharon-A.Frank

  6. Intelligent Information Food Chain T.Sharon-A.Frank

  7. Definition: Softbots • Softbots are intelligent agents that use software tools and services on a person’s behalf. • Make intensive use of artificial intelligence (AI) techniques: planning, scheduling, learning, etc. T.Sharon-A.Frank

  8. Softbot Family Tree BargainFinder Rodney Sims Simon ILA MetaCrawler InfoManifold Occam Ahoy! ShopBot T.Sharon-A.Frank

  9. General problems to be solved • Discovery • How to find new information sources (IS) ? • Extraction • What to send and how to parse the response ? • Translation • How to interpret the response in terms of internal concepts ? • Evaluation • How to evaluate the quality of IS ? T.Sharon-A.Frank

  10. Main Focus of the Robots Metacrawler Discovery, Evaluation: Ahoy! Extraction: ILA Translation: T.Sharon-A.Frank

  11. Meta Search Engine MetaCrawler Yahoo Web Crawler Open Text Lycos InfoSeek Inktomi Galaxy Excite T.Sharon-A.Frank

  12. Search Service - Motivation 1. The number and variety of Search services. 2. Each service provides an incomplete snapshot of Web. 3. Users are forced to try and retry their queries across different indices. 4. Each service has its own interface. 5. Irrelevant, outdated or unavailable responses. 6. There is no time for intelligence. 7. Each query is independent. 8. No individual customization. 9. The result is not homogenized. T.Sharon-A.Frank

  13. The Web Community Demands • Robustness • A working system, accessible 24 hours a day. • Speed • Transmitting useful information within seconds. • Added Value • Any increase in sophistication had better yield a tangible benefit to users. T.Sharon-A.Frank

  14. Premises of MetaCrawler • No single search is sufficient. • Problem in expressing the query. • Low quality references can be detected. T.Sharon-A.Frank

  15. MetaCrawler T.Sharon-A.Frank

  16. MetaCrawler is a Meta-Service • It doesn’t use a database of its own. • It uses other external search services that provide the information necessary to fulfill user queries. T.Sharon-A.Frank

  17. MetaCrawler Advantages • It access multiple databases and provides large number of higher quality references. • It does not depend upon the implementation or existence of any specific search service. • It access the search services simultaneously. • Users need not remember the address, interfaces, … of each search service. T.Sharon-A.Frank

  18. How It Works? • It currently accesses a few services: InfoSeek, Lycos, WebCrawler, Yahoo, etc. • It submits a query to every search service it knows in parallel. • It collates the results by merging all hits returned. • It has a sorting and verify option. • It presents a results page consisting of a list of references. T.Sharon-A.Frank

  19. Meta-Search • http://www.metacrawler.com T.Sharon-A.Frank

  20. Meta Search Results T.Sharon-A.Frank

More Related