1 / 17

CRAWLING THE WEB

CRAWLING THE WEB. CRAWLING THE WEB. What do you do when you need information from the internet? . Search Engines. directories. Open directory project (DMOZ). Meta-search engines. FINDING INFORMATION ON THE WEB. SEARCH ENGINES DIRECTORIES META-SEARCH ENGINES.

kent
Download Presentation

CRAWLING THE WEB

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CRAWLING THE WEB

  2. CRAWLING THE WEB • What do you do when you need information from the internet?

  3. Search Engines

  4. directories Open directory project (DMOZ)

  5. Meta-search engines

  6. FINDING INFORMATION ON THE WEB SEARCH ENGINES DIRECTORIES META-SEARCH ENGINES

  7. How does a SEARCH ENGINE work? • Search engines use a computer program called a SPIDER to roam the World Wide Web pages and their links.

  8. How does a search Engine work? • The spider collects the information and then indexes all the information.

  9. HOW does a search Engine Work? • Each search engine’s spider indexes and organizes the Web pages • While indexing, matches between keywords and Web pages are found. The sites with the best matches are displayed first. Each search engine has a different way of identifying the best sites.

  10. How does a search engine work?

  11. How does a search engine work? • ROAMS and COLLECTS INFORMATION • INDEXES ALL THE INFORMATION • MATCHES THE INFORMATION These 3 tasks are all done WITHOUT ANY HUMAN INVOLVEMENT– so a huge number of sites are indexed quickly.

  12. How does a directory work? • In a DIRECTORY, PEOPLE, not computers, put the index together.

  13. How does a directory work? • Editors evaluate Web sites and organize them into subject categories. • Because people have chosen them, the sites in directories may be of higher QUALITY.

  14. How does a directory work? • The number of sites in a DIRECTORY is usually much SMALLER than in a search engine’s index. • Many people use the term “SEARCH ENGINE” to describe either a search engine or a directory. That is because many search sites offer both services.

  15. How does a meta-search engine work? • A META-SEARCH ENGINE sends your keywords to several search engines at the same time. • The results from each search engine are organized and displayed on one page.

  16. How does a meta-search engine work? • This type of service is useful when your topic is very NARROW and you want to search as many Web sites as possible.

  17. Remember … • No one search engine, directory or meta-search engine covers the entire Web. So, don’t get stuck in a rut by using only one. Try them all!

More Related