1 / 28

Chapter 8 Web Structure Mining

Chapter 8 Web Structure Mining. Part-1. Web Structure Mining. Deals mainly with discovering the model underlying the link structure of the web Deals with the topology of hyperlinks with or without the description of the links. Why?. The model can be used to classify web pages.

masao
Download Presentation

Chapter 8 Web Structure Mining

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 8Web Structure Mining Part-1

  2. Web Structure Mining • Deals mainly with discovering the model underlying the link structure of the web • Deals with the topology of hyperlinks with or without the description of the links

  3. Why? • The model can be used to classify web pages. • Helpful to create information such as the similarity and relationship between different websites. • Useful for discovering website type.

  4. Website type • Web structure mining is a suitable tool for discovering authority sites and overview sites for the subjects • Authority sitescontain information about the subject • Overview sitespoint to many authority sites

  5. Web Content Mining/ Web Structure Mining • Web Content Mining explores the structure within the document • Web Structure Mining studies citation relationship of documents within the web.

  6. Algorithms for Web Structure Mining PageRank algorithm (Google Founders) • Looks at number of links to a website and importance of referring links • Computed before the user enters the query. HITS algorithm (Hyperlinked Induced Topic Search) • User receives two lists of pages for query (authority and link pages) • Computations are done after the user enters the query.

  7. PageRank

  8. PageRank Algorithm • The idea of the algorithm came from academic citation literature. • It was developed in 1998 as part of the Google search engine prototype • Studies citation relationship of documents within the web. • Google search engine ranks documents as a function of both the query terms and the hyperlink structure of the web.

  9. Definition of PageRank • The PageRank produces ranking independent of a user’s query. • The importance of a web page is determined by the number of other important web pages that are pointing to that page and the number of out linksfrom other web pages.

  10. An art draw drawn by Felipe MicaroniLalli (micaroni@gmail.com).

  11. Example of Backlinks Page A is a backlink of page B and page C, while page B and page C are backlinks of page D. Backlink = Outlink= OutDegree

  12. Example-1 PR(A)=0.25+0.25+0.25 PR(A)=0.75 A B C D

  13. Example-2 PR(A)= PR(B)/2+ PR(C)/1+ PR(D)/3 = 0.125+0.25+0.0833 =0.4583 A B D C

  14. Page Ranking A page will have high page rank if: • There are many pages pointing to it. • There are some pages pointing to it which have high page ranks. In other words: • Pages well sited from around the web are worth looking at. • Pages that only have one citation from high rating web page is worth looking at.

  15. Damping Factor • The PageRank theory holds that even an imaginary surfer who is randomly clicking on links will eventually stop clicking. The probability, at any step, that the person will continue is a damping factor d.

  16. Damping Factor d The damping factor is subtracted from 1 and this term is then added to the product of the damping factor and the sum of the incoming PageRank scores. So any page's PageRank is derived in large part from the PageRanks of other pages. The damping factor adjusts the derived value downward.

  17. Computing PageRank The PageRank of a page u is computed as follows: where, OutDegree(v)represents the number of links going out of the page v and parameter d be a damping factor, which can be a real number between 0 and 1. The value of d is generally taken as 0.85.

  18. PageRank Algorithm

  19. Applied Example

  20. A Simple Network of Pages(Ian Roger, 2006) • OutDegree(A) = 1 and OutDegree(B) = 1). • Here, we do not know what their PageRanks should be to begin with, so we can take a guess at 1.0 , assuming d=0.85, and perform following calculations • PageRank(A)= (1 – d) + d (PageRank(B)/1) • PageRank(B)= (1 – d) + d (PageRank(A)/1) • PageRank(A)= 0.15 + 0.85 * 1=1 • PageRank(B)= 0.15 + 0.85 * 1=1 • We calculated that the PageRank of A and B is 1.

  21. A Simple Network of Pages(Ian Roger, 2006) Now, we plug in 0 as the guess and perform calculations again: PageRank(A) = 0.15 + 0.85 * 0= 0.15 PageRank(B) = 0.15 + 0.85 * 0.15= 0.2775 We have now another guess for PageRank(A) so we use it to calculate PageRank(B) and continue: PageRank(A) = 0.15 + 0.85 * 0.2775 = 0.3859 PageRank(B) = 0.15 + 0.85 * 0.3859 = 0.4780

  22. Example-cont. Repeating the calculations, we get: PageRank(A) = 0.15 + 0.85 * 0.4780 = 0.5563 PageRank(B) = 0.15 + 0.85 * 0.5563 = 0.6229 If we repeat the calculations, eventually the PageRanks for both the pages converge to 1.

  23. Rank Sink • A, and B both have rank, but they will never circulate any rank. A A D

  24. Remarks on PageRank Remarks on PageRank Algorithm: • A page with no successors has no scope to send its importance. As well, a group of pages that have no links out of the group will eventually collect all the importance of the Web.

  25. PageRank Toolbar

  26. Sample Scores with Their Meaning

  27. Toolbar PageRank and Corresponding Real PageRank

  28. Activity • There is alink between page A to both B and C. Also there is a link from pages B and C to A. • Begin with intial value of PageRank as 0. • Complete 6 iterations A B C

More Related