The PageRank Citation Ranking: Bringing Order to the Web Lawrence Page and Sergey Brin , Stanford University. 1998.

1. The PageRank Citation Ranking: Bringing Order to the WebLawrence Page and Sergey Brin, Stanford University. 1998. Presented by Ashish Chawla and Vinit Asher

2. Agenda

3. Introduction • Challenges in Information Retrieval on Web • Large # of documents • Heterogeneous and Unstructured • WWW • Is hypertext • provides auxiliary information (other than the text of web pages) • Objective • Take advantage of this link structure.

4. Background • Academic Citations • link to other well known papers • peer reviewed • have quality control • Web : • Homogeneous in their quality, usage, citation & length • Quality measure (subjective to the user) • Importance of a page is a quantity that isn’t intuitively possible to capture

5. What does a user want? • Most applicable documents first • What is the job of a retrieval system? • Present more relevant documents upfront • Notion: Quality/Importance of Web Pages • Difficult to classify (depends on user) • We deal with the overall importance of a page, rather than individual sections of the page.

6. Link Structure • Forward Links • Back Links • Web has 150 million pages and 1.7 billion links (probably more now) • Use the concept of citation analysis • Highly linked pages are more “important" than pages with few links

7. Propagation of Ranking Page Rank: a page has high rank if the sum of the ranks of its back-links is high Some notations

8. Simplified Page Rank Calculation

9. Problem in Ranking? • Rank Sink: • Two web pages that point to each other but to no other page. Third page which points to one of them. • loop will accumulate rank but never distribute it (since there are no outedges).

10. Page Rank Definition Let E(u) be some vector over the Web pages that corresponds to a source of rank. Then, the PageRank of a set of Web pages is an assignment, R’, to the Web pages which satisfies such that c is maximized and ||R’||1 = 1 (||R’||1 denotes the L1 norm of R’).

11. Computing Page Rank

12. Random Surfer Model Random Surfer Clicks at random basis “Surfer” periodically gets bored.

13. Solution to Random Surfer Model Escape term: E(u) can be thought of as the random surfer gets bored periodically and jumps to a different page – not staying in the loop forever. We term this E to be a vector over all the web pages that accounts for each page’s escape probability (user defined parameter).

14. Another Problem – Dangling Links • What are dangling links? • Links that point to any page with no outgoing links. • Pages not downloaded yet. • Why is this a problem? • We don’t know how to distribute weight to these. • What do we do ? • Remove them from the system

15. Mathematical Basics • What is eigen vector and eigen value? • Given a vector v in the n-dimensional vector space, we can linear transform it to another vector space using a transformation matrix A. The transformed vector is Av. • An eigen vector is a vector that is scaled by a linear transformation, but not moved. The scaling factor is the eigen value. Eigen values and eigen vectors are not unique. We can compute them by Ax = x where is the eigen value of A and x is the corresponding eigen vector. • An eigenvector is a vector that 'points' in the same direction (has invariant direction cosines) under some transform. The eigenvalue is a number that describes how the magnitude of the eigenvector is scaled by the transform.

16. Mathematical Basics A is designated to be a matrix, u and v correspond to the columns of this matrix. Given that A is a matrix, and R be a vector over all the Web pages, the dominant eigenvector is the one associated with the maximal eigenvalue.

17. Example AT=

18. Example (contd..) • R = c A R = M R • c: eigenvalue • R : eigen vector of A A = A x = λ x | A - λI | x = 0 R = Normalized =

19. Implementation • Web crawler keeps a database of URLs so that it can discover all URLs on the web • To implement PageRank, the web crawler builds an index of the URLs as it crawls • Problems??? • Infinitely large sites • Incorrect HTML • Sites are down • Web is always changing

20. PageRank Implementation Convert each URL into unique integer ID Link structure sorted by the IDs Remove dangling links Make a initial assignment of ranks and iterate until convergence Add the dangling links back Iterate the process again to assign weights to all dangling links Link database A, is normally kept in RAM

21. Convergence Properties PageRank will scale very well for large collections as the scaling factor is roughly linear in log n.

22. Convergence Properties Here we interpret web as a expander like graph. A graph is said to be expander if every subsets of nodes S has a neighborhood that is larger than some factor α times |S| Mathematically we verify the same if the largest eigenvalue is sufficiently larger than the second-largest eigenvalue

23. Searching with PageRank • Two search engines implemented using PageRank. • Title based search engine • Matches titles of web pages with the given query • Ranks the results using PageRank • Works well for general queries having a large result set • Full text search engine (Google) • Scans the entire document for a match with the given query • Performs rank merging.

24. Types of Results • Information based result • Finds a site which contains great deal of information • Propagates textual matching score through the link structure • Common Case result • Most commonly used site (often commercial) relevant to the search query • PageRank results in good representation of the common case

25. Personalized PageRank • E vector • Corresponds to a distribution of web pages • Provides flexibility in adjustment of PageRanks • Uniform E causes highly linked web pages to achieve a very high ranking • Single page E results in important pages not related to the homepage to achieve a low PageRank • E consisting of root level pages of all web servers is a good compromise between uniform E and single page E

26. Applications • Estimating Web Traffic • Looking at differences between PageRank and actual usage statistics, it is possible to find things that people often look at, but do not want to link to their web pages • Backlink Predictor • Citation counts tends to get stuck in the local web pages • Using random surfer model, PageRank quickly finds the site homepage, and gives preference to its children resulting in an efficient, broad search • Hence PageRank potentially acts as a better backlink predictor since it builds up the entire website information faster

27. Other Applications Spam detection and prevention Sort the backlinks based on their importance

28. Issues Users are not random walkers. Starting point distribution (actual usage data as starting vector). Bias towards main pages. Linkage spam. No query specific rank.

29. Conclusion PageRank is a global ranking of all webpages, regardless of their content based solely on their location in the Web’s graph structure PageRank can be used to separate a small set of commonly used documents Full database is consulted only when small database is not adequate to answer the queries Personalized PageRank can be used to create a view of Web from a particular user’s perspective

30. Google Architecture..