1 / 125

Finding patterns in large, real networks

This presentation by Christos Faloutsos explores the patterns and properties of various real networks, such as protein interactions, internet maps, and food webs. It delves into the topology, laws, and generators that govern these networks and highlights the importance of understanding and analyzing them. The talk also discusses the application of these findings in areas like virus propagation, immunization strategies, and algorithm design.

mbroderick
Download Presentation

Finding patterns in large, real networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Finding patterns in large, real networks Christos Faloutsos CMU www.cs.cmu.edu/~christos/TALKS/UCLA-05

  2. Thanks to • Deepayan Chakrabarti (CMU) • Michalis Faloutsos (UCR) • George Siganos (UCR) (c) 2005 C. Faloutsos

  3. Introduction Protein Interactions [genomebiology.com] Internet Map [lumeta.com] Food Web [Martinez ’91] Graphs are everywhere! Friendship Network [Moody ’01] (c) 2005 C. Faloutsos

  4. Physical graphs • Physical networks • Physical Internet • Telephone lines • Commodity distribution networks (c) 2005 C. Faloutsos

  5. Networks derived from "behavior" • Telephone call patterns • Email, Blogs, Web, Databases, XML • Language processing • Web of trust, epinions.com (c) 2005 C. Faloutsos

  6. Outline Topology, ‘laws’ and generators • ‘Laws’ and patterns • Generators • Tools (c) 2005 C. Faloutsos

  7. Motivating questions • What do real graphs look like? • What properties of nodes, edges are important to model? • What local and global properties are important to measure? • How to generate realistic graphs? (c) 2005 C. Faloutsos

  8. Why should we care? • A1: extrapolations: how will the Internet/Web look like next year? • A2: algorithm design: what is a realistic network topology, • to try a new routing protocol? • to study virus/rumor propagation, and immunization? (c) 2005 C. Faloutsos

  9. Why should we care? (cont’d) • A3: Sampling: How to get a ‘good’ sample of a network? • A4: Abnormalities: is this sub-graph / sub-community / sub-network ‘normal’? (what is normal?) (c) 2005 C. Faloutsos

  10. Virus propagation • Who is the best person/computer to immunize against a virus? (c) 2005 C. Faloutsos

  11. Outline Topology, ‘laws’ and generators • ‘Laws’ and patterns • Generators • Tools (c) 2005 C. Faloutsos

  12. Topology How does the Internet look like? Any rules? (Looks random – right?) (c) 2005 C. Faloutsos

  13. Laws and patterns Real graphs are NOT random!! • Diameter • in- and out- degree distributions • other (surprising) patterns (c) 2005 C. Faloutsos

  14. Laws – degree distributions • Q: avg degree is ~2 - what is the most probable degree? count ?? degree 2 (c) 2005 C. Faloutsos

  15. WRONG ! count ?? count degree 2 2 Laws – degree distributions • Q: avg degree is ~3 - what is the most probable degree? degree (c) 2005 C. Faloutsos

  16. I.Power-law: outdegree O The plot is linear in log-log scale [FFF’99] freq = degree (-2.15) Frequency Exponent = slope O = -2.15 -2.15 Nov’97 Outdegree (c) 2005 C. Faloutsos

  17. II.Power-law: rank R • The plot is a line in log-log scale outdegree Exponent = slope R = -0.74 R Dec’98 Rank: nodes in decreasing outdegree order (c) 2005 C. Faloutsos

  18. B C A D III. Eigenvalues • Let A be the adjacency matrix of graph • and v is an eigenvalue/eigenvector pair if: A v =  v • Eigenvalues are strongly related to graph topology (c) 2005 C. Faloutsos

  19. III.Power-law: eigen E Eigenvalue • Eigenvalues in decreasing order (first 20) • [Mihail+, 02]: R = 2 * E Exponent = slope E = -0.48 Dec’98 Rank of decreasing eigenvalue (c) 2005 C. Faloutsos

  20. IV. The Node Neighborhood • N(h) = # of pairs of nodes within h hops (c) 2005 C. Faloutsos

  21. IV. The Node Neighborhood • Q: average degree = 3 - how many neighbors should I expect within 1,2,… h hops? • Potential answer: 1 hop -> 3 neighbors 2 hops -> 3 * 3 … h hops -> 3h (c) 2005 C. Faloutsos

  22. IV. The Node Neighborhood • Q: average degree = 3 - how many neighbors should I expect within 1,2,… h hops? • Potential answer: 1 hop -> 3 neighbors 2 hops -> 3 * 3 … h hops -> 3h WRONG! WE HAVE DUPLICATES! (c) 2005 C. Faloutsos

  23. IV. The Node Neighborhood • Q: average degree = 3 - how many neighbors should I expect within 1,2,… h hops? • Potential answer: 1 hop -> 3 neighbors 2 hops -> 3 * 3 … h hops -> 3h WRONG x 2! ‘avg’ degree: meaningless! (c) 2005 C. Faloutsos

  24. IV. Power-law: hopplot H H = 2.83 # of Pairs H = 4.86 # of Pairs Pairs of nodes as a function of hops N(h)= hH Hops Router level ’95 Dec 98 Hops (c) 2005 C. Faloutsos

  25. ... Observation • Q: Intuition behind ‘hop exponent’? • A: ‘intrinsic=fractal dimensionality’ of the network N(h) ~ h1 N(h) ~ h2 (c) 2005 C. Faloutsos

  26. Hop plots • More on fractal/intrinsic dimensionalities: very soon (c) 2005 C. Faloutsos

  27. But: • Q1: How about graphs from other domains? • Q2: How about temporal evolution? (c) 2005 C. Faloutsos

  28. The Peer-to-Peer Topology • Frequency versus degree • Number of adjacent peers follows a power-law [Jovanovic+] (c) 2005 C. Faloutsos

  29. More Power laws • Also hold for other web graphs [Barabasi+, ‘99], [Kumar+, ‘99] • citation graphs (see later) • and many more (c) 2005 C. Faloutsos

  30. Time Evolution: rank R Domain level #days since Nov. ‘97 The rank exponent has not changed! [Siganos+, ‘03] (c) 2005 C. Faloutsos

  31. Outline Part 1: Topology, ‘laws’ and generators • ‘Laws’ and patterns • Power laws for degree, eigenvalues, hop-plot • ??? • Generators • Tools Part 2: PageRank, HITS and eigenvalues (c) 2005 C. Faloutsos

  32. Any other ‘laws’? Yes! (c) 2005 C. Faloutsos

  33. Any other ‘laws’? Yes! • Small diameter • six degrees of separation / ‘Kevin Bacon’ • small worlds [Watts and Strogatz] (c) 2005 C. Faloutsos

  34. Any other ‘laws’? • Bow-tie, for the web [Kumar+ ‘99] • IN, SCC, OUT, ‘tendrils’ • disconnected components (c) 2005 C. Faloutsos

  35. Any other ‘laws’? • power-laws in communities (bi-partite cores) [Kumar+, ‘99] Log(count) n:1 n:3 n:2 2:3 core (m:n core) Log(m) (c) 2005 C. Faloutsos

  36. Any other ‘laws’? • “Jellyfish” for Internet [Tauro+ ’01] • core: ~clique • ~5 concentric layers • many 1-degree nodes (c) 2005 C. Faloutsos

  37. How do graphs evolve? • degree-exponent seems constant - anything else? (c) 2005 C. Faloutsos

  38. Evolution of diameter? • Prior analysis, on power-law-like graphs, hints that diameter ~ O(log(N)) or diameter ~ O( log(log(N))) • i.e.., slowly increasing with network size • Q: What is happening, in reality? (c) 2005 C. Faloutsos

  39. Evolution of diameter? • Prior analysis, on power-law-like graphs, hints that diameter ~ O(log(N)) or diameter ~ O( log(log(N))) • i.e.., slowly increasing with network size • Q: What is happening, in reality? • A: It shrinks(!!), towards a constant value (c) 2005 C. Faloutsos

  40. Shrinking diameter ArXiv physics papers and their citations [Leskovec+05a] (c) 2005 C. Faloutsos

  41. Shrinking diameter ArXiv: who co-authored with whom (c) 2005 C. Faloutsos

  42. Shrinking diameter U.S. patents citing each other (c) 2005 C. Faloutsos

  43. Shrinking diameter Autonomous systems (c) 2005 C. Faloutsos

  44. Temporal evolution of graphs • N(t) nodes; E(t) edges at time t • suppose that N(t+1) = 2 * N(t) • Q: what is your guess for E(t+1) =? ...* E(t) (c) 2005 C. Faloutsos

  45. Temporal evolution of graphs • N(t) nodes; E(t) edges at time t • suppose that N(t+1) = 2 * N(t) • Q: what is your guess for E(t+1) =? ...* E(t) • A: over-doubled! (c) 2005 C. Faloutsos

  46. Temporal evolution of graphs • A: over-doubled - but obeying: E(t) ~ N(t)a for all t where 1<a<2 a=1: constant avg degree a=2: ~full clique • Real graphs densify over time [Leskovec+05a] (c) 2005 C. Faloutsos

  47. Temporal evolution of graphs • A: over-doubled - but obeying: E(t) ~ N(t)a for all t • Identically: log(E(t)) / log(N(t)) = constant for all t (c) 2005 C. Faloutsos

  48. E(t) N(t) Densification Power Law ArXiv: Physics papers and their citations 1.69 (c) 2005 C. Faloutsos

  49. E(t) N(t) Densification Power Law U.S. Patents, citing each other 1.66 (c) 2005 C. Faloutsos

  50. E(t) N(t) Densification Power Law Autonomous Systems 1.18 (c) 2005 C. Faloutsos

More Related