Lecture 4: Connect (4/4)

1. Lecture 4: Connect (4/4) How the friendship we form connect us? Why we are within a few clicks on Facebook? COMS 4995-1: Introduction to Social Networks Tuesday, September 18th

2. Some announcements • This course is now officially “sexy [kinda]”congratulations! • 1st assign. due Thursday 4:10pm • Part A+C on papers! • Part B(+raw results of C) on dropbox • Sign the cover sheet • 1 late days: 5% (you have 3 free during semester)

3. Outline Milgram’s “small world” experiment It’s a “combinatorial small world” It’s a “complex small world” It’s an “algorithmic small world”

4. Small-world model • Collective dynamics of ‘small-world’ networks. • D. Watts, S. Strogatz, Nature (1998) • Main idea: social networks follows a structure with a random perturbation • Formal construction: • Connect all nodes at distance in a regular lattice • Rewire each edge uniformly with probability p(variant: connect each node to q neighbors, chosen uniformly)

5. Small-world model • Collective dynamics of ‘small-world’ networks. • D. Watts, S. Strogatz, Nature (1998) Main idea: social networks follows a structure with a random perturbation

6. Outline Milgram’s “small world” experiment It’s a “combinatorial small world” It’s a “complex small world” It’s an “algorithmic small world”

7. Where are we so far? Analogy with a cosmological principle Are you ready to accept a cosmological theory that does not predict life? In other words, let’s perform a simple sanity check

8. A thought experiment • Consider a randomly augmented lattice (N nodes)

9. A thought experiment • Consider a randomly augmented lattice (N nodes) • Perform “small world” Milgram experiment Can you tell what will happen? • The folder arrives in 6 hops • The folder arrives in O(ln(N)) hops • The folder never arrives • I need more information

10. A thought experiment • The folder arrives in 6 hops NOT TRUE • It actually does look like a naive answer • More precisely: • By previous result we know that shortest paths is of the order of ln(N), which contradicts this statement.

11. A thought experiment (b) The folder arrives in O(ln(N)) ACCORDING TO OUR PRINCIPLE, OUGHT TO BE TRUE BECAUSE IT WAS OBSERVED BY MILGRAM • A sufficient conditionfor this to be true is: • Milgram’s procedure extract shortest path • Answering this critical question boils down to an algorithmic problem

12. A thought experiment (c) The folder never arrives SEEMS UNLIKELY unless the procedure is badly designed (cycle) or we model people dropping or if the grid contains hole

13. A thought experiment (d) I need more information • In particular, how to model Milgram’s procedure • “If you do not know a target, forward the folder to your friend or acquaintance that is most likely to know her.”

14. What is Greedy Routing? • A mathematical model of what Milgram measured • Participants know where the target is located • They use grid information + shortcuts “incidentally” N.B.: Grid “dimensions” can describe geography or other sociological property (occupation, language) • Example:

15. How does greedy routing perform? • Does it extract the shortest path? • Not necessarily, this is why we need to analyze it! • Case study: dimension k=1, target t, starting from u0 • We introduce interval: • The greedy routing constructs a pathwe denote the end-point of the ith shortcuts as

16. Analysis of Greedy routing • CLAIM: If none of are in and we start from u0 outside • Then greedy routing needs at least min(n,l) steps

17. How does greedy routing perform? • Fixing , this event has proba ≤1/2 • So with proba ≥1/2, are not in • On this event, assuming s not in • Greedy routing needs more than n steps • Or it has to reach t from boundary of , using l steps

18. A thought experiment • In a line Milgram’s uses steps • square root is notsatisfying for small world • Not much better when k>1 ! • even worse, the proof applies to any distributed alg. • Our sanity check test has grandly failed! • “Small world” results explain that short paths exist … findingthem remains a daunting algorithmic task

19. Outline • Milgram’s “small world” experiment • It’s a “combinatorial small world” • It’s a “complex small world” • It’s an “algorithmic small world” • Beyond uniform random augmentation

20. Autopsy of “Small-world” failure • In a uniformly augmented lattice shortcuts do exist • About shorcuts leads to when • But they are dispersed among nodes • Moreover, previous steps do not lead to progress • So need about N/√N = √N trials • Is there another augmentation?

21. The 10 papers that will make you a social expert

22. 10 sociological must-reads S.Milgram, “The small world problem,” Psychology today, 1967. M. Granovetter, “The strength of weak ties: A network theory revisited,” Sociological theory, vol. 1, pp. 201–233, 1983. M. McPherson, L. Smith-Lovin, and J. M. Cook, “Birds of a Feather: Homophily in Social Networks,” Annual review of sociology, vol. 27, pp. 415–444, Jan. 2001. M. O. Lorenz, “Methods of measuring the concentration of wealth,” Publications of the American Statistical Association, vol. 9, no. 70, pp. 209–219, 1905. + H. Simon, “On a Class of Skew Distribution Functions,” Biometrika, vol. 42, no. 3, pp. 425–440, 1955. R. I. M. Dunbar, “Coevolution of Neocortical Size, Group-Size and Language in Humans,” Behav Brain Sci, vol. 16, no. 4, pp. 681–694, 1993. D. Cartwright and F. Harary, “Structural balance: a generalization of Heider's theory.,” Psychological Review, vol. 63, no. 5, pp. 277–293, 1956. M. Granovetter, “Threshold Models of Collective Behavior,” The American Journal of Sociology, vol. 83, no. 6, pp. 1420–1443, May 1978. B. Ryan and N. C. Gross, “The diffusion of hybrid seed corn in two Iowa communities,” Rural sociology, vol. 8, no. 1, pp. 15–24, 1943. + S. Asch, “Opinions and social pressure,” Scientific American, 1955. R. S. Burt, Structural Holes: The Social Structure of Competition. Harvard University Press, 1992. F. Galton, “Vox Populi,” Nature, vol. 75, no. 1949, pp. 450–451, Mar. 1907.

23. Homophily • People “love those who are like themselves”, “Similarity begets friendship” • Nichomachean Ethics, Aristotle & Phaedrus, Plato • Do you think homophilyproduces or hindersmall world?

24. Augmenting lattice with a bias • What if the augmentation exhibits a bias • Most of the people you know are near, • Occasionally, you know someone outside • Does this break the lower bound proof? Does finding a neighborhood of t becomes easier?

25. How to model augmentation bias • The small-world phenomenon: An algorithmic perspective. • J. Kleinberg, Proc. of ACM STOC (2000) • Formal construction: • Connect nodes at distance p in a regular lattice • Connect each node to q other nodes, chosen with a biased probability • p=q=1 to simplify

26. How to model augmentation bias • The small-world phenomenon: An algorithmic perspective. • J. Kleinberg, Proc. of ACM STOC (2000) • Formal construction: • Connect nodes at distance p in a regular lattice • Connect each node to q other nodes, chosen with a biased probability • r may be called the clustering coefficient • If a node is twice further, probability is times less

27. Impact of clustering coefficient Small values of r Approaches uniform augmentation Large values of r Approaches original lattice

28. Can we break the lower bound? (a) Yes, finding a neighborhood of t becomes easier A PRIORI NOT TRUE • It is easier only if you are already near the target • In general, it can take a larger number of steps

29. Can we break the lower bound? (b) Yes, for another reason • All positions are not equal, hence progress is possible • As shortcut are used recursively, probability increases • So we need to study the sequence of progress

30. The critical case • The small-world phenomenon: An algorithmic perspective. • J. Kleinberg, Proc. of ACM STOC (2000) • Assume r=k (dimension of the grid) • A neighborhood of t of radius d/2 • Contains (d/2)k nodes • Each may be chosen with probability roughly 1/(3d/2)k • Growth of ball compensatesprobability decreases! • Harmonic distribution.

31. Augmented lattice Navigable small world dist. alg need O(log2(N)) steps Combinatorial Small world (Short paths exist) dist. alg. need N(k-r)/(k+1) steps Not a small world (Short paths do not exist) alg. need N(r-k)/(r-(k-1)) steps r 0 r=k • The small-world phenomenon: An algorithmic perspective. • J. Kleinberg, Proc. of ACM STOC (2000)

32. Theoretical follow ups • Is the analysis of greedy routing tight? • Yes, greedy routing performs in Ω(log2 n) • Can we find path as short as log(n) (shortest path)? • Yes, with extra information on neighboring nodes • Or another augmentation • Can we build augmentation for an infinite lattice? • See homework exercice (check tomorrow night)

33. Theoretical follow ups (cont’d) • Can we augment other graphs? • G=(V,E) (i.e. a lattice) with distance known • Random augmentation adds one shortcut per node Is routing on G + shortcuts used incidentally efficient? • Indeed all these graphs are polylog augmentable: • Bounded ball growth, Doubling dimensions • Bounded “width” (Trees, bounded treewidth graphs) • What about all graphs? Lower Bound O(n1/√ln(n))

34. Practical follow up Can we observe harmonic distribution? Yes, using closeness rank instead of distance Can we prove it emerge? Recent results Through rewiring, mobility • Geographic routing in social networks. • D. Liben-Nowell et. al. PNAS (2005)

35. Summary • Milgram’s experiment prove that social networks are navigable • individuals can take advantage of short paths • with basic information • This is at odds with uniform random graphs • The key ingredients to explain navigability • A space easy to route (e.g. grid, trees, etc.). • A subtle harmonic augmentation (e.g. ball radius).