COMS 6998-06 Network Theory Week 7

1. COMS 6998-06 Network TheoryWeek 7 Dragomir R. Radev Wednesdays, 6:10-8 PM 325 Pupin Terrace Fall 2010

2. (17) Lexical networks

3. Free word associations The large-scale structure of semantic networks: statistical analyses and a model of semantic growth M. Steyvers, J. B. Tenenbaum (2005) Cognitive Science, 29(1)

4. Semantic network

5. Dependency network bought Meredith yesterday apples green

6. Dependency network

7. Random network

8. Language networks • A special case of networks where nodes are words or documents and edges link semantically related nodes • Other examples: • Words used in dictionary definitions • Names of people mentioned in the same story • Words that translate to the same word

9. Comparing the dependency graph to a random (Poisson) graph

10. Some real networks • From Newman 2002:

11. universe letter character nature world actor Properties of lexical networks • Entries in a thesaurus[Motter et al. 2002] • c/c0 = 260 (n=30,244; m=60) • Co-occurrence networks [Dorogovtsev and Mendes 2001, Sole and Ferrer i Cancho 2001] • c/c0 = 1,000 (n=400,000) • Mental lexicon [Vitevitch 2005] • c/c0 = 278 (n=19,340)

13. Sigman and Cecchi 2002 • Meronymy • Component (branch-tree) • Member (tree-forest) • Stuff (glass-bottle) • Polysemy • E.g., board • Hypernymy/hyponymy • E.g., oak-tree

14. n=8,992 m=117,687 alpha = 1.1 Masucci and Rodgers 2006 It was a bright cold day in April, and the clocks were striking thirteen. Winston Smith, his chin nuzzled into his breast in an effort to escape the vile wind, slipped quickly through the glass doors of Victory Mansions, though not quickly enough to prevent a swirl of gritty dust from entering along with him. The hallway smelt of boiled cabbage and old rag mats. At one end of it a coloured poster, too large for indoor display, had been tacked to the wall. It depicted simply an enormous face, more than a metre wide: the face of a man of about forty-five, with a heavy black moustache and ruggedly handsome features. Winston made for the stairs. It was no use trying the lift. Even at the best of times it was seldom working, and at present the electric current was cut off during daylight hours. It was part of the economy drive in preparation for Hate Week. The flat was seven flights up, and Winston, who was thirty-nine and had a varicose ulcer above his right ankle, went slowly, resting several times on the way. On each landing, opposite the lift-shaft, the poster with the enormous face gazed from the wall. It was one of those pictures which are so contrived that the eyes follow you about when you move. BIG BROTHER IS WATCHING YOU, the caption beneath it ran.

15. Masucci and Rodgers (cont’d)

16. Solé et al. 2004

17. Ferrer i Cancho et al. 2003

18. Based on [Mehler 2007]

19. Lexical centrality • In social networks, some people are called mavens/centers due to their popularity (or gregariousness) • What is the analog to a highly connected person in a language network? • In general, how can network structure be used to identify content in a language network?

21. How many ways to say it? Red Sox Win the World Series, More Titles Might Follow Back in the Red: World Series crown returns to Boston as Rockies hit the canvas Fans celebrate Red Sox win Rockies Vanish In Thin Air Police Arrest Dozens After Red Sox World Series Win Red Sox 4, Rockies 3 Boston Sweeps World Series Again Victory walk leads to dynasty talk Red Sox Win Baseball's World Series Title by Sweeping Rockies Boston enjoys sweep smell of success Sox sweep Rockies to win Series World Series: Red Sox sweep Rockies Red Sox cruise to World Series title Red Sox Are World Champs Boston sweep Colorado to win World Series Red Sox Sweep Colorado in World Series Red Sox Take World Series How sweep it is! Red Sox breeze to second World Series title Boston owners dedicate triumph to Red Sox Nation

22. Boston lowers the broom We Are the Champions: Red Sox 4, Rockies 3 Sweep and red sox for everybody Two titles four years apart impossible to compare Red Sox cash in The Boston Red Sox swept the Colorado Rockies to win the World Series Red Sox "do little things" to win second title in four years World Series victory for Red Sox Boston reigns supreme Rockies' heads held high despite loss Rockies just failed to execute Boston Fans Fill Streets To Celebrate Series Sweep It's easy to embrace these Red Sox Young stars lead Red Sox to World Series title They spend money in Boston, but they win Boston fans celebrate World Series win; 37 arrests reported Soxcess started upstairs Boston sweep is complete Red Sox sweep 2007 World Series in Denver Rookies rise to occasion! Sox sweep World Series again Poor pitching, poorer hitting doom Rockies Red Sox top Rockies and sweep to second World Series championship ... Sweeping off to Boston Another in a Series of sweeps Putting an end to baseball as we've known it Red Sox Sweep Rockies to Capture Second Title in Four Years Red Sox accomplished the expected, unbelievable Rockies Find Being Good Isnt Enough Boston sweep-walks to title Red Sox play party crashers in Denver Unhappy ending for Colorado - MLB Sox on, Rocks off in sweeping win Timlin gets to ring up another one Red Sox complete World Series sweep Wild celebrations in Boston after World Series win Red Sox seal sweep of Rockies Red Sox Win World Series Sox are kings of diamond Rockies: Sweep, sweep, swept Red Sox sweep World Series Monsters of Beantown: Red Sox win Series Red Sox claim World Series glory How sweep it is Red Sox: Dynasty in the making Red Sox sweep upstart Rockies Boston Sweeps Colorado To Win World Series Red Sox sweep Rockies, take World Series What curse? Red Sox win Series again Red Sox Win the World Series, More Titles Might Follow Back in the Red: World Series crown returns to Boston as Rockies hit the canvas Fans celebrate Red Sox win Rockies Vanish In Thin Air Police Arrest Dozens After Red Sox World Series Win Red Sox 4, Rockies 3 Boston Sweeps World Series Again Victory walk leads to dynasty talk Red Sox Win Baseball's World Series Title by Sweeping Rockies Boston enjoys sweep smell of success Sox sweep Rockies to win Series World Series: Red Sox sweep Rockies Red Sox cruise to World Series title Red Sox Are World Champs Boston sweep Colorado to win World Series Red Sox Sweep Colorado in World Series Red Sox Take World Series How sweep it is! Red Sox breeze to second World Series title Boston owners dedicate triumph to Red Sox Nation Red Sox win World Series Short wait for bosox this time World Series: Red Sox complete sweep of Rockies It's Leap Year Boston Red Sox blank Rockies to clinch World Series Red Sox Sweep Rockies 4-3 In Game 4 Red Sox claim World Series title Boston Red Sox win World Series, lose image Sox sweep Rockies for 2nd title in 4 seasons Red Sox Complete Sweep Of Rockies For World Series Victory Red sox wrap up world series rout Rockies feel the pain, but not the shame Red Sox, tarnished for so long, now baseball's gold standard Red sox take title Boston Celebrates World Series Win Bad news AL, Red Sox built to last Boston sweeps it Heartbreak is history for Red Sox Fans Celebrate Red Sox World Series Win From cursed to charmed: Red Sox sweep World Series Red Sox Sweep Rockies To Win World Series Red Sox make Boston jump for joy, Series Champs Crowds fill streets after Red Sox win Papelbon, Timlin savoring Series win Red Sox scale the Rockies Even Sox fan losing passion for winning Rookies respond in first crack at the big time Red Sox Get 2nd World Series Sweep In 4 Years Red Sox looking like a dynasty Boston Red Sox are America's team Red Sox Win 2007 World Series Believing Pays Off! Sox World Series Champs Red Sox go from cursed to blessed

23. List of topics • Red Sox • Sweep • World Series • Rockies • Celebrate • Boston • Colorado • Dynasty • 2007 • Four years • Second time • 4:3 score • Easy • Expectations • No curse • Young players • Timlin • not (baseball)

24. LexRank – Centrality in Text Graphs Vertices Units of text (sentences or documents) Edges Pairwise similarity between text (tf-idf cosine or language model)

25. LexRank – Centrality in Text Graphs Intuition LexRank score is propagated through edges Central vertices are those that are similar to other central vertices

26. LexRank – Centrality in Text Graphs Recurrence Relation 0.3 0.1 0.9 0.3 s 0.5 0.8 Can guarantee solution by allowing “jump” probability d/N. 0.2 0.4 0.2

29. 0.01718 Red Sox Win Baseball's World Series Title by Sweeping Rockies 0.01712 Red Sox Sweep Rockies To Win World Series 0.01647 World Series: Red Sox sweep Rockies 0.01630 Red Sox sweep Rockies, take World Series 0.01608 Red Sox 4, Rockies 3 Boston Sweeps World Series Again 0.01597 World Series: Red Sox complete sweep of Rockies 0.01584 Red Sox sweep World Series 0.01579 Red Sox Sweep Colorado in World Series 0.01573 Red Sox Complete Sweep Of Rockies For World Series Victory 0.01531 Red Sox complete World Series sweep ... 0.01057 Boston Red Sox blank Rockies to clinch World Series 0.01052 Red Sox: Dynasty in the making 0.01037 Sox sweep Rockies for 2nd title in 4 seasons 0.01034 Police Arrest Dozens After Red Sox World Series Win 0.01027 Rookies respond in first crack at the big time 0.01018 Rockies: Sweep, sweep, swept 0.01016 Sweeping off to Boston 0.01013 Rookies rise to occasion! 0.01012 Fans celebrate Red Sox win 0.01010 Short wait for bosox this time ... 0.00441 Sox are kings of diamond 0.00414 Rockies just failed to execute 0.00408 Rockies Find Being Good Isnt Enough 0.00407 Rockies' heads held high despite loss 0.00391 Boston lowers the broom 0.00390 Rockies Vanish In Thin Air 0.00390 Poor pitching, poorer hitting doom Rockies 0.00390 Rockies feel the pain, but not the shame 0.00375 Two titles four years apart impossible to compare 0.00362 Boston reigns supreme

30. 0.01718 Red SoxWin Baseball's World Series Title by Sweeping Rockies 0.01712 Red Sox Sweep Rockies To Win World Series 0.01647 World Series: Red Sox sweep Rockies 0.01630 Red Sox sweep Rockies, take World Series 0.01608 Red Sox 4, Rockies 3 Boston Sweeps World Series Again 0.01597 World Series: Red Sox complete sweep of Rockies 0.01584 Red Sox sweepWorld Series 0.01579 Red Sox Sweep Colorado in World Series 0.01573 Red Sox Complete Sweep Of Rockies For World Series Victory 0.01531 Red Sox complete World Series sweep ... 0.01057 Boston Red Soxblank Rockies to clinch World Series 0.01052 Red Sox: Dynasty in the making 0.01037 Sox sweep Rockies for 2nd title in 4 seasons 0.01034 Police Arrest Dozens After Red Sox World Series Win 0.01027 Rookies respond in first crack at the big time 0.01018 Rockies: Sweep, sweep, swept 0.01016 Sweeping off to Boston 0.01013 Rookies rise to occasion! 0.01012 Fans celebrate Red Sox win 0.01010 Short wait for bosox this time ... 0.00441 Sox are kings of diamond 0.00414 Rockies just failed to execute 0.00408 Rockies Find Being Good Isnt Enough 0.00407 Rockies' heads held high despite loss 0.00391 Boston lowers the broom 0.00390 Rockies Vanish In Thin Air 0.00390 Poor pitching, poorer hitting doom Rockies 0.00390 Rockies feel the pain, but not the shame 0.00375 Two titles four years apart impossible to compare 0.00362 Boston reigns supreme

31. Stationary solutions • The fundamental Ergodic Theorem for Markov chains [Grimmett and Stirzaker 1989] says that the Markov chain with kernel E has a stationary distribution p under three conditions: • E is stochastic • E is irreducible • E is aperiodic • To make these conditions true: • All rows of E add up to 1 (and no value is negative) • Make sure that E is strongly connected • Make sure that E is not bipartite • Example: PageRank [Brin and Page 1998]: use “teleportation”

32. LexRank (Cosine centrality)

33. d3s3 d2s3 d3s2 d3s1 d1s1 d4s1 d5s1 d2s1 d5s2 d5s3 d2s2 Lexical centrality (t=0.3)

34. d3s3 d2s3 d3s2 d3s1 d1s1 d4s1 d5s1 d2s1 d5s2 d5s3 d2s2 Lexical centrality (t=0.2)

35. d3s2 d3s3 d2s3 d3s2 d4s1 d3s1 d2s1 d1s1 d4s1 d5s1 d2s1 d5s2 d5s3 d2s2 Lexical centrality (t=0.1) Sentences vote for the most central sentence! Need to worry about diversity reranking.

36. DUC 2004 results Content-based evaluation (ROUGE)

38. Latent networks • When there is no explicit network, we can create one. • Latent networks – generating functions for networks. • Complex system with one parameter (akin to temperature). • With a change in temperature, latent links become active (or inactive). • Examples: social network in a small town; network of business connections. • Looking for phase transitions.

39. ap158 degree t cutoff counts