Text Summarization: News and Beyond

1. Text Summarization:News and Beyond Kathleen McKeown Department of Computer Science Columbia University

2. Homework questions and information • Data sets • Additional summary/document pairs added • Not homogeneous

3. Midterm Curve • A: 72-93 • A- 72-73 • A+ 91-93 • B: 58-71 • B- 58-19 • B+ 70-71 • C: 47-57 • C- 47-50 • C+ 56-57 • D: 39-46 • Solutions are posted on the syllabus

4. Questions (from Sparck Jones) • Should we take the reader into account and how? • “Similarly, the notion of a basic summary, i.e., one reflective of the source, makes hidden fact assumptions, for example that the subject knowledge of the output’s readers will be on a par with that of the readers for whom the source was intended. (p. 5)” • Is the state of the art sufficiently mature to allow summarization from intermediate representations and still allow robust processing of domain independent material?

5. Text Summarization at Columbia • Shallow analysis instead of information extraction • Extraction of phrases rather than sentences • Generation from surface representations in place of semantics

6. Problems with Sentence Extraction • Extraneous phrases • “The five were apprehended along Interstate 95, heading south in vehicles containing an array of gear including … ... authorities said.” • Dangling noun phrases and pronouns • “The five” • Misleading • Why would the media use this specific word (fundamentalists), so often with relation to Muslims? *Most of them are radical Baptists, Lutheran and Presbyterian groups.

7. Cut and Pastein Professional Summarization • Humans also reuse the input text to produce summaries • But they “cut and paste” the input rather than simply extract • our automatic corpus analysis • 300 summaries, 1,642 sentences • 81% sentences were constructed by cutting and pasting • linguistic studies

8. Major Cut and Paste Operations • (1) Sentence reduction ~~~~~~~~~~~~

9. Major Cut and Paste Operations • (1) Sentence reduction ~~~~~~~~~~~~

10. Major Cut and Paste Operations • (1) Sentence reduction • (2) Sentence Combination ~~~~~~~~~~~~ ~~~~~~~ ~~~~~~ ~~~~~~~

11. Major Cut and Paste Operations • (3) Syntactic Transformation • (4) Lexical paraphrasing ~~~~~ ~~~~~ ~~~~~~~~~~~ ~~~

12. Summarization at Columbia • News • Email • Meetings • Journal articles • Open-ended question-answering • What is a Loya Jurga? • Who is Mohammed Naeem Noor Khan? • What do people think of welfare reform?

13. Summarization at Columbia • News • Single Document • Multi-document • Email • Meetings • Journal articles • Open-ended question-answering • What is a Loya Jurga? • Who is Al Sadr? • What do people think of welfare reform?

14. Cut and Paste Based Single Document Summarization -- System Architecture Input: single document Extraction Extracted sentences Corpus Generation Parser Sentence reduction Decomposition Co-reference Sentence combination Lexicon Output: summary

15. (1) Decomposition of Human-written Summary Sentences • Input: • a human-written summary sentence • the original document • Decomposition analyzes how the summary sentence was constructed • The need for decomposition • provide training and testing data for studying cut and paste operations

16. Sample Decomposition Output Document sentences: S1: A proposed new law that would require web publishers to obtain parental consent before collecting personal information from children could destroy the spontaneous nature that makes the internet unique, a member of the Direct Marketing Association told a Senate panel Thursday. S2: Arthur B. Sackler, vice president for law and public policy of Time Warner Cable Inc., said the association supported efforts to protect children on-line, but he… S3: “For example, a child’s e-mail address is necessary …,” Sackler said in testimony to the Communications subcommittee of the Senate Commerce Committee. S5: The subcommittee is considering the Children’s Online Privacy Act, which was drafted… Summary sentence: Arthur B. Sackler, vice president for law and public policy of Time Warner Cable Inc. and a member of the direct marketing association told the Communications Subcommittee of the Senate Commerce Committee that legislation to protect children’s privacy on-line could destroy the spondtaneous nature that makes the Internet unique.

17. A Sample Decomposition Output Decomposition of human-written summaries Document sentences: S1: A proposed new law that would require web publishers to obtain parental consent before collecting personal information from children could destroy the spontaneous nature that makes the internet unique, a member of the Direct Marketing Association told a Senate panel Thursday. S2:Arthur B. Sackler, vice president for law and public policy of Time Warner Cable Inc., said the association supported efforts to protect children on-line, but he… S3: “For example, a child’s e-mail address is necessary …,” Sackler said in testimony to the Communications subcommittee of the Senate Commerce Committee. S5: The subcommittee is considering the Children’s Online Privacy Act, which was drafted… Summary sentence: Arthur B. Sackler, vice president for law and public policy of Time Warner Cable Inc. and a member of the direct marketing association told the Communications Subcommittee of the Senate Commerce Committee that legislation to protect children’s privacy on-line could destroy the spondtaneous nature that makes the Internet unique.

18. A Sample Decomposition Output Decomposition of human-written summaries Document sentences: S1: A proposed new law that would require web publishers to obtain parental consent before collecting personal information from children could destroy the spontaneous nature that makes the internet unique, a member of the Direct Marketing Association told a Senate panel Thursday. S2:Arthur B. Sackler, vice president for law and public policy of Time Warner Cable Inc., said the association supported efforts to protect children on-line, but he… S3: “For example, a child’s e-mail address is necessary …,” Sackler said in testimony to the Communications subcommittee of the Senate Commerce Committee. S5: The subcommittee is considering the Children’s Online Privacy Act, which was drafted… Summary sentence: Arthur B. Sackler, vice president for law and public policy of Time Warner Cable Inc. and a member of the direct marketing association told the Communications Subcommittee of the Senate Commerce Committee that legislation to protect children’s privacy on-line could destroy the spondtaneous nature that makes the Internet unique.

19. A Sample Decomposition Output Decomposition of human-written summaries Document sentences: S1: A proposed new law that would require web publishers to obtain parental consent before collecting personal information from children could destroy the spontaneous nature that makes the internet unique, a member of the Direct Marketing Association told a Senate panel Thursday. S2:Arthur B. Sackler, vice president for law and public policy of Time Warner Cable Inc., said the association supported efforts to protect children on-line, but he… S3: “For example, a child’s e-mail address is necessary …,” Sackler said in testimony to the Communications subcommittee of the Senate Commerce Committee. S5: The subcommittee is considering the Children’s Online Privacy Act, which was drafted… Summary sentence: Arthur B. Sackler, vice president for law and public policy of Time Warner Cable Inc. and a member of the direct marketing association toldthe Communications Subcommittee of the Senate Commerce Committeethat legislation to protect children’s privacy on-linecould destroy the spondtaneous nature that makes the Internet unique.

20. The Algorithm for Decomposition • A Hidden Markov Model based solution • Evaluations: • Human judgements • 50 summaries, 305 sentences • 93.8% of the sentences were decomposed correctly • Summary sentence alignment • Tested in a legal domain • Details in (Jing&McKeown-SIGIR99)

21. (2) Sentence Reduction • An example: Original Sentence:When it arrives sometime next year in new TV sets, the V-chip will give parents a new and potentially revolutionary device to block out programs they don’t want their children to see. Reduction Program: The V-chip will give parents a new and potentially revolutionary device to block out programs they don’t want their children to see. Professional: The V-chip will give parents a device to block out programs they don’t want their children to see.

22. The Algorithm for Sentence Reduction • Preprocess: syntactic parsing • Step 1: Use linguistic knowledge to decide what phrases MUST NOT be removed • Step 2: Determine what phrases are most important in the local context • Step 3: Compute the probabilities of humans removing a certain type of phrase • Step 4: Make the final decision

23. Step 1: Use linguistic knowledge to decide what MUST NOT be removed • Syntactic knowledge from a large-scale, reusable lexicon we have constructed convince: meaning 1: NP-PP :PVAL (“of”) (E.g., “He convinced me of his innocence”) NP-TO-INF-OC (E.g., “He convinced me to go to the party”) meaning 2: ... • Required syntactic arguments are not removed

24. Saudi Arabia on Tuesday decided to sign… The official Saudi Press Agency reported that King Fahd made the decision during a cabinet meeting in Riyadh, the Saudi capital. The meeting was called in response to … the Saudi foreign minister, that the Kingdom… An account of the Cabinet discussions and decisions at the meeting… The agency... Step 2: Determining context importance based on lexical links

25. Saudi Arabia on Tuesday decided to sign… The official Saudi Press Agency reported that King Fahd made the decision during a cabinet meeting in Riyadh, the Saudi capital. The meeting was called in response to … the Saudi foreign minister, that the Kingdom… An account of the Cabinet discussions and decisions at the meeting… The agency... Step 2: Determining context importance based on lexical links

26. Saudi Arabia on Tuesday decided to sign… The official Saudi Press Agency reported that King Fahd made the decision during a cabinet meeting in Riyadh, the Saudi capital. The meeting was called in response to … the Saudi foreign minister, that the Kingdom… An account of the Cabinet discussions and decisions at the meeting… The agency... Step 2: Determining context importance based on lexical links

27. Step 3: Compute probabilities of humans removing a phrase verb (will give) obj (device) vsubc (when) subj (V-chip) iobj (parents) ndet (a) adjp (and) rconj (revolutionary) lconj (new) Prob(“when_clause is removed”| “v=give”) Prob (“to_infinitive modifier is removed” | “n=device”)

28. Step 4: Make the final decision verb (will give) L Cn Pr obj (device) vsubc (when) L Cn Pr subj (V-chip) L Cn Pr iobj (parents) L Cn Pr L Cn Pr L Cn Pr ndet (a) adjp (and) L Cn Pr L -- linguistic Cn -- context Pr -- probabilities rconj (revolutionary) lconj (new) L Cn Pr L Cn Pr

29. Evaluation of Reduction • Success rate: 81.3% • 500 sentences reduced by humans • Baseline: 43.2% (remove all the clauses, prepositional phrases, to-infinitives,…) • Reduction rate: 32.7% • Professionals: 41.8% • Details in (Jing-ANLP00)

30. Multi-Document Summarization Research Focus • Monitor variety of online information sources • News, multilingual • Email • Gather information on events across source and time • Same day, multiple sources • Across time • Summarize • Highlighting similarities, new information, different perspectives, user specified interests in real-time

31. Our Approach • Use a hybrid of statistical and linguistic knowledge • Statistical analysis of multiple documents • Identify important new, contradictory information • Information fusion and rule-driven content selection • Generation of summary sentences • By re-using phrases • Automatic editing/rewriting summary

32. NewsblasterIntegrated in online environment for daily news updateshttp://newsblaster.cs.columbia.edu/ Ani Nenkova David Elson

33. Newsblaster http://newsblaster.cs.columbia.edu/ • Clustering articles into events • Categorization by broad topic • Multi-document summarization • Generation of summary sentences • Fusion • Editing of references

34. Crawl News Sites Newsblaster Architecture Form Clusters Categorize Title Clusters Summary Router Select Images Event Summary Biography Summary Multi- Event Convert Output to HTML

36. Fusion

37. Theme Computation • Input: A set of related documents • Output: Sets of sentences that “mean” the same thing • Algorithm • Compute similarity across sentences using the Cosine Metric • Can compare word overlap or phrase overlap • (PLACEHOLDER: IR vector space model)

38. Sentence Fusion Computation • Common information identification • Alignment of constituents in parsed theme sentences: only some subtrees match • Bottom-up local multi-sequence alignment • Similarity depends on • Word/paraphrase similarity • Tree structure similarity • Fusion lattice computation • Choose a basis sentence • Add subtrees from fusion not present in basis • Add alternative verbalizations • Remove subtrees from basis not present in fusion • Lattice linearization • Generate all possible sentences from the fusion lattice • Score sentences using statistical language model

41. Tracking Across Days • Users want to follow a story across time and watch it unfold • Network model for connecting clusters across days • Separately cluster events from today’s news • Connect new clusters with yesterday’s news • Allows for forking and merging of stories • Interface for viewing connections • Summaries that update a user on what’s new • Statistical metrics to identify differences between article pairs • Uses learned model of features • Identifies differences at clause and paragraph levels

46. Different Perspectives • Hierarchical clustering • Each event cluster is divided into clusters by country • Different perspectives can be viewed side by side • Experimenting with update summarizer to identify key differences between sets of stories

50. Multilingual Summarization • Given a set of documents on the same event • Some documents are in English • Some documents are translated from other languages