1 / 61

AUTOMATIC TEXT SUMMARIZATION

AUTOMATIC TEXT SUMMARIZATION. By Chetana Gavankar Subhabrata Mukherjee Kedharnath Narahari Sarbartha Sengupta. under guidance of: Prof Pushpak Bhattacharya. Motivation Types of summaries Challenges Single-Document Summarization Early work Machine Learning Methods

eagan
Download Presentation

AUTOMATIC TEXT SUMMARIZATION

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. AUTOMATIC TEXT SUMMARIZATION By Chetana Gavankar SubhabrataMukherjee KedharnathNarahari SarbarthaSengupta under guidance of: Prof Pushpak Bhattacharya

  2. Motivation • Types of summaries • Challenges • Single-Document Summarization • Early work • Machine Learning Methods • Supervised Methods • Unsupervised Method • Deep Natural Language Analysis methods • Multi-Document Summarization • Evaluation • Conclusion PRESENTATION CONTENT

  3. Download 1000 + papers and get the summary..   • You have list of emails about sports event  get the summary of those emails in one para… • You have to study loads of books for the exam and the summarizer gives the key concepts of the books as few pages notes… • Value for researchers • Get me everything Papers say about “Automatic Text Summarization” MOTIVATION

  4. MOTIVATION

  5. Automatic Summaries • Should be less than half of original text • Should convey important information • May be produced from single or multiple documents • (Radev et al) DEFINITION • Dipanjan Das, Andre F.T. Martins (2007). A Survey on Automatic Text Summarization. Literature Survey for the Language and Statistics II course at CMU, Pittsburg

  6. APPLICATIONS - NEWS AGGREGATOR http://24eyes.com/

  7. APPLICATIONS – MOVIE REVIEWS

  8. With respect to content: • Indicative: provide an idea what the text is about, but do not render the content • Informative: shortened versions of the text • With respect to the way of creating: • Extracts: identify important sections of the text • Abstracts: produce important material in a new way TYPES OF SUMMARIES • Dipanjan Das, Andre F.T. Martins (2007). A Survey on Automatic Text Summarization. Literature Survey for the Language and Statistics II course at CMU, Pittsburg

  9. With respect to Input • Restricted vs. Unrestricted domain • Single-document vs. Multiple-document • With respect to Purpose • Generic vs. Query based • Background vs. just-the-newsnd vs. just-the-news TYPES OF SUMMARIES Eduard Hovy and Lin C. Y. "Automated Text Summarization in summarist", MIT Press

  10. With respect to Input • Restricted vs. Unrestricted domain • Single-document vs. Multiple-document • With respect to Purpose • Generic vs. Query based • Background vs. just-the-news TYPES OF SUMMARIES Eduard Hovy and Lin C. Y. (1998 )"Automated Text Summarization and the summarist system", TIPSTER III Final Report (SUMMAC)

  11. TOOLS – WORD SUMMARIZER Microsoft Word 2007 - AutoSummarize

  12. TOOLS – GNOME SUMMARIZER

  13. TOOLS – SWESUM SUMMARIZER http://www.csc.kth.se/~xmartin/swesum_lab/index-eng.html

  14. CHALLENGES • Selecting pieces from the input and concatenating them to yield a summary • High reduction rates like headline  • Methods for evaluating summaries • Multiple languages • Multiple Hybrid sources Hahn U. and Mani I. (2000) “The Challenges of Automatic Summarization”, Computer, IEEE Computer Society

  15. ATS has its roots in the last 50’s and has been developed continuously… • A word frequency based ATS [Luhn, 1958]. • An ATS based on multiple features [Edmundson, 1969]. • …….. • ……. • Still unsolved ! EARLY WORK

  16. Content words indicate topic of a text • Frequency of a content word – measure of its significance • Retrieve the top n frequent occurring content words • Rank a sentence according to the frequency of those words present in it WORD FREQUENCY E FREQUENCY WORDS Resolving power of significant words Luhn, H. P. (1958). The automatic creation of literature abstracts. IBM Journal of Research Development, 2(2):159-165

  17. Position of a Sentence Sentences occurring under certain headings are positively relevant Topic sentences tend to occur very early or very late in a document and its paragraphs Optimum Position Policy a ranked list that indicates in what ordinal positions in the text the high-topic-bearing sentences tend to occur. (Lin and Hovy, 97). Ex: [T1, P1S1, P1S2, ...] for a News Article Edmundson, H. P. (1969). New methods in automatic extracting. Journal of the ACM, 16(2): 264-285

  18. Cue words in a Text Probable relevance of a sentence affected by Cue words: Bonus words: positively affecting the relevance of a sentence (e.g. “Significant”, “Greatest”) Stigma words: negatively affecting the relevance of a sentence (e.g. “Impossible”, “Hardly”) Null words: irrelevant Edmundson, H. P. (1969). New methods in automatic extracting. Journal of the ACM, 16(2): 264-285

  19. NAÏVE BAYES • Let s be a particular sentence, S the set of sentences making up the summary and F1, … Fk be the set of features • Assume feature independence • Additional Features • Sentence Length • Presence of Uppercase Words • Position, Cue features with Sentence Length performed best Kupiec, J., Pedersen, J., and Chen, F. (1995). A trainable document summarizer, In Proceedings SIGIR '95, pages 68-73

  20. Naïve Bayes Contd… • Richer features • Tf-idf to derive signature words • Named Entity Tagger to retrieve tokens • Shallow Discourse Analysis to maintain cohesion • Synonym and Morphological variants of lexical terms merged using WordNet. Aone, C., Okurowski, M. E., Gorlinsky, J., and Larsen, B. (1999), A trainable summarizer with knowledge acquired from robust nlp techniques, In Mani, I. and Maybury, M. T., editors, Advances in Automatic Text Summarization, pages 71-80

  21. Decision Tree • Feature independence assumption not valid in real world situation • Creation of feature vector • Baseline • Title • tf & tf-idf scores • Position score • Query Signature • IR Signature • Sentence Length • Average Lexical Connectivity • Numerical Data • Proper Name • Pronoun & Adjective • Weekday & Month • Quotation • First Sentence Lin, C.-Y. (1999). Training a selection function for extraction, In Proceedings of CIKM '99, pages 55-62

  22. Decision Tree Contd… • Scores of all the features combined by automated learning process using decision tree and normalized • Remarks • Decision Tree performs best over all dataset • Naïve combination beats Decision Tree in 3 topics • Possible Reason ???? • Features were Independent

  23. Hidden Markov Model • Drawbacks of earlier approches • Feature based bag-of-words model • Non-sequential • Use sequential model to account for local dependencies between sentences • Features • Position of sentence in the document • Number of terms in the sentence • Likeliness of the sentence terms given the document terms Conroy, J. M. and O'leary, D. P. (2001). Text summarization via hidden markov models, In Proceedings of SIGIR '01, pages 406-407

  24. Hidden Markov Model Contd… • 2s+1 states alternating between s summary states & s+1 non-summary states • Odd state summary state, Even state non-summary state • Transition Matrix M whose element (i,j) is the probability of transition from state i to j • Output function bi(o)= Pr(O|state i) where O is an observed vector of features • Assumption : features are multivariate normal • M & O learnt from training data

  25. Log-Linear Models • Let c be a label, d the item we are interested in labeling, fi the ith feature and λi the corresponding feature weight • Z(d) = Ʃcexp(Ʃi λifi(c,d)) is the normalization constant • fw,c’ (d,c)= 0 c≠c’ 1 c=c’ • Larger value of λi means fi is a strong indicator of class c • GIS, IIS used to iteratively tune model parameters. • Outperformed Naïve Bayes • DRAWBACK : Overfitting Osborne, M. (2002). Using maximum entropy for sentence extraction. In Proceedings of the ACL'02 Workshop on Automatic Summarization

  26. Drawbacks of Earlier Methods 1.Performance might degrade, if the text consists of multiple topics. 2.Anaphors in the extracted sentences might not have any antecedent in the summary. 3.The summary might be incoherent, since the sentences are just extracted from various parts of the text.

  27. Drawbacks Contd … • Consider the following two sequences: • 1. “Dr.Kenny has invented an anesthetic machine. This device controls the rate at which an anesthetic is pumped into the blood.” • 2. “Dr.Kenny has invented an anesthetic machine. The doctor spent two years on this research.” • “Dr.Kenny” appears once in both sequences and so does “machine”. But sequence 1 is about the machine, and sequence 2 is about the “doctor”.

  28. Cohesion • Cohesion ( Halliday & Hasan 1976) • “stitching together” different parts of the text • Use of semantically related terms, co-reference, ellipsis, conjunctions • Lexical Cohesion • Semantically related words • Reiteration category • Repetations, synonyms, hyponyms • Collocation category • Words occurring in same lexical context • Ex: She works as a teacher in the school.

  29. Lexical Chain General Approach • 1. Select a set of candidate words; • 2. For each candidate word, find an appropriate chain relying on a relatedness criterion among members of the chains; • 3. If it is found, insert the word in the chain and update it accordingly. • Relations • Extra-strong (between word & its repetation) • No restriction • Strong (between 2 words connected by WordNet reln) • Window of 7 sentences • Medium-strong (path length > 1 hop ) • Window of 3 sentences • Preference : Extra-strong > Strong > Medium-strong

  30. Drawback • Mr. Kenny is the person that invented an anesthetic machine which uses micro-computers to control the rate at which an anesthetic is pumped into the blood. Such machines are nothing new. But his device uses two micro-computers to achieve much closer monitoring of the pump feeding the anesthetic into the patient. (Morris & Hirst 1991) • [lex "Mr.", sense {mister, Mr.}] • [lex "person", sense {person, individual,someone, man, mortal, human, soul}]. • First sense of machine in WordNet – “an efficient person” – a holonym of “person” and thus wrongly disambiguated

  31. Component and Graph Connectivity Barzilay, R. and Elhadad, M. (1997). Using lexical chains for text summarization. In Proceedings ISTS'97

  32. Component & Graph Connectivity Contd…

  33. Multi Document Summarization • Multiple sources of information • Similarity between topics • Supplement each other • Occasionally contradictory • Key Tasks • Identifying Key concepts across documents • Coping with redundancy • Ensuring final Summary is coherent and complete • Applications: news clustering systems • Google News, Columbia NewsBlaster, News in Essence etc

  34. TOPIC-DRIVEN SUMMARIZATION Carbonell, J. and Goldstein, J. (1998). The use of MMR, diversity-based re-ranking for reordering documents and producing summaries.

  35. TOPIC DRIVEN SUMMARIZATION(contd.) Carbonell, J. and Goldstein, J. (1998). The use of MMR, diversity-based re-ranking for reordering documents and producing summaries. In proceedings of SIGIR '98.

  36. GRAPH SPREADING Mani, I. and Bloedorn, E. (1997). Multi-document summarization by graph search and matching. In AAAI/IAAI, pages 622-628.

  37. Example for nodes and links in the graph(mani and bloerdon 97’) Mani, I. and Bloedorn, E. (1997). Multi-document summarization by graph search and matching. In AAAI/IAAI, pages 622-628.

  38. GRAPH SPREADING(Contd..) • Words and phrases are intialized according to their TF-IDF scores. • For each sentence in both documents, two scores are computed. • One score that reflects the presence of common nodes, which is computed as the average weight of these nodes; • Other score that computes instead the average weights of difference nodes. • the sentences that have higher common and different scores are highlighted and accordingly the ouput is generated.

  39. CENTROID-BASED SUMMARY • Does not make use of language generation model. • Documents are modeled as bag-of-words. Topic Detection: • Clustering algorithm that uses TF-IDF vector repesentations of documents • Successively add documents and recomputescentroids. • Centroids are pseudo documents that include words with TF-IDF score above some threshold. d’-truncated document C j – jth cluster Radev, D. R., Jing, H., and Budzikowska, M. (2000). Centroid-based summarization of multiple documents: sentence extraction, utility-based evaluation, and user studies.

  40. CENTROID BASED SUMMARIZATION Sentence Identification: 2metrics • Cluster Based Relative Utility-how relevant a sentence is to particular topic of the cluster. • Cross Sentence Informational Subsumption-measure of redundancy among sentences. Radev, D. R., Jing, H., and Budzikowska, M. (2000). Centroid-based summarization of multiple documents: sentence extraction, utility-based evaluation, and user studies. In NAACL-ANLP 2000 Workshop on Automatic summarization, pages 21-30, Morristown, NJ, USA.

  41. MULTILINGUAL MULTI-DOCUMENT SUMMARY • Target language(English) in which summary is written. • Source documents present in both preferred language and foreign language(Arabic). • Use IBM’s translational model to translate documents in source language to target language. • Check for similarity between translated sentences in two documents. • If similarities found, retain documents in source language, since they can be more grammatically correct. Evans, D. K. (2005). Similarity-based multilingual multi-document summarization. Technical Report CUCS-014-05, Columbia University.

  42. SHORT SUMMARIES Witbrock, M. J. and Mittal, V. O. (1999). Ultra-summarization (poster abstract): a statistical approach to generating highly condensed non-extractive summaries.

  43. Witbrock, M. J. and Mittal, V. O. (1999). Ultra-summarization (poster abstract): a statistical approach to generating highly condensed non-extractive summaries. In Proceedings of SIGIR '99, pages 315{316, New York, NY, USA.

  44. SENTENCE COMPRESSION • Compression of sentence used for summarization. • Uses noisy-channel model which considers that one starts with a short summary s, according to source model P(s). • Subjected to noisy-channel to make full sentence t, in a process guided by channel model, P(t/s). • Now observing t, recover the summary as: s’=argmaxsP(s/t)= argmaxsP(s).P(t/s) • Advantage of decoupling the goals of grammatical correctness and preserving important information. Knight, K. and Marcu, D. (2000). Statistics-based summarization - step one: Sentence compression. In AAAI/IAAI, pages 703-710

  45. Knight, K. and Marcu, D. (2000). Statistics-based summarization - step one: Sentence compression. In AAAI/IAAI, pages 703-710.

  46. Evaluation • Difficult task. (There does not exist an ideal summary for a given document or set of document.) • Agreement between human summarizers is quite low. • Difficult to evaluate the summary content. • Absence of a standard human or automatic evaluation metric.

  47. Evaluation • Lin and Hovy (2002). • describe and compare various human and automatic metrics to evaluate summaries. • Focus on the evaluation procedure used in the Document Understanding Conference 2001(DUC-2001). • compared manually written ideal summaries with summaries generated automatically by summarization systems and baseline summaries. Lin, C.-Y. and Hovy, E. (2002). Manual and automatic evaluation of summaries. In Proceedings of the ACL-02 Workshop on Automatic Summarization, pages 45-51

  48. Evaluation • Lin and Hovy (2002). • Each text was decomposed into a list of units (sentences). • stepped through each model unit (MU) from the ideal summaries. • marked all system units (SU) sharing content with the current model unit. • All (4) • Most (3) • Some (2) or • Hardly any (1) • Grammaticality, cohesion, and coherence were also rated

  49. Evaluation • Lin and Hovy (2002). • The weighted recall at threshold ‘t’ (t=1 to 4). • outline an accumulative n-gram matching score (which they call NAMS), Taken from: Dipanjan Das, Andre F.T. Martins, A Survey on Automatic Text Summarization2007

  50. Evaluation • Recall-Oriented Understudy for Gisting Evaluation (ROUGE) (Lin 2004). • Let be a set of reference summary, and let be a summary generated automatically by a system. Let be a binary vector representing n-grams contained in a document d. • The metric ROUGE-N is an n-gram recall based statistic where denotes the usual inner product of vectors Lin, C.-Y. (2004). Rouge: A package for automatic evaluation of summaries. In Marie-Francine Moens, S. S., editor, Text Summarization Branches Out: Proceedings of the ACL-04 Workshop, pages 74-81

More Related