A Rank-Rewrite Framework for XML Data Summarization

1. A Rank-Rewrite Framework for XML Data Summarization Maya Ramanath Kondreddi Sarath Kumar Max-Planck Institute for Informatics Saarbruecken, Germany

2. Example Text Document – news report Egyptians Suffer Second Air Tragedy in a Year CAIRO, Egypt -- The crash of a Gulf Air flight that killed 143 people in Bahrain is a disturbing deja vu for Egyptians: It is the second plane crash within a year to devastate this Arab country. Sixty-three Egyptians were on board the Airbus A320, which crashed into shallow Persian Gulf waters Wednesday night after circling and trying to land in Bahrain. On Oct. 31, 1999, a plane carrying 217 mostly Egyptian passengers crashed into the Atlantic Ocean off Massachusetts. The cause has not been determined, providing no closure to the families, whose grief was reopened this month with the release of a factual report by the National Transportation Safety Board. Walid Mourad, head of the Egyptian Pilots Association and a voice often heard in relation to the EgyptAir investigation, said Wednesday's crash is a tragedy for the Arab people as a whole. "We are all family and brothers. We all have something in this," Mourad said. "But for the Egyptians, this is a double blow. Two disasters in a row for the Egyptians." Many of the passengers on the Gulf Air flight were headed for jobs in Bahrain or elsewhere in the Gulf. Rida Hassan was one of those escaping Egypt's moribund economy for work in the oil-rich Gulf. Hassan's uncle said he rushed to the Cairo airport after hearing a list of the passengers read on television. The uncle, who would not give his name, said his nephew had come home to get married and stayed only a month. In the hours just after the crash, relatives at the Cairo airport expressed anger and frustration at Gulf Air for the slow release of information. Women screamed and men tried vainly to calm them. "No information is being given to us. Absolutely nothing," Mohammed Ibrahim el-Naggar said hours after the crash. "We were told that there were some survivors but no names were given." El-Naggar said his cousin, her husband who works in Dubai, and their two children aged 2 and 3 were on the downed plane. Gulf Air said it was sending a special plane to carry 134 relatives to Manama airport later Thursday. "All necessary measures have been taken to receive the families of the victims," Mohammed al-Sayed Abbas, the Egyptian ambassador to Bahrain, told Egyptian television. "The embassy staff will be with them step by step until they identify the deceased." In Bahrain, relatives were beginning the wrenching process of identifying the victims from photographs taken after the bodies were retrieved from the Gulf. Egypt, which lacks the oil wealth of the Gulf and has an economy struggling to revive from decades of socialist stagnation, has a long tradition of sending workers to the Gulf to fill everything from skilled to menial jobs. Remittances from citizens working abroad make up Egypt's biggest source of foreign exchange.

3. “Important” Sentences Egyptians Suffer Second Air Tragedy in a Year CAIRO, Egypt -- The crash of a Gulf Air flight that killed 143 people in Bahrain is a disturbing deja vu for Egyptians: It is the second plane crash within a year to devastate this Arab country. Sixty-three Egyptians were on board the Airbus A320, which crashed into shallow Persian Gulf waters Wednesday night after circling and trying to land in Bahrain. On Oct. 31, 1999, a plane carrying 217 mostly Egyptian passengers crashed into the Atlantic Ocean off Massachusetts. The cause has not been determined, providing no closure to the families, whose grief was reopened this month with the release of a factual report by the National Transportation Safety Board. Walid Mourad, head of the Egyptian Pilots Association and a voice often heard in relation to the EgyptAir investigation, said Wednesday's crash is a tragedy for the Arab people as a whole. "We are all family and brothers. We all have something in this," Mourad said. "But for the Egyptians, this is a double blow. Two disasters in a row for the Egyptians." Many of the passengers on the Gulf Air flight were headed for jobs in Bahrain or elsewhere in the Gulf. Rida Hassan was one of those escaping Egypt's moribund economy for work in the oil-rich Gulf. Hassan's uncle said he rushed to the Cairo airport after hearing a list of the passengers read on television. The uncle, who would not give his name, said his nephew had come home to get married and stayed only a month. In the hours just after the crash, relatives at the Cairo airport expressed anger and frustration at Gulf Air for the slow release of information. Women screamed and men tried vainly to calm them. "No information is being given to us. Absolutely nothing," Mohammed Ibrahim el-Naggar said hours after the crash. "We were told that there were some survivors but no names were given." El-Naggar said his cousin, her husband who works in Dubai, and their two children aged 2 and 3 were on the downed plane. Gulf Air said it was sending a special plane to carry 134 relatives to Manama airport later Thursday. "All necessary measures have been taken to receive the families of the victims," Mohammed al-Sayed Abbas, the Egyptian ambassador to Bahrain, told Egyptian television. "The embassy staff will be with them step by step until they identify the deceased." In Bahrain, relatives were beginning the wrenching process of identifying the victims from photographs taken after the bodies were retrieved from the Gulf. Egypt, which lacks the oil wealth of the Gulf and has an economy struggling to revive from decades of socialist stagnation, has a long tradition of sending workers to the Gulf to fill everything from skilled to menial jobs. Remittances from citizens working abroad make up Egypt's biggest source of foreign exchange.

4. Possible Summary [Generated byMEAD, Radev et al.] • Extract • Abstract • Indicative • Informative • Critical • Query-biased • Query-independent • Single document • Multiple documents CAIRO, Egypt -- The crash of a Gulf Air flight that killed 143 people in Bahrain is a disturbing deja vu for Egyptians: It is the second plane crash within a year to devastate this Arab country. Sixty-three Egyptians were on board the Airbus A320, which crashed into shallow Persian Gulf waters Wednesday night after circling and trying to land in Bahrain. On Oct. 31, 1999, a plane carrying 217 mostly Egyptian passengers crashed into the Atlantic Ocean off Massachusetts. Many of the passengers on the Gulf Air flight were headed for jobs in Bahrain or elsewhere in the Gulf. In the hours just after the crash, relatives at the Cairo airport expressed anger and frustration at Gulf Air for the slow release of information.

5. General Technique • Identify text spans • Rank the text spans • Include them in the summary • Rewrite if required

6. Desired Features • Importance • Non-redundant • Coherent • Coverage

7. Outline • Introduction • XML data summarization • Ranking text and tags • Generating Summary • Evaluation • Discussion

8. Challenges • Features involve structure and value • They have different characteristics • Data-oriented vs. Document-oriented • Text summarization techniques can be re-used, but are not enough • Lots of redundancy • Structural redundancy is inherent • Rewriting • Involves both structure and value

9. Example Document • 2001: A Space Odyssey title, production_year, director, name, job, colourinfo, production_country, production_language, genre, plot, soundmix, alternative_title, alternate_version, goof, keyword

10. title 2001: A Space Odyssey prod_year 1968 name Kubrick, Stanley producer job producer director Kubrick, Stanley actor Dullea, Keir cast casting role Dave Bowman actor Lockwood, Gary movie cast casting role Dr. Frank Poole actor Sylvester, William cast casting role Dr. Haywood R. Floyd prod_location Shepperton Studios, Shepperton, Surrey, England, UK prod_location Borehamwood, Hertfordshire, England, UK prod_location Isle of Harris, Scotland, UK prod_location Monument Valley, Utah, USA This movie is concerned with intelligence as the division between animal and human, then asks a question; what is the next division? Technology is plot

11. title 2001: A Space Odyssey prod_year 1968 name Kubrick, Stanley producer job producer director Kubrick, Stanley actor Dullea, Keir cast casting role Dave Bowman actor Lockwood, Gary movie cast casting role Dr. Frank Poole actor Sylvester, William cast casting role Dr. Haywood R. Floyd prod_location Shepperton Studios, Shepperton, Surrey, England, UK prod_location Borehamwood, Hertfordshire, England, UK prod_location Isle of Harris, Scotland, UK prod_location Monument Valley, Utah, USA This movie is concerned with intelligence as the division between animal and human, then asks a question; what is the next division? Technology is plot

12. movie title movie prod_year movie producer name movie producer job movie director movie cast casting actor movie cast casting role movie prod_location movie plot

13. 2001: A Space Odyssey 1968 Kubrick, Stanley producer Kubrick, Stanley Dullea, Keir Dave Bowman Lockwood, Gary Dr. Frank Poole Sylvester, William Dr. Haywood R. Floyd Shepperton Studios, Shepperton, Surrey, England, UK Borehamwood, Hertfordshire, England, UK Isle of Harris, Scotland, UK Monument Valley, Utah, USA This movie is concerned with intelligence as the division between animal and human, then asks a question; what is the next division? Technology is

14. movie title 2001: A Space Odyssey movie prod_year 1968 movie producer name Kubrick, Stanley movie producer job producer movie director Kubrick, Stanley Dullea, Keir Lockwood, Gary Sylvester, William … movie cast casting actor David Bowman Dr. Frank Poole Dr. Haywood Floyd … movie cast casting role Shepperton Studios, Shepperton, Surrey, England, UK Borehamwood, Hertfordshire, England, UK Isle of Harris, Scotland, UK Monument Valley, Utah, USA movie prod_location This movie is concerned with intelligence as the division between animal and human, then asks a question; what is the next division? Technology is … movie plot

15. Ranking – what is it? • Ranking based on importance • Importance based on frequency of occurrence, representativeness, popularity

16. Types of Text • Named Entities actor, role, director, name Dullea, Keir Lockwood, Gary Sylvester, William … David Bowman Dr. Frank Poole Dr. Haywood Floyd … • Others (text which can be converted to terms) plot, production_location, trivia_item Shepperton Studios, Shepperton, Surrey, England, UK Borehamwood, Hertfordshire, England, UK Isle of Harris, Scotland, UK Monument Valley, Utah, USA

17. Ranking Context Tag context Document context Corpus context

18. Named Entities actor List of 50 actors (and corresponding roles) Goal: Pick top 2 Method: choose context, compute popularity Context: document (augmented with corpus, if required) Result: actors: Dullea, Keir (1) Lockwood, Gary (2) <trivia> According to Keir Dullea (Dave Bowman), Davenport, Nigel and Balsam, Martin were hired and later replaced before Rain, Douglas finally landed the role of HAL. Nigel Davenport was actually was on-set in England during filming, reading HALs lines off-camera so that Dullea and Gary Lockwood could react to them.

19. Others (text converted to terms) production_location Shepperton Studios, Shepperton, Surrey, England, UK Borehamwood, Hertfordshire, England, UK Isle of Harris, Scotland, UK MGM British Studios, Borehamwood, Hertfordshire, England, UK Monument Valley, Utah, USA uk: 4 england: 3 hertfordshir: 2 borehamwood: 2 studio: 2 Goal: Pick 2 samples production_location MGM British Studios, Borehamwood, Hertfordshire, England, UK Borehamwood, Hertfordshire, England, UK

20. Important Tags movie title movie production_year movie producer name cast casting actor movie movie director • Tags which are salient in the corpus

21. Outline • Introduction • XML data summarization • Ranking text and tags • Generating Summary • Evaluation • Discussion

22. Summary Generation Given: A ranked list of tags and text, size constraint Required: A “good” summary

23. Summary Generation • Generate “first-cut” summary • Maintain tag proportions as in the original document • Fill in values according to rankings • Iterative refinement • Remove least important text unit of least important tag • Repeat until required size is reached

24. Step 1: Initial Summary title 2001: A Space Odyssey prod_year 1968 name Kubrick, Stanley producer job producer director Kubrick, Stanley actor Dullea, Keir cast casting role Dave Bowman actor movie Lockwood, Gary cast casting role Dr. Frank Poole Shepperton Studios, Shepperton, Surrey, England, UK prod_location

25. Step 2: Iterative Refinement role Dave Bowman role Dr. Frank Poole Shepperton Studios, Shepperton, Surrey, England, UK prod_location title 2001: A Space Odyssey prod_year 1968 name Kubrick, Stanley producer job producer director Kubrick, Stanley actor Dullea, Keir cast casting actor movie Lockwood, Gary cast casting

26. Outline • Introduction • XML data summarization • Ranking text and tags • Generating Summary • Evaluation • Conclusion

27. Data Sets • IMDB documents • Titanic (200KB) • Fear and Loathing in Las Vegas (77KB) • Fight Club (85KB) • 2001: A Space Odyssey (129KB) • DBLP documents

28. Setup and Metrics • 512B and 1KB summaries • 3 evaluators per document • Intrinsic Evaluation • Grade summary quality from 1 (best) to 5 (worst) • Top-5 evaluation • Top-5 tags • Top-5 values (IMDB – actor)

29. Results for IMDB

30. Conclusions and Ongoing Work • Ranking Text • Diversity in choosing sample values • Choosing better named entity samples • Ranking Tag • Special tags (oscar_winner) • Summary Generation • Current method is not best suited for high coverage • Evaluation

31. THANKS!