1 / 14

Bursty Event Detection from Text Streams for Disaster Management

Bursty Event Detection from Text Streams for Disaster Management. 2012-04-17 Sungjun Lee, Sangjin Lee, Kwanho Kim , and Jonghun Park goalwisk@snu.ac.kr Information Management Lab. Dept. of Industrial Engineering Seoul National University. Introduction.

pier
Download Presentation

Bursty Event Detection from Text Streams for Disaster Management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. BurstyEvent Detection from Text Streams for Disaster Management 2012-04-17 Sungjun Lee, Sangjin Lee, Kwanho Kim, and Jonghun Park goalwisk@snu.ac.kr Information Management Lab. Dept. of Industrial Engineering Seoul National University

  2. Introduction • Identify disaster related bursty events from multiple text streams. • Characterize bursty terms in terms of • Skewness, consistency, periodicity, and variation. Scoring a term to determine whether or not it is bursty term. Normal terms Disaster related terms nice catastrophe happy bad Observations good fine die nightmare … Streams Stream 2 Stream K Stream 1 Real world states normal disaster

  3. Motivation example • The distribution of the frequency of terms observed in AP news stream on Feb. 27, 2010 and Mar. 1, 2010. On Feb. 27, 2010, earthquake hit Chile. On Mar. 1, 2010, the trial about a Bosnian politician,Radovan Karadzic, began.

  4. Skewness feature • A bursty term appears intensively in a specific time period during the corresponding event occurs. where . The change of the term frequency distribution of “tsunami” Probability Probability Term frequency during L days Term frequency during L days

  5. Consistency feature • The frequency of a bursty term soars across multiple streams. The change of the term appearance of “tsunami” Twitterer focusing on tsunami research Stream 1 Article not containing “tsunami” Stream 2 Twitterer focusing on travel … Article containing “tsunami” Stream K

  6. Periodicity feature • Periodic terms are less likely to be bursty terms. • Penalize terms exhibiting the periodicity. Periodicity of “Sunday” Periodicity of “earthquake” period=6.8966 period=3.4843

  7. Variation feature • To cope with different writing styles among streams. • Reduce the possibility of identifying a term with high frequency only in a specific stream as a bursty term. The change of the term appearance of “AP” with a fixed signature “AP news” AP news Start to publish articles Article not containing “AP” Stream 1 Stream 2 … Article containing “AP” Stream K

  8. Putting them all together to measure burstyness • Combine the four scores of different features based on different rationale and scales. • The final term weighting scheme, burst, as follows: where

  9. Experiment setting • 6 news channels are collected • Sources: CNN, AP, Reuters, Times Online, Wall Street Journal, New York Times • Category: World news • Period: 1 Oct. 2009 – 15 Mar. 2010 • Source Type: RSS feed Data channels Google reader API Google ReaderRepository Experiment DB

  10. Example of bursty terms

  11. Experiment results • Comparison of bursty term detection results with methods proposed by Whitney et al. (2009), Fung et al. (2005), Chen et al. (2007), and He et al. (2005). • Bold terms: bursty terms assumed to be correct. • Underlined terms: topical terms. • Starred terms: general terms.

  12. Experiment results • Comparison of the performance of retrieving documents related with bursty events.

  13. Further work Bursty terms MI CV Chi-Square Chernoff Divergence Skewness Self-Similarity KL Divergence Union of “Statistically Sufficient” Conditions

  14. Conclusion • Focus on identifying bursty terms to detect disaster related bursty events. • Bursty terms can help people in properly reacting in decision critical situations. • Bursty terms can be characterized by using four perspectives. • Skewness, consistency, periodicity, and variation. • The final scoring function to detect bursty terms is proposed. • The experiment results showed that the proposed approach is effective to detect bursty terms compared to the existing alternatives.

More Related