1 / 21

Discovering Evolutionary Theme Patterns from Text - An Exploration of Temporal Text Mining

Discovering Evolutionary Theme Patterns from Text - An Exploration of Temporal Text Mining. Qiaozhu Mei, ChengXiang Zhai Department of Computer Science University of Illinois at Urbana-Champaign U.S.A. Motivation. Most text collections bear time stamps

elliot
Download Presentation

Discovering Evolutionary Theme Patterns from Text - An Exploration of Temporal Text Mining

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Discovering Evolutionary Theme Patterns from Text - An Exploration of Temporal Text Mining Qiaozhu Mei, ChengXiang Zhai Department of Computer Science University of Illinois at Urbana-Champaign U.S.A

  2. Motivation • Most text collections bear time stamps • News articles, scientific literature, emails, etc. • Many useful temporal patterns exist • Emerging topics/themes • Decaying topics/themes • Topic evolution thread • Topic/theme life cycles • … • How do we discover and exploit such patterns?

  3. Theme Evolution Graph (Asia Tsunami) Time Theme evolution thread Statistics of Death and loss Statistics of further impact Immediate Reports Personal Experience of Survivors Donations from countries Doc1 Doc3 Doc .. Aid from Local Areas Aid from the world … Specific Events of Aid … Lessons from Tsunami Research inspired Theme spans Evolutionary transitions • Useful for summarizing the news…

  4. Theme Life Cycle (SIGIR Proceedings) Theme Strength Time 1980 1990 1998 2003 TF-IDF Retrieval Language Model Text Categorization IR Applications • Useful for revealing historical trends and hot topics…

  5. Problem Definition • Evolutionary Theme Pattern (ETP) • Theme Evolution Graph • A weighted directed graph in which each vertex is a theme span and each edge is an evolutionary transition • Theme Life Cycle • The strength of a theme over the whole time line • Given a text collection with time stamps, the problem of discovering ETP is to • Extract a theme evolution graph • Model the life cycles of the most salient themes

  6. Research Questions • How to represent a theme? • How to extract themes from a collection automatically? • How to model the transitions of themes? • How to segment the collection with themes? • How to model and compute the strength of each theme at a given time period?

  7. Our Approach Computing Theme Strength Task II. Transition Modeling … … … … Model theme transitions Decoding Collection t Task III. Theme Segmentation θ1 θ2 Theme spans extraction Model theme shifts B θ3 Task I. Theme Extraction Partitioning Extracting global salient themes Theme Life cycles t Theme Evolution Graph s 11 31 21 … 12 t 22 13 3k Collection with time stamps t1 t2 t3, …, t

  8. Our Approach (Cont.) • Extracting Theme Evolution Graph • Partition collection into time intervals • Extract themes from each time span (task I) • Model transitions between theme spans (task II) • Modeling theme life cycles • Extract most salient themes from the whole collection (task I) • Segment the collection with themes (task III ) • Compute the strength of each theme over time

  9. Task I: Theme Extraction Document d 1 ? 2 ? ? k ? ? B ? Parameters: B=noise-level (manually set) ’s and ’s are estimated with Maximum Likelihood • There are k themes in the collection (or a time span), each document is a sample of words generated by multiple themes • Infer the best theme language models that fit our data warning 0.3 system 0.2.. Theme 1 d,1 “Generating” word w in doc d in the collection Aid 0.1donation 0.05support 0.02 .. Theme 2 d,2 1 - B d, k W … statistics 0.2loss 0.1dead 0.05 .. B Theme k Is 0.05the 0.04a 0.03 .. Background B

  10. Task II: Transition Modeling Evolutionary Transition Theme similarity = • Theme spans in an earlier time interval could evolve into theme spans in a later time interval t1 … t2 T microarray 0.2gene 0.1protein 0.05 ? B Information 0.2topic 0.1 classification 0.1text 0.05 A ? web 0.3classification 0.1topic 0.1 C • Similarity/distance between two theme spans is modeled with KL Divergence between two distributions

  11. Task III: Theme Segmentation Decoding Collection θ1 θ2 B θ3 Train transition probabilities output probability = P (w|θ) • View the whole collection as a sequence ordered by time, Model the theme shifts in documents with a Hidden Markov Model w w w w w w w w w w w w w w w w w w w … … Background Theme 1 Theme 2 Theme 3 The Collection

  12. Our Approach: Revisit Computing Theme Strength … … … … Model theme transitions Decoding Collection t θ1 θ2 Theme spans extraction Model theme shifts B θ3 Partitioning Extracting global salient themes Theme Life cycles t Theme Evolution Graph s 11 31 21 … 12 t 22 13 3k Collection with time stamps t1 t2 t3, …, t

  13. Experiments • Two data sets: • Asia Tsunami: 7468 news articles spanning 50 days from 10 news sources • KDD Abstracts: 496 abstracts from 6 years’ KDD conference proceedings • On each data set, we extract a theme evolution graph and model the life cycles of global salient themes

  14. Theme Evolution Graph: Tsunami 01/05/05 01/15/05 … 12/28/04 T system 0.0104 Bush 0.008 warning 0.007conference 0.005US 0.005 … aid 0.020 relief 0.016U.S. 0.013military 0.011U.N. 0.011 … … … Bush 0.016U.S. 0.015$ 0.009 relief 0.008 million 0.008 … Indonesian 0.01 military 0.01islands 0.008 foreign 0.008aid 0.007 … system 0.008 China 0.007 warning 0.005Chinese 0.005 … warning 0.012system 0.012 Islands 0.009 Japan 0.005quake 0.003 … … … … … …

  15. Theme Life Cycles: Tsunami $ 0.0173million 0.0135relief 0.0134aid 0.0099U.N. 0.0066 … Aid from the world Personal experiences Research Aid for children I 0.0322wave 0.0061beach 0.0051saw 0.0046sea 0.0046 … statistics CNN, Absolute Strength

  16. Theme Life Cycles: Tsunami dollars 0.0226million 0.0204aid 0.0118U.N. 0.0102 reconstruction0.0062 … Aid from the world Scene and Experiences Research Aid from China China 0.0391yuan 0.0180 Beijing 0.0089 $ 0.0058donation 0.0052 … statistics XINHUA News, Absolute Strength

  17. Theme Life Cycles: Tsunami $ 0.0173million 0.0135relief 0.0134aid 0.0099U.N. 0.0066 … Aid from the world Scene and Experiences Research Aid from China China 0.0391yuan 0.0180 Beijing 0.0089 $ 0.0058donation 0.0052 … statistics XINHUA News , Normalized Strength

  18. Theme Evolution Graph: KDD 1999 2000 2001 2002 2003 2004 T web 0.009classifica –tion 0.007features0.006topic 0.005… SVM 0.007criteria 0.007classifica – tion 0.006linear 0.005 … mixture 0.005random 0.006cluster 0.006clustering 0.005 variables 0.005… topic 0.010mixture 0.008LDA 0.006 semantic 0.005 … decision 0.006tree 0.006classifier 0.005class 0.005Bayes 0.005 … … Classifica - tion 0.015text 0.013unlabeled 0.012document 0.008labeled 0.008learning 0.007 … Informa - tion 0.012web 0.010social 0.008retrieval 0.007distance 0.005networks 0.004 … … … …

  19. Theme Life Cycles: KDD gene 0.0173expressions 0.0096probability 0.0081microarray 0.0038… marketing 0.0087customer 0.0086 model 0.0079business 0.0048… rules 0.0142association 0.0064 support 0.0053… Global Themes life cycles of KDD Abstracts

  20. Summary and Future Work • We defined a new problem of temporal text mining, which is to discover evolutionary theme patterns • We proposed an algorithm to extract theme evolution graph and model theme life cycles from text collection • Experiments on two data sets show that this algorithm is effective to discover interesting ETPs. • Future Work: • Define a formal evaluation measure and evaluate possible approaches • Further improve the model, integrate the two parts together, and adopt prior knowledge • Extend this model to compare theme evolutions in multiple collections (e.g. KDD proceedings and SIGMOD proceedings)

  21. Thanks!

More Related