Text Categorization Moshe Koppel Lecture 3:Authorship Attribution

1. Text CategorizationMoshe KoppelLecture 3:Authorship Attribution Mostly my own stuff together with Jonathan Schler, Shlomo Argamon, Ido Dagan, Jamie Pennebaker, Jonathan Fine, Kfir Zigdon, Iris Zigdon, Navot Akiva, Dror Mughaz, Eran Messeri

2. The Authorship Attribution Problem Given the known writings of a small number of authors A1,…,An , determine which of them wrote document X. The vanilla version is actually not very difficult.

3. History of Authorship Attribution • Researchers have been trying to solve this problem since the 19th century • Early research focused on classical works: Bible, Shakespeare, etc. • The most famous work is Mosteller & Wallace (1964) on the Federalist Papers • The proliferation of online text offers many new research opportunities

4. Authorship – Realistic Versions • No suspects – Can we profile the author of X? • One suspect – Can we ascertain guilt or innocence? • Thousands of suspects – Can we pick a needle from the haystack?

5. Text Categorization

6. Features for Authorship Attribution What features do we want? • Used consistently by given author regardless of document type • Eliminates topic words • Might be used differently by different authors • Our challenge: identify these

7. Olden Days • Early researchers sought a single “magic” feature • Mainly, they looked at complexity features: • Sentence or word length • Type/token ratio • Hapax legomena • The idea was the magic feature had a distinct constant value for each author • This approach isn’t taken seriously these days.

8. Features for Categorizing by Style • Function words (and, of, the,..)[Mosteller-Wallace 64]

9. Features for Categorizing by Style • Function words • Syntax – POS n-grams

10. Features for Categorizing by Style • Function words • Syntax – POS n-grams • SFL trees

11. SFL Trees Conjunctions ConjExtension and, or, but, yet, however,… ConjElaboration for_example, indeed,… ConjEnhancement ConjSpatiotemporal then, beforehand, afterwards, while, during,… ConjCausalConditional if, because, since, hence, thus, therefore,… • Similar trees for prepositions, pronouns, modal verbs • Covers all function words, some POS • Doesn’t require POS tagging

12. Features for Categorizing by Style • Function words • Syntax • SFL trees • Morphology (grammatical prefixes and suffixes,e.g.–ing, re-)

13. Features for Categorizing by Style • Function words • Syntax • SFL trees • Morphology • Complexity measures (e.g. avg. word length, avg. sentence length, various entropy measures)

14. Features for Categorizing by Style • Function words • Syntax • SFL trees • Morphology • Complexity measures • “Unstable” words (features that might be replaced in a rewrite, e.g. huge:large)

15. Measuring Instability An interesting idea: Translate many documents to another language and translate back. See what sticks. We did just that on Reuters 21578 and BNC*. * We used five different languages to ensure robustness.

16. Word Stability -- Examples huge .0 ratio .0 device .0 onto .06 has .28 over .55 nine .98 September .99

17. Features for Categorizing by Style • Function words • Syntax • SFL trees • Morphology • Complexity measures • “Unstable” words • Idiosyncrasies

18. Idiosyncrasies Human attribution experts use idiosyncrasies e.g. spelling errors, neologisms, quirky syntax • Use spelling/grammar checker to identify “errors” • Categorize errors by type e.g. ie  ei, nn  n

19. Some problematic feature sets • Many researchers use content words • A really easy feature set is character n-grams • These sets are problematic since they ought to fail when an author changes topics • In practice, they work very well

20. Learning -- Historical • From 1964 until 1990’s all work had flavor of Naïve Bayes • Idea was to characterize all of each candidate author’s work and see which one X is most similar to • Various proximity measures used (often after dimension reduction) • Since 1990’s variety of learning methods applied

21. Learning Algorithms Let’s try some old friends (and one new one): • Naïve Bayes • Decision Trees • Winnow • SVM • Bayesian regression

22. Three testbed corpora • Emails between two correspondents (date split) • Some as short as one word • Two books by each of nine authors (book split) • Different books have different content • Twenty bloggers (date split) • Lots of authors

23. Email corpus – 2 authors

24. Literature corpus – 9 authors

25. Blog corpus – 20 authors

26. Some Lessons Some lessons: • SVM and Bayesian regression are the best learners. • FW and single POS are generally enough. • In fact, SFL trees alone are often enough. • Content words and character n-grams work surprisingly well Some other facts worth knowing: • Common unstable words are better than FW. • For unedited texts, idiosyncrasies are best. • For some languages, morphology is needed.