Privacy & Stylometry: Practical Attacks Against Authorship Recognition Techniques

1. Privacy & Stylometry: Practical Attacks Against Authorship Recognition Techniques Michael Brennan and Rachel Greenstadt {mb553,greenie}@cs.drexel.edu

2. Overview • Introduction to Stylometry/Authorship Recognition • The Threat to Privacy & Anonymity • Experimental Results • Attacking Authorship Recognition • Study Setup • Methodology • Results • Future Work 2

3. What is Authorship Recognition? • The basic question: “who wrote this document?” • Stylometry: The study of attributing authorship to documents based only on the linguistic style they exhibit. • “Linguistic Style” Features: sentence length, word choices, syntactic structure, etc. • Handwriting, content-based features, and contextual features are not considered. • In this presentation, stylometry and authorship recognition are used interchangeably.

4. Stylometry: A Brief History • The classic stylometry problem: The Federalist Papers. • 85 anonymous papers to persuade ratification of the Constitution. 12 of these have disputed authorship. • Stylometry has been used to show Madison authored the disputed documents. • Used as a data set for countless stylometry studies. • Modern Stylometry Based in Machine Learning • SVMs, Genetic Algorithms, Neural Networks, Bayesian Classifiers… used extensively.

5. Why is Stylometry Important? • Great, you can figure out who wrote a 200 year old document, so what? • From the Institute for Linguistic Evidence: • “In some criminal, civil, and security matters, language can be evidence… When you are faced with a suspicious document, whether you need to know who wrote it, or if it is a real threat or real suicide note, or if it is too close for comfort to some other document, you need reliable, validated methods.” • Plagiarism, Forensics, Anonymity…

6. Stylometry: the Threat to Privacy • Good techniques for location privacy (Tor, Mixes, etc). • But it may be insufficient! • Stylometry can identify authors based on their writing. • Can anonymous authors defend against this? • ~6500 words to leak identity – Rao, Rohatgi. 2000. • “The Multidisciplinary Requirement for Privacy” – Carlisle Adams. 2006. 6

7. Supervised Stylometry • Given a set of documents of known authorship, classify a document of unknown authorship. • Classifier trained on undisputed text. • Scenario: Alice the Anonymous Blogger vs. Bob the Abusive Employer. • Alice blogs about abuses at Bob’s company. • Blog posted anonymously (Tor, pseudonym, etc). • Bob obtains 5000-10000 words of each employee’s writing. • Bob uses stylometry to identify Alice as the blogger. 7

8. Unsupervised Stylometry • Given a set of documents of unknown authorship, cluster them into author groups. • No pre-existing author information. • “Similarity Detection” • Scenario: Anonymous Forum vs. Oppressive Government. • Participants organize protests. • Posts are completely unlabeled (no pseudonyms) • Unknown organizational structure, number of authors, etc. • The government applies unsupervised stylometric techniques. • Number of authors may be discovered, author profiles created. • Results fed into supervised stylometry system to identify individuals. 8

9. The Threat in Numbers (Unsupervised) • 9 Anonymous authors, raw forum data: • Only 35% Accuracy. • Posts 25-200 words long • 5 authors, same data set, modified for longer posts: • 88.2% Accuracy • Passages 500-750 words long • Other Research on 50 Unique Authors (WritePrints): • Unsupervised “Pairing”: 95% 9

10. The Threat in Numbers (Supervised) • Supervised Classification of 2-5 Authors: 79-99% Accuracy (IAAI ‘09). • Random sets taken from 15 subject pool. • 3 Different Methods Used • Other Research on 50 Unique Authors (WritePrints): • Supervised: 90% 10

11. Protecting Privacy: Attacking Stylometry • Problem • Can stylometry be attacked? How so? How Easily? • Evaluation • In depth study on attacking multiple methods of Stylometry. • Conclusion • Stylometry is very vulnerable to attack by inexperienced human adversaries. • Attacks can be used to protect privacy. 11

12. Related Work • Adversarial stylometry is not a well-researched part of the field. • Somers & Tweedie, 2003: Authorship Attribution & Pastiche. Mixed Results. • Kacmarcik & Gamon, 2006: Obfuscating Document Stylometry to Preserve Anonymity. Encouraging results. • Our goal: Assess the general vulnerability of authorship recognition systems when under attack by human adversaries.

13. We’re Under Attack! • Obfuscation Attack • An author attempts to write a document in such a way that their personal writing style will not be recognized. • Imitation Attack • An author attempts to write a document such that the writing style will be recognized as that of another specific author.

14. Study Setup & Format • 3 representative methods of stylometry. • 15 Individual Authors. Participation had three parts: • Submit 5000 words of pre-existing writing from a formal source. • Write a new 500 word passage as an obfuscation attack. • Task: Describe your neighborhood. • Write a new 500 word passage as an imitation attack. • Task: Imitate Cormac McCarthy, describe your day. • Authors had no formal training or knowledge in linguistics or stylometry.

15. Imitating Cormac McCarthy “On the far side of the river valley the road passed through a stark black burn. Charred and limbless trunks of trees stretching away on every side. Ash moving over the road and the sagging hands of blind wire strung from the blackened lightpoles whining thinly in the wind.”

16. Imitation Attack Examples • “Light sliced through the blinds, and construction began in the adjacent apartment. The harsh cacophony crashed through the wall.” • “Hot water in the mug. Brush in the mug. The blade read ‘Wilkinson Sword’ on the layered wax paper packaging.” • “He fills the coffee pot with water, after cleaning out the putrid remains of yesterday's brew. The beans are in the freezer, he remembers.”

17. Methodology & Evaluation • Validation Experiment: Verify accuracy claims of all 3 methods. • Attack Experiment: Determine accuracy when attacked using the same experimental conditions. • Experimental Conditions: • Each test was applied to four random sets of 2, 3, 4, and 5 distinct authors. • Same sets used in each method. • Cross-validation used to measure accuracy (repeated sub sampling and leave-one-out).

18. Machine Learning: Quick Introduction • Machine Learning is AI • Learn & recognize patterns automatically, make decisions. • Most common incarnation of ML: Classifiers • Spam Filter: Bayesian Classifier • Given a message, what is the probability that it is spam or not spam given the extracted features. • Important to pick good features.

19. A Closer Look at Linguistic Features • Basic Measurements: • Average syllable/word/sentence count, letter distribution, punctuation. • Lexical Density • Unique_Words / Total_Words • Gunning-Fog Readability Index: • 0.4 * ( Average_Sentence_Length + 100 * Complex_Word_Ratio ) • Result: years of formal education required to read the text.

20. Stylometry Methods • Method 1: Signature Stylometric System. • Three features: word length, letter usage, punctuation usage. 95% base accuracy.

21. Stylometry Methods • Method 2: Neural Network Classifier • Based on Singh & Holmes, 1996. • 9 linguistic features. • Lexical density, unique word count, character counts with and without spaces, two readability indexes, sentence count, average sentence length, and average syllables per word. • 78.5% base accuracy. • NN: simple switches,complex overall system. Hidden Input Output

22. Stylometry Methods • Method 3: Synonym-Based Classifier. • Based on Clark & Hannon, 2007. • Examines word choice when presented with all possible synonyms. 88-99% accuracy. (Diagram from “A Classifier System for Author Recognition Using Synonym-Based Features” by Clark & Hannon)

23. Accuracy on Training Set All three methods were highly accurate in recognizing the correct author of the training set.

24. Obfuscation Attack Detection Obfuscation attacks brought the accuracy of each system to that of random chance or worse.

25. Imitation Attack Detection Imitation attacks all but completely circumvented all three authorship recognition methods.

26. Imitation Attack Success The imitation attacks were very successful in targeting the intended victim of the attack.

27. Questioning the Results • Why are some authors more susceptible to attack? • Why did certain methods perform better than others? • Would these results hold up in a stricter domain? • What if the participants were skilled in linguistics or stylometry? • What features and methods could offer some resistance to these attacks?

28. Using Attacks to Improve Privacy • What you can do: • Important: These these methods failed, others might not. • Best: Try to imitate someone else. • Write less. • What is needed • A tool that will assist in anonymizing writing styles. 28

29. Building a Tool • What a style-modifying tool needs: • A variety of methods & features. • A large corpus of existing authors. • AI-Assisted (Human oversight to preserve content) • Non-Stylometry Features: • Contextual Clue Identifiers (Time, Content) • Building a Corpus • IRB Approval • Multiple Languages • Your help!

30. Future Work • Testing more stylometric methods. • Imitation attacks against other authors. • Effectiveness in unique domains (aggregated short messages) • Extracting additional information (demographic) • Better demonstration of effective unsupervised stylometry.

31. Conclusion • Not the end of stylometry in sensitive areas • New methods should test for adversarial threats. • Stylometry is useful, but can also present a threat to privacy. • Attacking stylometry to preserve privacy has high potential. • Potential for arms race: • Developing attack-resistant methods of stylometry vs. creating new attacks to preserve privacy.

32. Questions? • Contact: • Mike Brennan: mb553@drexel.edu • Rachel Greenstadt: greenie@cs.drexel.edu • www.cs.drexel.edu/~{mb553,greenie} • Interested in learning more? • “Can Pseudonymity Really Guarantee Privacy?” – Josyula Rao, Pankaj Rohatgi. 2000. • “The Multidisciplinary Requirement for Privacy” – Carlisle Adams. 2006. • “Writeprints: A stylometric approach to identity-level identification and similarity detection in cyberspace” – Ahmed Abbasi, Hsinchun Chen. 2008.

34. FAQ • “When will your data set be released?” • “I can think of features for a stylometry system that I bet these authors did not consider” • “I know a person in a certain Three Letter Organization that has subtle ways of detecting authorship that would surely not be broken by these attacks.” • “What about function words?”

35. Message Board Posts vs. Original Data

36. Study Setup: Detailed • 3 representative methods of stylometry. • 15 Individual Authors. Participation had three parts: • Submit 5000 words of pre-existing writing from a formal source. • Formal means school essays, professional reports, etc. No slang, abbreviations, casual conversation. • Write a new 500 word passage as an obfuscation attack. • Task: Describe your neighborhood. • Write a new 500 word passage as an imitation attack. • Task: Imitate Cormac McCarthy using a passage from The Road, write a third person narrative about your day starting from when you wake up. • Authors had no formal training or knowledge in linguistics or stylometry.

37. Discussion • Training Set Content and Size • The amount of training text for each author not exceptionally large, but the number of authors is significantly larger than most studies. • Allowed us to see interesting patterns, such as authors who were better at creating attacks. • Certain authors particularly susceptible to obfuscation attacks. Do they have a “generic” writing style? • Could be beneficial to study the effects of adversarial attacks in stricter domains. • Participant Skill Level • The lack of linguistic expertise of the participants strengthens our conclusions. It would be reasonable to expect authors with some level of expertise to do a better job at attacking these methods. • Examining what features and methods might offer resistance to these attacks is a viable avenue of research.