Enforced Community Standards For Research on Users of the Tor Anonymity Network

1. Enforced Community Standards For Research on Users of the TorAnonymity Network Christopher Soghoian

2. McCoy et al (‘08): Shining a light on dark places • Researchers created Tor exit server. • During 4 day period in December 2007, they logged the 150 bytes of each network packet that went through their network. • This revealed the kind of traffic and the specific sites that users were visiting. • The researchers also ran a tor entry server for 15 days. • They gathered source IP addresses of users in order to later geo-locate them.

3. McCoy et al (‘08): Shining a light on dark places • The researchers did not seek or obtain prior legal analysis of their work. • They did speak informally to a law professor at their university for a few minutes who they said told them that “that area of the law is ill defined.“ • Based on this, they decided that is was “unnecessary to follow up with other lawyers .“ • The researchers also did not consult with an Institutional Review Board. • “We were advised that it wasn't necessary," one of the researchers said, adding that the IRB review process is used “used more in medical and psychology research at our university," and was not generally consulted in computer science projects.

4. Community response to McCoy et al • The researchers did not receive a warm welcome after presenting their work at the Privacy Enhancing Technologies Symposium. • When questioned by an audience member after the presentation, the researchers admitted that they had retained a copy of the logged Tor traffic, and further, that it was not held on an encrypted storage device. • This disclosure was met with boos from the audience, even after the researchers stressed that the data was kept in a “secure" location.

5. McCoy et al and their university IRB • After some bad press (ahem), the university of Colorado quickly launched an investigation into the research. Within days, the researchers were cleared. • “Based on our assessment and understanding of the issues involved in your work, our opinion was that by any reasonable standard, the work in question was not classifiable as human subject research, nor did it involve the collection of personally identifying information. While the underlying issues are certainly interesting and complex, our opinion is that in this case, no rules were violated by your not having subjected your proposed work to prior [IRB] scrutiny. Our analysis was conned to this [IRB] issue."

6. Castellucciaet al (‘10): Private Information Disclosure from Web Searches • In this project, the researchers discovered that with stolen session cookies (captured via open WiFi networks), it was possible to reconstruct users’ search history. • In addition to demonstrating the flaw, the researchers also sought to determine the degree to which users are vulnerable (how many users conduct web searches when “logged in" to a search engine and how many have enabled Google's Web History feature).

7. Castellucciaet al (‘10): Private Information Disclosure from Web Searches • In order to determine this information, the researchers collected data via three different methods: • First, network traces for the 500-600 daily users at their own research center were collected and analyzed. These accounts were not actively attacked. The researchers merely engaged in passive analysis of these network traces. • Second, the researchers received opt-in consent from 10 users, whose Google session cookies the researchers sniffed, and then used to actively reconstruct the individuals‘ search history information.

8. Castellucciaet al (‘10): Private Information Disclosure from Web Searches • Third, the researchers established a rogue tor exit server. • During the one week period in which the researchers collected data from the Tor network, 1803 distinct Google users were observed, 46% of which were logged into their accounts. • For each of these logged-in users, the researchers used the sniffed Google session cookies and attempted to access the users' first and last name; locations searched using Google Maps (along with the “default location", when available); blogs followed using Google Reader; full Web History (when accessible without re-entering credentials); financeportfolio; and bookmarks. • The researchers stressed that their research application did not store any individual users' data. Only aggregate statistical information was retained.

9. Privacy of colleagues > Tor users • In their paper, the researchers describe why they sought consent to engage in active attacks against the 10 users who volunteered for their study: • “it would have been otherwise impossible to conduct our study on uninformed users without incurring legal and ethical issues.“ • Unclear why it was legally and ethically OK to violate the privacy of Tor users. • Also noteworthy that they were wiling to engage in active attacks against Tor, but only passive sniffing on their employer’s network.

10. Analysis of these two studies • McCoy et alspecifically sought to learn more about users of the Tor network, whereas Castelluccia simply used Tor users‘ network activity to assist in drawing broader conclusions about general Internet behavior • The fact that Castellucciaet al. performed only passive network monitoring on their own colleagues but actively attacked the accounts of Tor users likely indicates that the researchers knew they were engaging in morally and ethically dubious behavior. • If there were no problems with what they were doing, why would they not do it to their friends and colleagues, but were willing to do it to users who ha specifically signaled a desire to protect their own privacy?

11. Analysis of these two studies • Neither research team submitted their study to an IRB • McCoy et al did not believe they had to, while Castelluccia et al. did not have an IRB at their research institution. • Castellucciaet al. specifically designed their research tool to analyze individual users' data in-memory, and only retained aggregate statistical data. • McCoy et al. retained individual users' browsing data, and performed statistical analysis of it after the fact.

12. Many other studies since • These are not the only academic studies to collect data via Tor exit servers. • There have been several others since (one even won a best paper award). • Unless something is done, this trend will likely continue.

13. Our publishing system offers skewed incentives • Some of the attendees at PETS 2010 did not respond positively to the study. • When the community response to the ‘08 PETS paper by McCoy et al was raised, the researchers stated that they were not at PETS ’08, and had no idea that it had resulted in such criticism. How should they know? Where was it documented? • Negative community response is not mentioned on PETS 08 site or ACM Portal. • McCoy et al’s own person home pages and CVs do not mention it. • The very same thing applies to the paper from PETS ‘10. • To the casual observer, it looks like the community approves of this kind of research (since the papers are approved at conferences with low accept rates).

14. We are a guild. It is time to flex our power.

15. Establishing a standard for acceptable research • My ideas: • Research should be focused on users of the Tor network • Not just using Tor as an easy way to get data on Internet users’ browsing activity. • Minimize user data collection and retention. • The research should be legal in the country where it is performed. • This may mean no rogue academic exit servers in the US • Research studies should be vetted by an IRB, if one exists

16. Enforcing this standard • The academic community needs to pick a reasonable, common-sense standard. • Then, they need to enforce it. • Papers that do not meet such standards should not be accepted by any of the respected conferences. • Example: SOUPs now requires researchers to include a section on how ethical concerns were addressed, including consulting with an IRB. • If SOUPs can do it, PETS and Oakland can and should do so too.