Freenet - A Distributed Anonymous Information Storage and Retrieval System

1. Freenet Freenet - A Distributed Anonymous Information Storage and Retrieval System Topics in Reliable Distributed Computing Yoav Levy 2004

2. Freenet Freenet – Presentation Outline • Motivation & Goals • Anonymity & Censorship Resistant Networks • Small World Phenomena • Architecture • Keys - content-hash (CHK) and signed-subspace (SSK) keys • Search • Routing: the small-world paradigm • Storage Management • Freenet Performance • Comparing with Other Systems • Free Discussion

3. Freenet Sources • Protecting Free Expression Online with Freenet • I. Clarke, T. Hong, O. Sandberg • IEEE Internet Computing, January 2002: http://freenet.sourceforge.net/papers/freenet-ieee.pdf • Performance in Decentralized File-sharing Networks • T. Hong. In Proc. of the O’Reilly Peer-to-Peer Conference, San Francisco, California, February 14-16, 2001 • Censorship Resistant P2P Content Addressable Networks • A. Fiat & J Saia 2002 • Navigation in a small world • Jon M. Kleinberg, Nature, Aug. 2000 • Freenet explained: http://freenetgw.bishopston.net/freenet-explained

4. Freenet Freenet – Goals & Motivation • Preserving the privacy of information producers, consumers, and holders • Resistance to information censorship • Can Freenet withstand hackers? The government? • High availability and reliability through decentralization • The Slashdot effect • Efficient, scalable, and adaptive storage and routing • Resistance to denial attacks • What are the current trends?

5. Freenet From the Press… "...האם אין זה מתבקש, שטכנולוגיית החלפת הקבצים תכבוש גם את שידורי הטלוויזיה ותנסה להעביר אותם דרך האיטרנט לכל דורש ובלא תשלום?" "...כל מה שצריך לעשות הוא לחבר את הטלוויזיה למחשבןלהשתמש בתוכנה מתאימה שתפאשר להפיץ את השידור בין מאות ואלפי גולשים, שכל אחש מהם ישמש תחנת ממסר קטנה, המפזרת את עוצמת העיבוד ורוחב הפס הנדרשים להפצת שידור הטלויזיה." יובל דרור הארץ 2.11.2004

6. Freenet BPI wins court order against British P2P users (Oct. 14 2004) “British Phonographic Industry, the UK equivalent of the RIAA, has won the first court round in its fight against British P2P users, when British courts granted a court order against 28 P2P users BPI raided earlier this month. The court order means that users' ISPs have to hand over users' personal details (names and addresses) within 14 days to the BPI. According to BPI's claims, all of the users now sued are so-called "heavy uploaders", people who cotribute to P2P networks by sharing vast amounts of music rather, rather than downloading tracks.” Source: BBC

7. Freenet The Effect of File Sharing on Record Sales - An Empirical Analysis (March 2004) "We considerthe specific case of file sharing and itseffect on the legal sales of music. A datasetcontaining 0.01% of the world’s downloads ismatched to U.S. sales data for a largenumber ofalbums…Downloadshave an effect on sales which is statisticallyindistinguishable from zero, despite ratherprecise estimates.” Felix Oberholzer (Harvard Business School), Koleman Strumpf (UNC Chapel Hill)

8. Freenet Remailers and Mixnets • Traceable remailers • Keeping internal lists of senders – enables replies • Pseudonymous remailers – use cryptography • Problems? • Anonymous remailer • Do not keep any list • Is this really safe? • Mixmaster • By Lance Cottrell • Scrambles message contents • Mixnets • Introduced by David Chaum • A network composed of “mixers” - is a third party that combines and forwards messages from several senders to several recipients

9. Freenet Anonymity & Censorship Resistant Networks • A work by A. Fiat & Jared Saia • A content addressable network is defined as a distributed, scalable, indexing scheme for peer-to-peer networks. • Random vs. Adversarial: Resistance to Adversarial Node Deletion • P2P robust against random attacks “by design” • Tapestry used within Oceanstore, robust against random faults • Spam Resistance

10. Freenet Navigation in a small world • The small-world model • Milgram: six degrees of separation • Watts: between order and randomness • short-distance clustering + long-distance shortcuts

11. Freenet Webometrics • Ademic 1999 (.edu research) • AltaVista crawl (1999)

12. Freenet Chaos & The Power Law Effect

13. Freenet Technical Issues • Success Rates • Can we find the data? • Length of query paths • Scalability • Logarithmic / linear / polynomial • Robustness • Participants are unreliable • Different failure modes possible

14. Freenet Architecture • Each participant hosts a local data store as well as a routing table • Requests for files are made using location independent keys • Routing is propagated through chain of proxy requests • Graph structure adapts and evolves over time • Files may migrate between nodes

15. Freenet Key Based Searching • Content-hash keys (CHK):generated by hashing the contents of the file to be stored • Gives every file a unique absolute identifier • Identical copies of a file inserted by different people automatically coalesced • Signed-subspace key (SSK): sets up a personal namespace that anyone can read but only its owner can write to • Used as a “human” filename (as opposed CHK that may be thought of as i-nodes) • Enable easy file updates • Facilitate trust by guaranteeing that the same pseudonymous person created all files in the subspace, even though the subspace is not tied to a real-world identity

16. Freenet Retrieving Files • How do we locate the keys? • Hypertext spider • Indirect files – published with KSK of search words • Publish bookmarks • File retrieval • Request forwarded to node in RT with closest lexicographic match for the binary key • Request routing follows steepest-ascent hill climbing: first choice  failure  backtrack  second choice

17. Freenet c a b f e d Retrieving Files (contd.) • Request thread length controlled by use of timers and number-of-hops • Files are cached all along the retrieval path • Performance pro or con? • Self-reinforcing cycle – results in key expertise

18. Freenet Self Reinforced Routing • Snapshots using 300 requests with hops = 500 • As network converges it drops to 6 - “six degrees of separation”

19. Freenet Publishing • Similar to retrieval but, 2 step process • Detect collisions – ‘all clear’ if no collision • Publish to node in RT with closest key match • Are CD and publish paths same? • Can result in collision during publish step • Inserts allow new nodes to advertise themselves  Key-squatting is not effective

20. Freenet Data Management • Finite data stores - nodes resort to LRU • Routing table entries linger after data eviction • Outdated (or unpopular) docs disappear automatically • Bipartite eviction – short term policy • New files replace most recent files • Prevents established files being evicted by attacks

21. Freenet Network Growth • New nodes have to know one or more guys • Problem: How to consistently decide on what key the new node specializes in? • Needs to be consensus decision – else denial attacks • Advertisement  IP + H(random seed s0) • Commitment - H(H(H(s0) ^ H(s1)) ^ H(s2))……. • Key for new node = XOR of all seeds • Each node adds a RT entry for the new node

22. Freenet Network Growth • Key assigned to new nodes = H(IP) • Scales as log(n) until n ~ 40000 • At 40000, RTs are full

23. Freenet Fault Resilience • Median path length < 20 at 30% node failures? • N/w becomes ineffective at 40% failures ???

24. Freenet P(n) ~ 1/n1.5 Links in the small world • “Scale-free” link distribution • P(n) = 1/nk • most nodes have only a few connections • some have a lot of links • important for binding disparate regions together • e.g. Tim O’Reilly

25. Freenet The importance of routing • Existence of short paths is not enough – they must be found • Adaptivity helps Freenet find good paths • Compare: a random-routing network

26. Freenet Scalability • Real-world networks are much larger • nearly 400,000 downloads of Freenet • 50 million Napster users • How well does Freenet scale?

27. Freenet Random failure

28. Freenet Targeted attack

29. Freenet Security & Privacy • Is true anonymity guaranteed? • File integrity - KSK vulnerable to dictionary attacks • DOS attacks – Hash Cash to slow down • Attempts to displace valid files are constrained by the insert procedure

30. Freenet Other Systems • Queries • Freenet, Chord - routed queries • Napster - centralized lookup: simple, but O(N) state and a single point of failure • Gnutella – flooded queries: robust, but worst case O(N) messages per lookup • Freenet and Chord • Unlike Chord, Freenet does not guarantee that a query will return a specific document that exist on the network. • Unlike Chord, Freenet does not assign keys to specific nodes, thus maintaining data store anonymity

31. Freenet Other Systems (Contd.) • Freenet and Gnutella • Both Gnutella and Freenet are distributed Information systems. • They differ significantly in both goals and implementation. • Basically, • Each is a system for searching for information • Each returns information without telling you where it came from • Freenet and Publius • Both provide publisher anonymity, deniability, and censorship resistance • Freenet provides anonymity for retrievers and servers, as well • Cost is high: data must be cached at many nodes • Publius provides persistence of data, Freenet does not

32. Freenet Free Discussion • Does the solution answer the requirements? The need? • Paper Strengths & Weaknesses

33. Freenet Thank You!