340 likes | 526 Views
Unstructured P2P Networks. Achieving Robustness and Scalability in Data Dissemination Scenarios Michael Mirold Seminar on Advanced Topics in Distributed Computing 2007/2008. Structured P2P Node graph with predefined structure Examples Chord Pastry CAN. Unstructured P2P
E N D
Unstructured P2P Networks Achieving Robustness and Scalability in Data Dissemination Scenarios Michael Mirold Seminar on Advanced Topics in Distributed Computing 2007/2008
Structured P2P Node graph with predefined structure Examples Chord Pastry CAN Unstructured P2P Random node graph Examples Gnutella BitTorrent Introduction
Part I Do Ut Des, Tit-for-Tat or “How Leech Proof is BitTorrent Really?” • Bases on papers: • “Incentives Build Robustness in BitTorrent”, Bram Cohen • “Do Incentives Build Robustness in BitTorrent?”, • Michael Piatek, Tomas Isdal, Thomas Anderson, • Arvind Krishnamurthy, Arun Venkataramani
What is BitTorrent? • Used for distribution of single files • File is split into pieces (32kB – 2MB) • Pieces are distributed within a swarm • denotes all nodes that are interested in the file • Downloaded pieces are redistributed • No single “server”: True peer-to-peer (except perhaps of tracker)
How Does BitTorrent Work? (1) Torrent File - url of tracker - name: name of file - piece length - pieces (conc. of SHA1 hashes of all pieces) - file length “Ordinary” Web Server Tracker 1.2 Put Torrent-File onto web server 1.3 Register as “downloader” Want really badly to make everyone enjoy my new holiday pictures (HDR some GB) 1.1 Create Torrent-File Initial Seed
How Does BitTorrent Work? (2) “Ordinary” Web Server Tracker 2.2 Register as downloader 2.1 Request Torrent-File 2.3 Send peer set (“Local Neighborhood”) 2.4 Open connection 2.5 Handshake 2.6 Request Initial Seed 2.7 Send pieces
At some Unspecial Moment… active set statically sized swarm B active set choke active our node seed peer set non-seed
Achieving Fairness in BitTorrent • Basis: Tit-for-Tat strategy • Famous in game theory (considered “best” strategy for winning Prisoners’ Dilemma) • Idea: • Be cooperative to others • Penalize defective behaviour but don’t be too unforgiving • Put in BitTorrent context • Grant upload capacity to n best uploaders • n: size of active set + # of optimistic unchokes • “Choke”, i.e. stop uploading to, peers that don’t perform well • recompute choking every 10 seconds • However: “Optimistically unchoke” peers • twice every 30 seconds Reward good uploaders with capacity
Sounds good in theory, but… • what if you have LOTS of upload capacity? • most of your peers are slower (see later) • nevertheless you must choose a few • fastest peers probably already done • AND you have split your upload capacity equally! • you give your capacity away for free • this is called “ALTRUISM” • Altruism THE source of unfairness in BitTorrent • Unfair peers only need • be slightly “better” than average • prefer rich girls, i.e. highly altruistic peers remember: active set has static size
Unfairness/Altruism Illustrated Measure of altruism
Unfairness/Altruism Illustrated (2) Altruism as wasted upload capacity fast clients contribute more than necessary slow clients never get reciprocated every byte is wasted
Real World Observations This is what you are competing against be a bit faster! If you can upload > 14kB/s reciprocation prob > 99% !! Reference implementation uses active set size of
The Optimal Active Set Size * Optimal Set Size * for a peer with 300 kB/s UL capacity
BitTyrant – A Selfish BitTorrent Client • Based on Azureus Client • publicly available at http://bittyrant.cs.washington.edu • Exploiting the unfairness in BitTorrent • Minimizing altruism to improve efficiency • Mechanisms: • choose only “best” peers with respect to UL/DL ratio (see next slide) • deviate from equal split • optimize active set size
Peer Choice Optimization Algorithm • Step invariant: Maintain up, dp of peer p • dp: Estimated download performance from p • up: Estimated upload needed for reciprocation with p • Initially: set according to theor. distribution • Each step: Rank order peers according to and choose best peers for unchoking • After each round: • Has p unchoked us? • Has p not unchoked us? • Has p unchoked us for the last r rounds?
Experiences using BitTyrant • One BitTyrant peer: • median: 72% • Multiple BitTyrant peers: • depends on a few factors and not easily comparable • strategic, i.e. peers use adapted choking algorithm swarm performance improved compared to BitTorrent • strategic & selfish, i.e. peer doesn’t give back excess capacity • swarm performance decreases dramatically
Personal Opinion • Paper shows nice “hack” • Paper shows that there is no perfect fairness • Paper shows a sensible optimization • But: I think, model is too restricted • People’s goals unconsidered • Altruistic people are often just that: altruistic (they don’t mind performing suboptimal) • Everyone glad with BitTorrent, why optimize? • And: will this paper make the world a better place?
Part II Getting Almost Reliable Broadcast with Almost no Pain: “Lightweight Probabilistic Broadcast” • Bases on papers: • “Lightweight Probabilistic Broadcast”, 2003, Eugster, Guerraoui, Handurukande & Kouznetsov
Background & Motivation • Large scale event dissemination • Processes p1, …, pn subscribe for topic t • Event e with topic t is delivered to p1, …, pn • Reliable Broadcast • scales poorly • Network level Broadcast/Multicast • lacks reliability guarantees • also scalability problems • Complete view on the network leads to unsustainable memory demands
The lpbcast Protocol • System contains n processes Π = {p1, …, pn} • dynamically joining and leaving • Processes subscribe to a single topic • easily extendible to multiple topics • joining/leaving == subscribing/unsubscribing • Gossip sent to F random nodes in viewiof process pi • F is “fanout” of process • viewi is subset of procs currently known by pi • Gossips sent out periodically (non-synchronized)
Gossips • gossip: all-in-one record containing • gossip.events: event notifications • gossip.subs: subscriptions, • gossip.unsubs: unsubscriptions, • gossip.eventIds: history of events ids received so far • Containers don’t contain duplicates, i.e. they are set-like lists
Processes • Every process p has several buffers: • view(fixed maximum size l): • contains processes that are known to / seen by p • subs(fixed maximum size |subs|M): • contains subscriptions received by p • unsubs(fixed maximum size |unsubs|M): • contains unsubscriptions received by p • events(fixed maximum size |events|M): • contains event notifications since last gossip emission • eventIds (fixed maximum size |eventIds|M): • ids of events seen so far • retrieveBuf: • contains ids of events seen in gossip.eventids but not known
Example Process pi Fanout: 3 |view|M = 8 view (pi)
lpbcast procedures • Upon receiving a gossip message • Update view and unSubs with unsubscriptions • Update view with new subscriptions • Update events with new notifications • deliver notifications • update retrieveBuf for later retrieval of notifications • Perform housekeeping (truncate containers) • When sending a gossip message • fill gossip message accordingly
Example: Zeit T0 GOSSIP 1 2 GOSSIP 3 GOSSIP
Example: Zeit T1 GOSSIP 1 2 GOSSIP GOSSIP 3 GOSSIP
Analytical Evaluation of lpbcast • Assumptions: • Π constant during evaluation • synchronous rounds • upper bound on latency • identical fanout F for all processes • probability of message loss ε • probability of crash τ • random view ind. uniformly distributed
Analytical Evaluation • Turns out that throughput is independent from view size l • provided that views are uniformly distributed • Membership stability (no partitioning) • increases with growing view size and/or system size • partitioning probability increases slowly with rounds: 1012 rounds for n=50, l=3
Practical Observations • Throughput does depend a bit on l • Explanation: • views not as uniformly distributed as assumed
Time to Infect all Processes • Simulation meets measurements pretty well • Fanout 3 • 1 msg injected • System size varies