VoD BitTorrent Framework

1. VoD BitTorrent Framework

2. Background • BT is a very popular peer to peer protocol with many implementations: • http://en.wikipedia.org/wiki/Comparison_of_BitTorrent_clients • The first one was created in Python by Bram Cohen.

3. Real world motivation • BitTorrent is common and Free • Most chances there is a swarm for your movie • VoD servers are expensive and complicated • Its better to use P2P for VoD

4. Some definitions • StratUp Delay (D) = How much time I am allowed to hold before starting to play M (Movie). • Prefetch Time (P) = How much time before I watch byte B from M, it should arrive. • Playback Rate (R) = How many bytes/second I watch • B(t) – the index of the bit that must arrive by t. • B(t) = R *(P+t-D)

5. Rules and Examples • B(t) = R *(P+t-D) • If R=5 KB/sec and D=6 seconds and P=2 seconds, than: • At t=6 seconds, I must have 10KB continuous from the start of the movie. • At t=18 seconds I must have 70KB continuous from the start of the movie.

6. What does my server suppose to do • Your server suppose to get from the Bittorent clients pieces of data needed to play the video. • Each piece needs to arrive by its deadline. • If the piece does not arrive by the deadline you “can download it freely” from the server. • In this case you encounter a penalty • The number of penalty points you accumulated whole getting/playing the video is your (negative) score.

7. How do I test my server/idea • Create my own VoD BitTorrent client. • Simulate VoD server – implement a function that copies from a seed directly to my file (Checkpoint 1). • Count the bytes your client received from the VoD Server (Checkpoint 1) • Aim at minimizing the part of the movie downloaded from VoD Server.

8. Tracker BT BT BT BT BT BT BT BT BT BT VOD BT Execution example run-all.sh

9. How does it work? • Run tracker • Run the “world” • Run your client (if given)

10. How does it work? • Both tracker and the “world” run on the same computer • we have • pc-hanoch-w3 • Please reserve time in: • http://bt-p2p-vod.wikispaces.com/ • Or from course www - "Scheduling computers for tests."

11. The main script - run_all.sh • Location: • /users/courses/hanochworkshop/bittorrent • run_all_vod.sh: • Parameters: • Start / stop • My client – optional

12. run_all_vod.sh operation • Start: • Create download directory • Run the “world” • Each world client starts with different “saved” amount. • Run “my client” – if given • All upload rates are restricted (simulate practical world). • Stop • Kill all python threads

13. To run client separately: • BitTornado-My-Client/btdownloadheadless.py • --saveas /vol/scratch/bt-download/picture.tst.bmp • --max_upload_rate 70 • --security 0 • ./picture.pc-hanoch-w3.bmp.torrent (Implement in Checkpoint 1): • -- Rate 60 • -- Delay 6 • -- Prefetch 2 • Not necessarily on the same machine as the world.

14. What implementation we use • The clients are a modified version based on an up to date version of Bram Python Client. • Modifications: • “World” clients • do not print as it would only make a mess • reincarnate after the finish to make a real world feel. • “My Client” template – kills all clients when finished

15. Python • Very common script language • Intuitive and readable • References: • python.org :http://docs.python.org/reference/ • http://wiki.python.org/moin/BeginnersGuide/NonProgrammers • http://wiki.python.org/moin/BeginnersGuide/Programmers • The book "Think like a Computer Scientist in Python" has a very good reputation:

16. My Client • We supply a default “My Client” In: • BitTornado-My-Client/btdownloadheadless.py and “BitTornado - Your Code” link. • It is like the “standard” one.

17. Getting started • Run the environment • ssh pc-hanoch-w3 or pc-hanoch-w4 • cd /users/courses/hanochworkshop/bittorrent • run_all_vod.sh • start • [BitTornado-My-Client/btdownloadheadless.py] • Note tracker prints a message for each GET. • You can connect to the tracker, from the CS network, with a browser at: • http://pc-hanoch-wk:6970/ • Do ls -l /home/pc-hanoch-w4/home/hillelav/$USER to see downloaded files

18. What is it doing? • Run bttrack. • Copy From FreshFiles a set of “downloaded” files to download directory. • Run X seeds • Run a python client per “downloaded” file. • Run my client if given

19. Directory Structure • BitTornado-My-Client – • high level application • BitTornado-My-Client/BitTornado – • services • BitTornado-My-Client/BitTornado/BT1 – • Actual work

20. Client Operation • btdownloadheadless.py – First to run. • Performs initializations. • get_response – parse the .torrent • Infohash – the ID of the .torrent • startEngine – Start connection with peers • startRerequester - Start connection with tracker • Run rawserver.listen_forever from RawServer.py. • listen_forever is: • Polling port • executing func() from a task queue.

21. Receiving messages • From listen_forever loop it goes to: • BT1/Connecter.py/got_message • CHOKE • UNCHOKE • INTERESTED • NOT_INTERESTED • BITFIELD • REQUEST • CANCEL • PIECE • HAVE

22. Influencing decisions • Can be done in in one of the callbacks which spring from Connecter.py / got_message: • Downloader.py • HAVE->got_have->send_interested • Tune for VoD?

23. BT tracker • Send GET request in Rerequester.py / announce • Receive GET data in thread in Rerequester.py / _rerequest

24. Influencing decisions • In Encrypter.py / start_connection or _start_connection_from_queue For instance try different max_connections. _rerequest_single – check timout for scheduling.

25. Policy • Choker.py/_round_robin – decide who to choke / optimistically un choke / send_have to. • Downloader.py / _request_more. • PiecePicker.py / next - rarest piece, etc. Tune for VoD? • got_unchoke->_request_more->next…

26. More about the code • Start your project from “BitTornado - Your Code” link. • Click “BitTornado - Code Guide” for basic explanations about the code.