CS 598IG: Adv. Topics in Distributed Systems Spring 2006

1. CS 598IG: Adv. Topics in Distributed SystemsSpring 2006 Indranil Gupta Lecture 1-29 January 1, 2006-May 2, 2006

2. Agenda for this Class • Course Review • Articles • Course evaluations

3. Can you name some examples of Operating Systems?

4. Can you name some examples of Operating Systems? … Linux WinXP Unix FreeBSD Mac 2K Aegis Scout Hydra Mach SPIN OS/2 Express Flux Hope Spring AntaresOS EOS LOS SQOS LittleOS TINOS PalmOS WinCE …

5. What is an Operating System?

6. What is an Operating System? • User interface to hardware (device driver) • Provides abstractions (processes, file system) • Resource manager (scheduler) • Means of communication (networking) • …

7. Can you name some examples of Distributed Systems?

8. Client-server (e.g., NFS) The Internet The Web An ad-hoc network A sensor network DNS Kazaa (peer to peer overlays) Distributed Systems Examples

9. What is a Distributed System?

10. The definition we started with A distributed system is a collection of entities, each of which is autonomous, programmable, asynchronous and failure-prone, and communicating through an unreliable communication medium. • Our interest in distributed systems involves • algorithmics, design and implementation, maintenance, study • Entity=a process on a device (PC, PDA, mote) • Communication Medium=Wired or wireless network

11. A range of interesting problems for Distributed System designers • Routing [IP,BGP] • Multicast [IP multicast, SRM, RMTP] • Post and retrieve [Usenet] • Search [Kazaa, Google] • Storage [Databases] • Coordination [SETI@Home]

12. A range of challenges • Failures • Asynchrony • Scalability • Security

13. Laundry List of Topics we’ve Covered • DHTs • Sensor motes and TinyOS • Impossibility of consensus • Overlays • Epidemics • Peer to peer applications – file systems • Cooperative web caching • Sensor net routing • In-network processing in sensor nets • Sources of unreliability in the Internet and wireless nets • Real characteristics of p2p systems

14. Laundry List 2 • Design of Protocols • Practical theory perspectives • A Step Back • The Grid • Structure of networks • Handling Stress • Membership protocols • Publish-Subscribe • Security • Selfish algorithms • Feynman

15. CS 598: Scattered Systems Infrastructured D.S.’s e.g., Internet-based Distributed System (D.S.) Theory Non-infrastructured D.S.’s e.g., ad-hoc network based

16. CS 598: Scattered Systems Peer to peer systems D.S. Theory Sensor Networks

17. CS 598: Scattered Systems …Chord, Pastry, File systems, Web caching, Security, trust and copyright… D.S. Theory …Smart Dust, TinyOS, Aggregation, In-network processing…

18. Interesting: Area Overlaps Epidemics NNTP Gossip-based ad-hoc routing

19. Course Projects • Long Vu, Characterizing the PPLive ,edia streaming system • Ercan Ucan, New algorithms for scheduling gossips • Ravi Sathyam, Reducing communication overhead in P2P networks with currency schemes • Thirumala Sayampu, A study of commonalities among P2P systems • Juan Jose Jaramillo and Muyuan Wang, Reputation-based algorithms for mesh networks • Brandt Dusthimer and Mike Earnhart, SharedFS: a deployable P2P file System • Praveen Jayachandran and Raghu K. Ganti, Content-based publish-subscribe in sensor networks • Hussam Abu-Libdeh and Mehdi Bakht, Shopping Buddy: a P2P deal finder • Bach Bui and Maifi Khan, A deterministic hard real-time routing protocol for sensor networks

20. CS 598IG Ongoing Projects P2P D.S. Theory Ad-hoc/ Sensor networks

21. CS 598IG Ongoing Projects P2P D.S. Theory Ad-hoc/ Sensor networks Juan Jose Jaramillo and Muyuan Wang, Reputation-based algorithms for mesh networks Praveen Jayachandran and Raghu K. Ganti, Content-based publish-subscribe in sensor networks Hussam Abu-Libdeh and Mehdi Bakht, Shopping Buddy: a P2P deal finder Bach Bui and Maifi Khan, A deterministic hard real-time routing protocol for sensor networks

22. CS 598IG Ongoing Projects P2P D.S. Theory Ad-hoc/ Sensor networks Juan Jose Jaramillo and Muyuan Wang, Reputation-based algorithms for mesh networks Praveen Jayachandran and Raghu K. Ganti, Content-based publish-subscribe in sensor networks Hussam Abu-Libdeh and Mehdi Bakht, Shopping Buddy: a P2P deal finder Bach Bui and Maifi Khan, A deterministic hard real-time routing protocol for sensor networks

23. CS 598IG Ongoing Projects P2P D.S. Theory Ad-hoc/ Sensor networks Ercan Ucan, New algorithms for Scheduling Gossips Ravi Sathyam, Reducing communication overhead in P2P networks with currency schemes Juan Jose Jaramillo and Muyuan Wang, Reputation-based algorithms for mesh networks Hussam Abu-Libdeh and Mehdi Bakht, Shopping Buddy: a P2P deal finder

24. CS 598IG Ongoing Projects P2P D.S. Theory Ad-hoc/ Sensor networks Long Vu, Characterizing the PPLive Media Streaming System Thirumala Sayampu, A study of commonality among P2P systems Brandt Dusthimer and Mike Earnhart, SharedFS: a deployable P2P file System

25. CS 598IG Ongoing Projects P2P D.S. Theory Ad-hoc/ Sensor networks All Projects well-placed to explore overlaps from across all areas!

26. Leftover Work • Final Project Report Submissions – 11.59 pm, Friday May 12th, 2006 (email softcopy to indy@ad.uiuc.edu, turn hardcopy in to 3112 SC). • At most 12 pages, at least 12 pt font • Final extension, Hard deadline • (should contain hard and comprehensive data) • Three Best Projects will be up on website asap after the 12th

27. Presentations I hope you liked the selection of papers. Special mention presentations • Everyone! (difficult to pick “best ones”) • General comments to all for future: • Keep an eye on the clock • Defer questions to end or offline if necessary • Plan for > 1 minute per slide

28. Reviews Tough work, but only way to ensure you remember main ideas in paper and your thoughts when you read it Please preserve your reviews! I hope you enjoyed writing them. If your complaint is about the large number of papers….

29. Reviews Tough work, but only way to ensure you remember main ideas in paper and your thoughts when you read it Please preserve your reviews! I hope you enjoyed writing them. If your complaint is about the large number of papers….you’re right

30. Next Semester • CS425 – Distributed Systems course (undergraduate) • New projects in Distributed Systems • Follow up of interesting projects from this semester • DPRG – Distributed Protocols Research Group • Weekly group meets next Fall (1 hour), to • discuss conference papers • explore new research ideas • explore collaborations with other research groups in department and university • get credit through CS 591IG enrolment • Subscribe to distsysl@cs.uiuc.edu to keep up to date

31. Articles

32. Articles for this Class • Roald Hoffman, “Why Buy that Theory” • C. M. Christensen, “How can great firms fail? Insights from the hard disk drive industry” • Levin and Redell, “How (and how not to) write a good SOSP paper” • R. P. Feynman, “The Chief Research Scientist…”

33. Why Buy that Theory A theory that explains an observable phenomenon • Occam’s Razor – “Plurality should not be assumed beyond necessity” The simplest explanation of a phenomenon is the best one • Is Portable: are lessons applicable to other areas? • Stimulates other Research: other people to work in the same / similar areas • Story telling matters: breaking the complex world down into simple and understandable parts (article taken from “Best American Science Writing, 2003”, Ed: J. Cohen)

34. Levin-Redell, Christensen • Levin and Redell, “How (and how not to) write a good SOSP paper” • original idea to a real problem • comprehensive and mature evaluation • chronological and logical presentation • C. M. Christensen, “How can great firms fail? Insights from the hard disk drive industry” • Disruptive technologies : Fig 1.7, page 17 (article taken from Innovator’s Dilemma, C. M. Christensen)

35. Feynman • R. P. Feynman, “The Chief Research Scientist of the Metaplast Corporation” • The wilder the idea, the better it is. But only as long as you keep working on it. (article taken from “Surely you’re joking, Mr. Feynman”, R.P. Feynman)

36. Questions?

37. Course Evaluations • Main purpose: to evaluate how useful this course was to you (and to improve future versions of the course) • I won’t see these evaluations until after you see your grades • Use pencil only • Please omit item 5 about student gender (top box) • Optional Instructor Questions (please write answers on reverse side) • Item E: Should the course projects remain open or be assigned like Machine Problems? • Item F: How much would the following help with respect to the project: (a) peer reviews (among class students), (b) poster presentations • Volunteer: please collect all reviews, and drop envelope in campus mail box