Middleware Lab SS 2008 Introduction & Organization

1. Middleware Lab SS 2008Introduction & Organization Christof Leng, Wesley Terpstra Databases and Distributed Systems Group Department of Computer Science Technische Universität Darmstadt

2. Contents • Introduction • Organization • Objectives • Deliverables • Marking Scheme • Schedule • Form groups After a short break: • Tasks for first project phase

3. What is Peer-to-Peer? • “A peer-to-peer (or P2P) computer network relies primarily on the computing power and bandwidth of the participants in the network rather than concentrating it in a relatively low number of servers. P2P networks are typically used for connecting nodes via largely ad hoc connections. Such networks are useful for many purposes.” (from Wikipedia) • P2P networks are cheap and typically easy to set up • P2P networks may scale to millions of nodes • Nodes come and go and may even fail without interrupting the system’s operation • The nodes form an overlay network eMule Groove

4. Unstructured Overlays Most practical implementations E.g. Gnutella, FastTrack, Skype Not all are well engineered Limited scalability Bad for ID lookups Good for complex queries Structured Overlays Popular among academia E.g. Chord, CAN, Pastry, Kademlia, P-Grid Mostly distributed hash tables (DHTs) Not all are suited for the real world Excellent for ID lookups Bad for complex queries Structured vs. Unstructured Overlays

5. Project • Implement the Bit Zipper Rendezvous in FreePastry • Use Bit Zipper for full-text search on Wikipedia data • What is Bit Zipper? • A simple approach to solve complex queries with DHTs • Combines ideas of structured and unstructured overlays • More information after the break…

6. What you will learn • How to develop a distributed application • (hopefully one interesting for you) • The ideas behind: • P2P applications • State-of-the-art distributed search • Full-text search • Experience with: • Networking • Debugging and documenting • Working with external libraries (esp. FreePastry) • Work as a team • Present and write in English (optional)

7. What we expect • A working and useful framework • Report, source documentation, presentations • Participation in programming, documentation, and presentations • Everyone in the group! • Approximately 160h of work (=P4) • That you ask for help if there are problems

8. Organization • Teams of 3 students • Every team builds its own implementation • Project divided into 7 phases • Each phase is 2 weeks in duration • A meeting after every project phase

9. Mark Breakdown • 40% Implementation + JavaDoc • Intermediate results count! • Coverage • Design • Correctness • Documentation • 30% Meetings + Presentations • Participation • Presentation (clarity and completeness) • Slides • 30% Lab Report • Overview of implementation and user manual • Documentation of development process • 10-15 pages total • Work-in-progress has to be delivered on each meeting • PDF or HTML

10. Meetings • Students deliver the lab report sketch • Each group presents • Current state of implementation (live demo!) • Approach taken • Progress • Problems • 4-5 slides(as PDF and delivered midnight before the meeting date) • We present • The outline of the next phase • Relevant background theory • We answer your questions

11. Infrastructure • Course Homepage • http://www.dvs.tu-darmstadt.de/teaching/mwlab/2008/ • Subversion access provided for source code • https://www.dvs.tu-darmstadt.de/svn/lectures/mwlab/ss08/<groupname>/ • Use Subclipse Plugin in JavaSVN mode • Use Mailing list • mwlab2008@dvs.tu-darmstadt.de • Our lab is available 24/7 • Transponders are available from RBG service center C119

12. Project Schedule 15.04. (1) Orientation, Reading 29.04. (2) Bit Zipper Routing 13.05. (3) Index Maintenance 27.05. (4) Index Updates 10.06. (5) Full-text search 24.06. (6) Large-scale testing 08.07. (7) Performance Evaluation 22.07. (8) Final demonstration (05.08. Backup phase) April Summer Term (Vorlesungszeit) May June July August

13. The Sorting Hat • Choose your house!