Large-scale Deployment in P2P Experiments Using the JXTA Distributed Framework

1. Large-scale Deployment in P2P Experiments Using the JXTA Distributed Framework Gabriel Antoniu, Luc Bougé, Mathieu Jan & Sébastien Monnet PARIS Research Group IRISA / INRIA & University of Rennes I & ENS Cachan, France Euro-Par 2004 Pisa, 3rd September

2. How to reproduce and test P2P systems? • Peer-to-Peer (P2P) systems • Very large-scale • Volatile peers • Heterogeneous architecture • Extremely difficult to reproduce and analyze! • Solutions used for testing P2P systems • Simulation • Emulation • Tests on real testbeds

3. Solutions used for testing P2P prototypes • Experiments on real testbed or via emulation • Crucial step when validating software!

4. Testing P2P systems: a difficult problem! • Papers on P2P systems • Lack of experiments • Mostly simulation • Real experiments using up to a few ten of physical nodes • Large-scale (thousands of peers) via emulation • The methodology for testing is not discussed • Deployment? Volatility? • Technical infrastructures to support testing activities are needed! • Allow developers to focus on interesting problems

5. “The five commandments of P2P experiments”

6. Commandment C1 Thou shalt easily specify the requested virtual network of peers • What is needed? • Type of peers (application-specific) • How they are interconnected • Which communication protocols they should use • Where they should de deployed • When they should be bootstrapped • etc

7. Commandment C2 Thou shalt enable designers of P2P prototypes to trace the behavior of their system • What is needed? • Retrieve outputs of each peer • Log files, result files, etc • Store them on the control node of the experiment • Conflicting names

8. Commandment C3 Thou shalt efficiently deploy peers on a large number of physical nodes • What is needed for grid deployment ? • Hierarchical deployment on a federation of distributed clusters • Handling batch systems: PBS, Sun Grid Engine, etc

9. Commandment C4 Thou shalt allow peers to synchronize between stages of a test • What is needed? • A peer should have the possibility to wait for other peers • Support complex tests consisting of several stages

10. Commandment C5 Thou shalt control the simulation of peers’ volatility • What is needed? • Artificially enforce various volatile behaviors • The testbed is assumed to be stable • Simulate correlated failures, network partitions • Take into account the status of the peer to play the role of the enemy

11. Example: deploying JXTA • JXTA: open-source framework for programming P2P applications • Specify a set of protocols • Different type of peers • Edge peers • Rendezvous peers • Relay peers Edge Edge Rdv Edge Edge Edge Rdv Rdv Rdv Rdv Edge Edge Rdv Edge Edge

12. Conducting JXTA-based experiments:the JXTA Distributed Framework (JDF)

13. The JXTA Distributed Framework (JDF) • A framework for automated testing of JXTA-based systems from a single node (control node) • Started by Sun Microsystems • http://jdf.jxta.org • Hypothesis • All the nodes must be “visible” by the control node • Requirements • Java Virtual Machine • Bourne shell • ssh/rsh configured to run with no password on each node • Set of shell scripts • Deploy, configure, run, log, analyze, kill, cleanup, update, etc

14. Required JDF files to specify a distributed test • Network description file • Defines the requested JXTA-based network • Notion of peer profile • Set of Java classes extending the JDF framework • Behavior of each peer • Analyze the results • Node file • List of physical nodes • Path of the JVM on each physical node • Package file • List of jar files to deploy on each physical node

15. The JDF specification language (C1) <network analyze-class=“prototype.test.Analyze”> <profile name=“Rendezvous”> <peer instances=“1”/> <rdvs is-rdv=“true”/> <transports> <tcp base-port=“13000”/> <transports/> <bootstrap class=“prototype.test.Rendezvous”/> </profile> <profile name=“Edge”> <peer instances=“3”/> <rdvs is-rdv=“false”> <rdv profile=“Rendezvous”/> <rdvs/> <transports> <tcp base-port=“13000”/> <transports/> <bootstrap class=“prototype.test.Edge”/> </profile> </network>

16. A basic Java test class inside JDF’s framework public class Edge extends JxtaBootStrapper { public static void main(String[] args) { Edge peer = new Edge(); peer.start(args); peer.stop(); } public void start(String[] args) { // Start the test on the local peer super.startJxta(); p2pService = group.lookupService(...); p2pService.doSomething(); } public void stop() { // Stop the test on the local peer p2pService.stopApp(); super.stopJxta(); } protected void updateProperties() { // Store the local results super.updateProperties(); setProperty(PROPERTY_TAG, property_result); }

17. Does JDF observe the 5 commandments? • JDF specification language (C1) • Is it enough? • Collecting log and result files (C2) • And that’s all!

18. Our contribution to the JXTA Distributed Framework (JDF)

19. Improving JDF: a more concise specification language (C1) • Original JDF requires 1 profile for each physical node • Not scalable! • Enhanced network description file • Ability to specify that a single profile can be shared by multiple physical nodes <profile name=“RendezVous1” replicas=“1”/> <profile name=“RendezVous2” replicas=“1”/> <profile name=“Edge1” replicas=“9”/> <profile name=“Edge2” replicas=“9”/>

20. Improving JDF: a more concise specification language (C1) • Original JDF requires 1 profile for each physical node • Not scalable! • Enhanced network description file • Ability to specify that a single profile can be shared by multiple physical nodes <profile name=“RendezVous1” replicas=“1”/> <profile name=“RendezVous2” replicas=“1”/> <profile name=“Edge1” replicas=“99”/> <profile name=“Edge2” replicas=“99”/>

21. Improving JDF: interacting with grid resource management systems (C2) • Original JDF CAN’T use clusters managed by batch systems • Many clusters are managed via a batch system • Improvement • JDF now supports PBS and Globus • The node file is dynamically created • Possible future extension • How to deal with co-scheduling jobs across multiple batch systems • Beyond the scope of JDF-like tools

22. Improving JDF: controlling the simulation of volatility (C5) • High volatility • Most significant feature of P2P systems • Improvement • Basic support of failure injection in JDF • Pre-computed failure control file • Global MTBF of the network peers • Possible future extensions • More complex failure injection mechanisms • Correlated failures? • Network partitions?

23. Using JDF with JXTA-based projects • Several projects • JXTA’s benchmarks (part of JDF project) • Deploy Meteor peers • Chord-like system above JXTA • Our JXTA-based service: JuxMem • Data sharing service for grid computing environments • Compromise between DSM systems and P2P systems • Transparent access to mutable data via a global ID • Use of JDF in JuxMem • Tests • Benchmarks

24. Preliminary results: deployment, configuration and update JXTA and its dependancies =~ 4MB

25. Preliminary results: various volatility conditions Global MTBF of 1 minute

26. Conclusion • Testing P2P systems is difficult! • We state “The five commandments of P2P experiments” • We enhanced JDF to fulfill some commandments • More precise and concise specification language (C1) • Handling batch systems (C3) • Basic control of the volatility conditions (C5) ? Are we missing other commandments?

27. Ongoing work: fully observe the 5 commandments • Enhancement to the specification language (C1) • We can always do better, so … • Hierarchical deployment (C3) • Like Ka-run/Taktuk (ID IMAG, France) • Distributed synchronization mechanism (C4) • Support more complex tests • Improved volatility control (C5) • Part of Sébastien Monnet’s PhD (IRISA, Rennes)