Deploying DIET and JuxMem: GoDIET JDF

1. Deploying DIET and JuxMem:GoDIET + JDF Mathieu Jan PARIS Research Group IRISA INRIA & ENS Cachan / Brittany Extension Rennes

2. 2 Conducting JXTA-based experiments:the JXTA Distributed Framework (JDF) A framework for automated testing of JXTA-based systems from a single node (control node) Original work from Sun Microsystems http://jdf.jxta.org/ JDF: several shell scripts Deployment Jar files and script used on each node Configuration of JXTA peers Launch peers Collect logs and results files of each node Analyze results on the control node Cleanup deployed and generated files Kill remaining processes Update resource

3. 3 How to define a test using JDF? An XML description file of the JXTA-based network Type of peers (rendezvous, edge peers) How peers are interconnected, etc A set of Java classes describing the behavior of each peer Extend the JDF’s framework (start, stop JXTA, etc) A Java class for analyzing collected results A file containing the list of nodes and the path of the JVM on each node

4. 4 Deploying a JuxMem network (1/2)

5. 5 Deploying a JuxMem network (2/2) <profile name=“clusterManagerA" instances="1"> … </profile> <profile name=“clusterManagerB" instances="1"> … </profile> <profile name=“clusterManagerC" instances="1"> … </profile> <profile name="providerA" instances=“42“ > <peer base-name="providerA" instances=“4“ /> <rdv cluster=“clusterManagerA"/> … </profile> <profile name="providerB" instances=“42“ > <peer base-name="providerC" instances=“5“ /> <rdv cluster=“clusterManagerB"/> … </profile> <profile name="providerC" instances=“35“ > <peer base-name="providerC" instances=“6“ /> <rdv cluster=“clusterManagerC"/> … </profile>

6. 6 Usage of JDF’s scripts runAll.sh [<flags>] <list-of-hosts> <network-descriptor> -debug: show all script commands executed -unsecure: use rsh instead of ssh -cleanup: cleanup JDF directory on each host -bundle: create bundle for distribution -install: install distribution bundle -update: update files on each peer -config: configure JXTA network -kill: kill existing JDF processes -run: run test -nohup: run and return without waiting for peers to exit -analyze: analyze test results -log: keep test results and log4j logs from peers -save: save JXTA platformConfig files -restore: restore JXTA platformConfig files

7. 7 Common architecture: DIET + JuxMem

8. 8 Common deployment: GoDIET + JDF 2 differents tools with different goals DIET = GoDIET JuxMem = JDF Idea for merging tools GoDIET reads the XML configuration file Create the test.xml and host.txt files for JDF Call JDF to configure and run the test In the correct order

9. 9 Modified DTD of GoDIET Idea: 2 different hierarchies The matching is done by the server tag <!ELEMENT deployment (diet_services, diet_hierarchy,juxmem_hierarchy?)> <!ELEMENT juxmem_hierarchy (manager+)> <!ELEMENT manager (client*|provider*)> <!ATTLIST manager server CDATA #REQUIRED port CDATA #IMPLIED> <!ELEMENT client EMPTY> <!ATTLIST client server CDATA #REQUIRED port CDATA #IMPLIED> <!ELEMENT provider EMPTY> <!ATTLIST provider server CDATA #REQUIRED port CDATA #IMPLIED>

10. 10 Launching DIET and JuxMem entities DIET entities MA LA SeD JuxMem entities Cluster managers (rdv peers) Providers / clients (edge peers) DIET + JuxMem MA / LA + cluster managers Providers SeD + JuxMem clients

11. 11 Example of a common deployment (1/2) Physical architecture 4 clusters: A, B, C, D 15 nodes in each cluster Logical architecture 1 MA on cluster A 4 LA, one on each cluster 4 cluster managers, one on each cluster 3 providers per manager 8 SeD + client JuxMem A DIET-JuxMem client needs to connect to 1 MA in order to use DIET 1 cluster manager in order to use JuxMem

12. 12 Example of a common deployment: DIET hierarchy <diet_hierarchy> <master_agent> <config server="clusterA-0" remote_binary="dietAgent"/> <local_agent> <config server="clusterA-0" remote_binary="dietAgent"/> <SeD><config server="clusterA-1" remote_binary="scalar_server"/></SeD> ...... <SeD><config server="clusterA-8" remote_binary="scalar_server"/></SeD> </local_agent> <local_agent> <config server="clusterB-0" remote_binary="dietAgent"/> <SeD><config server="clusterB-1" remote_binary="scalar_server"/></SeD> ...... <SeD><config server="clusterB-8" remote_binary="scalar_server"/></SeD> </local_agent> …. </master_agent> </diet_hierarchy>

13. 13 Example of a common deployment: JuxMem hierarchy <juxmem_hierarchy> <manager server="clusterA-0" /> <client server="clusterA-1, . . . ,clusterA-8" /> <provider server="clusterA-9, . . . ,clusterA-11" /> </manager> <manager server="clusterB-0" /> <client server="clusterB-1, . . . ,clusterB-8" /> <provider server="clusterB-9, . . . ,clusterB-11" /> </manager> <manager server="clusterC-0" /> <client server="clusterC-1, . . . ,clusterC-8" /> <provider server="clusterC-9, . . . ,clusterC-11" /> </manager> <manager server="clusterD-0" /> <client server="clusterD-1, . . . ,clusterD-8" /> <provider server="clusterD-9, . . . ,clusterD-11" /> </manager> </juxmem_hierarchy>

14. 14 Required modifications in GoDIET Handling new XML tags Manager, client, provider Writing the correct test.xml and host.txt JDF files No need to write a files.txt New commands in GoDIET shell Deploy JuxMem bundle Cleanup files, update files Retrieve log files and results files Modified launch command += -config -run

15. 15 Work schedule for GoDIET + JDF Modified DTD of GoDIET Do we agree ? Handling new XML tags Creating the correct test.xml and host.txt files New GoDIET shell commands => Holly and / or Mathieu ?

16. 16 Work schedule for DIET + JuxMem Deploying JXTA-C peers using JDF Almost done JuxMem client using JXTA-C Started (July) DIET using JuxMem clients code Using API given by JuxMem (August-September)

17. 17 Conclusion Common architecture for DIET and JuxMem A SeD is a client JuxMem Common deployment using GoDIET and JDF Required step before a first DIET-JuxMem prototype Work schedule on GoDIET + JDF DIET + JuxMem