1 / 13

Deploying Gfarm and JXTA-based applications using the ADAGE deployment tool

Deploying Gfarm and JXTA-based applications using the ADAGE deployment tool. Landry Breuil, Loïc Cudennec and Christian Perez IRISA / INRIA, PARIS project-team. Motivations (1/2) Offering persistent storage to a grid data sharing service. Data sharing for grid-based applications

devaki
Download Presentation

Deploying Gfarm and JXTA-based applications using the ADAGE deployment tool

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Deploying Gfarm and JXTA-based applicationsusing the ADAGE deployment tool Landry Breuil, Loïc Cudennec and Christian Perez IRISA / INRIA, PARIS project-team

  2. Motivations (1/2)Offering persistent storage to a grid data sharing service • Data sharing for grid-based applications • Numerical simulations, collaborative design, distributed databases, code-coupled applications • Desirable features: transparency, consistency, uniform access, persistency • Grid data sharing service (JuxMem, INRIA/IRISA) • Transparent access to replicated data • Fault tolerance and consistency mechanisms • Data stored in physical memory • Global distributed file system (Gfarm, AIST/Univ.Tsukuba) • File fragmentation and replication • Smart replica selection • Data stored on disk: secondary persistent storage

  3. Motivations (2/2)Proposition of common architecture (#1 out of 3 propositions) • One particular JuxMem provider (GDG leader) flushes data to Gfarm • Then, other Gfarm copies can be created using Gfarm’s gfrep command JuxMem Global Data Group (GDG) JuxMem Provider GDG Leader JuxMem Provider JuxMem Provider GFSD GFSD GFSD GFSD GFarm Cluster #1 Cluster #2

  4. Towards a mixed deploymentUsing the ADAGE deployment tool • Deploying the Gfarm file system • 4 roles: metadata server, metadata cache server, file system node and client node • Dependencies between roles: • Tree-based • a role should start after its father • Deploying the JuxMem service (JXTA-based application) • 2 types of peers: rendezvous and edges • Dependencies between peers: • Tree-based • a peer should start after its rendezvous peer • Deployment constraints • Gfarm should be launched before JuxMem • The Gfarm client should share the same node than the JuxMem provider • Generic deployment tool ADAGE (INRIA/IRISA)

  5. Gfarm file system (1/2)Tree-based dependencies Configuration file Server IP, tcp port server Server IP, tcp port Cache IP, tcp port cache cache Server IP, tcp port Cache IP, tcpport client client storage storage client storage

  6. Gfarm file system (2/2)Describing the application <server name="metadata_server" port_master="10602" port_gfmd="10601" port_gfsd="10600" /> <procs> <proc name="metadata_cache_server" role="agent" cardinality="1" /> <proc name="fs_node" role="gfsd" cardinality="1" agent="metadata_cache_server" /> <proc name="gf_client" role="client" binary="true" cardinality="1" agent="metadata_cache_server" /> </procs> Defining tcp ports for Gfarm roles Naming this metadata cache server Defining which cache server to connect with Binary to execute on the client node

  7. JuxMem (1/3)is a JXTA-based application edge PlatformConfig file: tcp port, IP and tcp port of seed rdv rdv rdv edge edge edge edge

  8. JuxMem (2/3)Describing the JXTA application <profiles> <profile name="p_manager"> <services rdv="true" relay="false" /> <networks tcp="true" port="9701" /> <behavior binding="c" filename="simple_manager" args="120"/> </profile> </profiles> <overlay> <rdvs> <rdv id="manager" profile_name="p_manager" cardinality="1"> </rdv> </rdvs> <edges> <edge id="provider" profile_name="p_provider » cardinality="1" rdv="manager" /> <edge id="writer" profile_name="p_writer » cardinality="1" rdv="manager" /> <edge id="reader" profile_name="p_reader » cardinality="1" rdv="manager" /> </edges> </overlay> Defining a profile for the manager Rendezvous instanciation Edge instanciation Seed name Langage: JDL (Mathieu Jan)

  9. JuxMem (3/3)Deploying step-by-step Writing PlatformConfig Waiting peer up node1 Rdv: 9701 Callback=9701 node2 Topology rdv rdv ADAGE edge1 edge3 edge2

  10. JuxMem (3/3)Deploying step-by-step, dynamic tcp ports assignation node1 Rdv: 9701 Edge2: 9703 Edge1: 9701 Edge3: 9705 Callback=9703 Edge1(seed=node1:9701) Edge3(seed=node1:9701) node2 Topology Edge2(seed=node1:9701) rdv Callback=9701 Callback=9705 ADAGE edge1 edge3 edge2

  11. cluster group Meta plugin (1/2)Mixed Architecture JuxMem manager JuxMem Client JuxMem Provider GFarm client GFarm file system node GFarm server GFarm agent

  12. Meta plugin (2/2)Describing the mixed architecture <specific id="gfarm-desc » path="./tests/gfarm-appl.xml"/> <specific id="juxmem-desc » path="./tests/jxta-appl.xml"/> <dependency deptype="acked" what="provider"> <onwhat>gf_client_0</onwhat> </dependency> Pointing to the Gfarm description Pointing to the JuxMem description Expressing ack dependency <submission_method name="ssh"/> <planner mode="random"/> <transfer_files method="scp"/> <plugin name="JXTA » file="tests/meta-gfarm-jxta-ctrl-params-spec.xml"/> <placement_constraints> <collocate id="gf_client_0" with="provider"/> </placement_constraints> Expressing colocation

  13. Towards a mixed deploymentUsing the ADAGE deployment tool • Some ADAGE features used by these plugins • File transfer • Wait for file creation • Callback mechanism • Some experimentations • Deploying up to 40 Gfarm roles • Adapting the JuxMem basic tutorial to ADAGE • Future work • Using large scale configurations http://adage.gforge.inria.fr/wiki

More Related