1 / 23

4.1 Environment

4.1 Environment. Deploying. ProActive Parallel Suite (1). ProActive Parallel Suite (1). Large Equipment. Internet. Internet. Internet. Internet. Servlets. EJBs. Databases. Job management for embarrassingly parallel application (e.g. SETI). Parallel Machine. Clusters.

taite
Download Presentation

4.1 Environment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 4.1 Environment Deploying Denis Caromel

  2. ProActive Parallel Suite (1) Denis Caromel

  3. ProActive Parallel Suite (1) Denis Caromel

  4. Large Equipment Internet Internet Internet Internet Servlets EJBs Databases Job management for embarrassingly parallel application (e.g. SETI) Parallel Machine Clusters How to deploy on the Various Kinds of Grid Denis Caromel

  5. 4.1 Abstract Deployment Model • Problem: • Difficulties and lack of flexibility in deployment • Avoid scripting for: configuration, getting nodes, connecting, etc. • A key principle: Virtual Node (VN) + XML deployment file • Abstract Away from source code: • Machines • Creation Protocols • Lookup and Registry Protocols • Protocols and infrastructures: • Globus, ssh, rsh, LSF, PBS, … Web Services, WSRF, ... Denis Caromel

  6. Deployment Descriptor Virtual architecture ActivateMapping(«  VN1 ») ActivateMapping(«  VN2») Abstract deployment model • Separates design from deployment infrastructure • Virtual nodes • Dynamic enactement of the deployment, from the application Source code Runtime Physical infrastructure • Host names • Creation protocols • of • JVMs, • Lookup, • registration computer2 JVM2 VN1 computer3 JVM3 VN2 JVM4 JVM5 Denis Caromel

  7. Context Grid • Scalable • Heterogeneous resources • OS, CPU, Memory • Security policies: firewall, NAT, private IP addresses,… • Model,... Painless deployment • Based on well-known technologies • Java, XML • Abstract Deployment model Denis Caromel

  8. Abstract Deployment Model Problem: • Difficulties and lack of flexibility in deployment • Avoid scripting for: configuration, getting nodes, connecting, etc. A key principle: Virtual Node (VN) in XML deployment file • Abstract Away from source code: • Machines names • Creation/Connection Protocols • Lookup and Registry Protocols • Interface with various protocols and infrastructures: • Cluster: LSF, PBS, SGE , OAR and PRUN(custom protocols) • Intranet P2P, LAN: intranet protocols: rsh, rlogin, ssh • Grid: Globus, Web services, ssh, gsissh Denis Caromel

  9. XML Deployment files Virtual Node (VN): • Identified as a string name • Used in program source • Configured (mapped) in the XML descriptor file --> Nodes Operations specified in descriptors: • Mapping of VN to JVMs (leads to Node in a JVM on Host) • Register or Lookup VNs • Create or Acquire JVMs • Security Program Source Descriptor (RunTime) |-------------------------------------------| |----------------------------------------| Activities (AO)--> VN--> Nodes VN --> JVMs --> Hosts Runtime structured entities: 1 VN --> n Nodes in m JVMs on k Hosts Denis Caromel

  10. Descriptors: Virtual Nodes in Programs • Descriptor pad = ProActive.getDescriptor("file:DeploymentDescriptor.xml"); • VirtualNode vn = pad.activateMapping ("Dispatcher"); • // Triggers the JVMs and Nodes creation • Node node = vn.getNode(); • C3D c3d = ProActive.newActive("C3D", param, node); • log ( ... "created at: " + node.name() + node.JVM() + node.host() ); Denis Caromel

  11. Descriptors: Virtual Nodes in Programs • Descriptor pad = ProActive.getDescriptor("file:DeploymentDescriptor.xml"); • VirtualNode vn = pad.activateMapping ("Dispatcher"); • // Triggers the JVMs and Nodes creation • Node node = vn.getNode(); • C3D c3d = ProActive.newActive("C3D", param, node); • log ( ... "created at: " + node.name() + node.JVM() + node.host() ); • // Cyclic mapping: set of nodes • VirtualNode vn = pad.activateMapping ("RendererSet"); • while ( … vn.getNbNodes … ) { • Node node = vn.getNode(); • Renderer re = ProActive.newActive(”Renderer", param, node); Denis Caromel

  12. Descriptors: Mapping Virtual Nodes • Component Dependencies: • Provides: … Uses: ... • VirtualNodes: • Dispatcher <RegisterIn RMIregistry, Globus, Grid Service, … > • RendererSet • Mapping: • Dispatcher --> DispatcherJVM • RendererSet --> JVMset • JVMs: • DispatcherJVM = Current // (the current JVM) • JVMset=//ClusterSophia.inria.fr/ <Protocol GlobusGram … 10 > • ... Example of an XML file descriptor: Denis Caromel

  13. XML Deployment (Not in source) VNa VNb VNc = VN(a,b) C C A B A B Separate or Co-allocation Denis Caromel

  14. Model and Tools for Grid Deployment Abstract away machines, creation, registry, lookup protocols Use only Virtual Nodes in the source code Describe mapping of virtual nodes in XML Deployment Descriptors Interfaced with various protocols for creation/lookup: rsh,ssh,Jini, LSF,PBS,Globus,OGSA... VirtualNodes: Dispatcher <RegisterIn RMIregistry, Globus, Grid Service, …> RendererSet Mapping: Dispatcher --> DispatcherJVM RendererSet --> JVMset JVMs: DispatcherJVM = <javapath …/> <classpath…/> sshProcess JVMset = <javapath …/> <classpath…/> GlobusProcess Processes: sshProcess = <hostname di.unice.fr/> <login../> GlobusProcess = <hostname cluster.inria.fr/> <gram port 2119/> <node 10/> Other option: acquire ProActive runtimes from voluntary PCs (P2P computing) running a ProActive P2P infrastructure Unification of various deployment systems through ProActive runtimes Denis Caromel

  15. Mapping Virtual Nodes: example (1) Definition of Virtual Nodes • <virtualNodesDefinition> • <virtualNode name="Dispatcher"/> • </virtualNodesDefinition> • <map virtualNode="Dispatcher"> • <jvmSet> • <vmName value="Jvm1"/> • </jvmSet> • </map> • <jvm name="Jvm1"> • <acquisition method="rmi"/> • <creation> • <processReference refid="linuxJVM"/> • </creation> • </jvm> Definitions and mapping Mapping of Virtual Nodes Denis Caromel

  16. Mapping Virtual Nodes: example (2) • <virtualNodesDefinition> • <virtualNode name="Jem3DNode"/> • </virtualNodesDefinition> • <map virtualNode=" Jem3DNode"> • <jvmSet> • <vmName value=”clusterJvm"/> • </jvmSet> • </map> • <jvm name="clusterJvm"> • <acquisition method="rmi"/> • <creation> • <processReference refid=”clusterProcess"/> • </creation> • </jvm> Definition of Virtual Nodes Definitions and mapping Mapping of Virtual Nodes Denis Caromel

  17. JVM on the current Host JVM started using RSH Mapping Virtual Nodes: example (3) • <processDefinition id="linuxJVM"> • <jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/> • </processDefinition> • <processDefinition id="rshProcess"> • <rshProcess class="org.objectweb.proactive.core.process.rsh.RSHJVMProcess" • hostname="sea.inria.fr"> • <processReference refid="linuxJVM"/> • </rshProcess> • </processDefinition> Infrastructure informations Denis Caromel

  18. Definition of bsub process Mapping Virtual Nodes: example (4) • <processDefinition id=”singleJVM"> • <jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/> • </processDefinition> • <processDefinition id=" clusterProcess"> • <bsubProcess class="org.objectweb.proactive.core.process.lsf.LSFBSubProcess" • hostname=”cluster.inria.fr"> • <processReference refid=”singleJVM"/> • <bsubOption> • <processor>12</processor> • </bsubOption> • </bsubProcess> • </processDefinition> Infrastructure information Denis Caromel

  19. Definition of LSF deployment, … Globus Mapping Virtual Nodes: example (5) • <processDefinition id=" clusterProcess"> • <bsubProcess class="org.objectweb.proactive.core.process.lsf.LSFBSubProcess" • hostname=”cluster.inria.fr"> • <processReference refid=”singleJVM"/> • <bsubOption> • <processor>12</processor> • </bsubOption> • </bsubProcess> • </processDefinition> Infrastructure information Denis Caromel

  20. JVM on the current Host JVM started using SSH Mapping Virtual Nodes: example (6) • <processDefinition id="linuxJVM"> • <jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/> • </processDefinition> • <processDefinition id=”sshProcess"> • <sshProcess class="org.objectweb.proactive.core.process.ssh.SSHJVMProcess" • hostname="sea.inria.fr"> • <processReference refid="linuxJVM"/> • </sshProcess> • </processDefinition> Infrastructure information Denis Caromel

  21. Definition of SGE process JVM created on the remote Hosts Mapping Virtual Nodes: example (7) • <processDefinition id=" plugtestProcess"> • <sgeProcess class="org.objectweb.proactive.core.process.sge.SGEProcess” hostname=” frontal"> • <processReference refid=”localJVMprocess"/> • <sgeOptions> • <processor>12</processor> • </sgeOptions> • </sgeProcess> • </processDefinition> Infrastructure information <processDefinition id=”localJVMprocess"> <jvmProcess class="org.objectweb.proactive.core.process. JVMNodeProcess"/> </processDefinition> Denis Caromel

  22. Mapping Virtual Nodes (8): Mixed Protocol • <processDefinition id=”plugtestProcess"> • <sshProcess class="org.objectweb.proactive.core.process.ssh.SSHProcess" • hostname="sea.inria.fr"> • <processReference refid=”clusterProcess"/> • </sshProcess> • </processDefinition> • <processDefinition id=" clusterProcess"> • <pbsProcess class="org.objectweb.proactive.core.process.pbs.PBSProcess” • <processReference refid=”localJVMprocess"/> • ... • </pbsProcess> • </processDefinition> Infrastructure information <processDefinition id=”localJVMprocess"> <jvmProcess class="org.objectweb.proactive.core. process.JVMNodeProcess"/> </processDefinition> Denis Caromel

  23. Same application, many deployments Internet • User constraints can be considered, but are manually expressed in deployment descriptors • Can interface with meta-Grid Schedulers / mappers Local Grid Distributed Grids One Host Denis Caromel

More Related