1 / 84

Denis Caromel, et al. ProActive.ObjectWeb OASIS Team

Open Source Middleware for Grid and Parallelism:. Denis Caromel, et al. http://ProActive.ObjectWeb.org OASIS Team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis, IUF. GRIDs and Parallelism Active Objects + Groups Components Deployment GUI Applications (BLAST).

kieve
Download Presentation

Denis Caromel, et al. ProActive.ObjectWeb OASIS Team

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Open Source Middleware for Grid and Parallelism: Denis Caromel, et al. http://ProActive.ObjectWeb.org OASIS Team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis, IUF • GRIDs and Parallelism • Active Objects + Groups • Components • Deployment • GUI • Applications (BLAST) Denis Caromel

  2. ProActive Parallel Suite (1) Open Source + PROFESSIONAL SUPPORT Denis Caromel

  3. 1. The GRIDs PCs : 1 Billion in 2002 (25 years) Forecast: 2 Billions in 2008 Denis Caromel

  4. The Grid Concept • GRID = Electric Network • Computer Power (CPU cycles) Electricity • Can hardly be stored, if not used Lost • Global management, • Mutual sharing of the resource 2 important aspects : 1. Diversity 2. Computational + Data Grid Denis Caromel

  5. Enterprise Grids Internet Apache Servlets EJB Databases 5 Denis Caromel

  6. Scientific Grids Large Equipment Internet Parallel Machine Clusters 6 Denis Caromel

  7. Internet Grids Internet Job management for embarrassingly parallel application (e.g. SETI) 7 Denis Caromel

  8. Intranet Grids - Desktop Grids Internet Using Spare Cycles of Desktops Potentially: - Non-embarrassingly parallel application - Across several sites How to Program ? How to Deploy ? 8 Denis Caromel

  9. The multiple GRIDs Scientific Grids : Enterprise Grids : Internet Grids, (miscalled P2P grid): Intranet Desktop Grids Strong convergence needed and somehow in progress! 2 New Disturbing Factors: End of Moore Law and Multi-Cores For SBGrid: The Multiple Scientific GRIDs 2D and 3D graphics SB workstations EM laboratory Clusters Global grids and other federally funded equipments 9 Denis Caromel

  10. Grid Computing with ProActive Shanghai Nice Boston Hierarchical Deployment Challenges: Programming Model, Scale, Latency, Heterogeneity, Versatility (FT, protocols, firewalls, etc.) Beijing 10 Denis Caromel

  11. 2. Distributed and ParallelObjectsProActiveProgramming 11 Denis Caromel

  12. ProActive Parallel Suite Denis Caromel

  13. ProActive Parallel Suite Denis Caromel

  14. ProActive : Active objects JVM A A WBN! A ag =newActive (“A”, […], VirtualNode) V v1 = ag.foo (param); V v2 = ag.bar (param); ... v1.bar(); //Wait-By-Necessity JVM ag v2 v1 V Wait-By-Necessity is a Dataflow Synchronization Java Object Active Object Req. Queue Future Object Proxy Thread Request 14 Denis Caromel

  15. ProActive: Explicit Synchronizations Explicit Synchronization: - ProActive.isAwaited (v); // Test if available - .waitFor (v); // Wait until availab. Vectors of Futures: - .waitForAll (Vector); // Wait All - .waitForAny (Vector); // Get First A ag =newActive (“A”, […], VirtualNode) V v = ag.foo(param); ... v.bar(); //Wait-by-necessity 15 Denis Caromel

  16. Wait-By-Necessity: First Class Futures V= b.bar () b v v Futures are Global Single-Assignment Variables a b c.gee (V) c c 16 Denis Caromel

  17. Proofs in GREEK Denis Caromel

  18. Standard system at Runtime:No Sharing NoC: Network On Chip 18 Denis Caromel

  19. TYPED ASYNCHRONOUS GROUPS 19 Denis Caromel

  20. Creating AO and Groups A V A ag =newActiveGroup (“A”, […], VirtualNode) V v = ag.foo(param); ... v.bar(); //Wait-by-necessity JVM Group, Type, and Asynchrony are crucial for Cpt. and GRID Typed Group Java or Active Object 20 Denis Caromel

  21. Broadcast and Scatter ag JVM c3 c3 c3 c3 c3 c3 c3 c1 c1 c1 c1 c1 c1 c1 c2 c2 c2 c2 c2 c2 c2 JVM JVM s s JVM • Broadcast is the default behavior • Use a group as parameter, Scattered depends on rankings cg ag.bar(cg); // broadcast cg ProActive.setScatterGroup(cg); ag.bar(cg); // scatter cg 21 Denis Caromel

  22. Dynamic Dispatch Group c4 c4 c4 c6 c6 c6 c5 c5 c5 c7 c7 c7 c8 c8 c8 c0 c0 c0 c9 c9 c9 c3 c3 c3 c1 c1 c1 c2 c2 c2 JVM Slowest ag cg JVM Fastest JVM ag.bar(cg); JVM 22 Denis Caromel

  23. ProActive Parallel Suite Denis Caromel

  24. ProActive Parallel Suite Denis Caromel

  25. Parallel, Distributed, Hierarchical3. Componentsfor the GridComposing, Wrapping code to make them fit for the Grid 25 Denis Caromel

  26. A Provide/Require Component Component interface My Business Component Interface Reference OFFERED REQUIRED Event sinks Event sources Attributes Courtesy of Philippe Merle, Lille, OpenCCM platform 26 Denis Caromel

  27. Building Component Applications =Assembling Components with XML ADL Provide + Use, but flat assembly 27 Denis Caromel

  28. Objects to Distributed Components (1) ComponentIdentity Cpt =newActiveComponent (params); A a = Cpt … .getFcInterface ("interfaceName"); V v = a.foo(param); IoC: Inversion Of Control (set in XML) A Example of component instance V Truly Distributed Components Typed Group Java or Active Object JVM 28 Denis Caromel

  29. P A B A A C B D C Group proxy Group proxy Groups in Components (1) A parallel component! Broadcast at binding, on client interface At composition, on composite inner server interface 29 Denis Caromel

  30. GCM:Grid Component Model A Strong Grid Standard with ETSI: European Telecommunications Standards Institute Denis Caromel

  31. GCM Scopes and Objectives: Grid Codes that Compose and Deploy No programming, No Scripting, … No Pain Innovation: Abstract Deployment Composite Components Multicast and GatherCast MultiCast GatherCast

  32. GCM features provided in ProActive: • GCM Fractal ADL (XML) • (Architecture Description Language) • GCM Management (Java, C, WSDL API) • GCM Application Description (XML) • GCM Interoperability Deployment (XML) Denis Caromel

  33. Ho to use GCM & Components for SBGrid ? • Description of a piece of code to make it fit for the Grid (& scheduling): • Required: Hardware, OS, Configuration • Input needed • Output Produced • Optional, on a case by case basis: • Wrapping the code in a portable manner, adding necessary control • Description of Services Offered, and Required (CP, + Services) • Making it possible to compose SBGrid software Denis Caromel

  34. ProActive Parallel Suite Denis Caromel

  35. ProActive Parallel Suite Denis Caromel

  36. ProActive Parallel Suite (1) Denis Caromel

  37. ProActive Parallel Suite (1) Denis Caromel

  38. 4. Deployment Deploying Denis Caromel

  39. Large Equipment Internet Internet Internet Internet Servlets EJBs Databases Job management for embarrassingly parallel application (e.g. SETI) Parallel Machine Clusters How to deploy on the Various Kinds of Grid Denis Caromel

  40. Abstract Deployment Model • Problem: • Difficulties and lack of flexibility in deployment • Avoid scripting for: configuration, getting nodes, connecting, etc. • A key principle: Virtual Node (VN) + XML deployment file • Abstract Away from source code, and Wrapping code: • Machines • Creation Protocols • Lookup and Registry Protocols • Protocols and infrastructures: • Globus, ssh, rsh, LSF, PBS, … Web Services, WSRF, ... Denis Caromel

  41. GCM XML Deployment Descriptor Virtual architecture Abstract deployment model • Separates design from deployment infrastructure • Virtual Nodes (VN) • Dynamic enactment of the deployment, from the application Source code Runtime Physical infrastructure computer2 Host names Creation protocols Lookup, Registration Multi-Cores JVM2 VN1 computer3 JVM3 VN2 JVM4 JVM5 Denis Caromel

  42. Data management: File transfer • Integrated with Deployment: • Resource acquisition file transfer when available: unicore, nordugrid, Globus • Other non-deployment protocols: scp, rcp, … • ProActive own protocol when other fails: ProActive Failsafe File Transfer • ProActive supports the following transfers: • Push: To a remote node • Pull: From a remote node • Triangle: Triggered from node A, occurring between node B  C • It can be used at: • Deployment time (XML) • Execution at any time (API) • Retrieval time (XML) Denis Caromel

  43. File Transfer in Deployment Descriptors Denis Caromel

  44. API: Asynchronous File Transfer with Futures • Immediately returns a File Future to the caller (subject to Wait-By-Necessity) • VN RETRIEVE: pullFile is invoked, and an array of futures (File[]) is returned • Futures: increase the performance when transferring files between peers: • Outperforms rcp when Pushing while Pulling a file! Denis Caromel

  45. Performance Comparison100Mbit LAN network with a 0.25[ms] ping,network configured at 10[Mbitsec] duplex • First, ProActive failsafe and native scp perform the same • Second, ProActive deploys the files in parallel to the nodes, faster than scp: • several invocations of push, on a set of nodes, • are executed in parallel by those nodes Denis Caromel

  46. GUI in ProActive Parallel Suite Denis Caromel

  47. GUI in ProActive Parallel Suite Denis Caromel

  48. 5. IC2D Interactive Control & Debug for Distribution Eclipse GUI for the GRID Denis Caromel

  49. Programmer Interface for Analyzing and Optimizing Denis Caromel

  50. Denis Caromel

More Related