Denis Caromel Institut universitaire de France (IUF) OASIS Team

1. GRID and Components for Scientific Computing Denis Caromel Institut universitaire de France (IUF) OASIS Team INRIA -- CNRS - I3S -- Univ. of Nice Sophia-Antipolis REUNA, Santiago, May 2004 • 1. Grid principles • 2. Distributed Objects and Components: • ObjectWeb ProActive LGPL environment • 3. Application: 3D Electromagnetism • 4. Towards Peer-To-Peer (P2P)

2. The GRID PCs : 1 Milliard in 2002 (25 years) Forecast: 2 Milliards in 2008

3. The Grid idea • GRID = Electric Network • Computer Power (CPU cycles) <==> Electricity • Can hardly be stored, if not used --> Lost • Global management, • Mutual sharing of the resource • But CPU cycles much harder to share than electricity: • Production cannot be adjusted! • Cannot really be delivered where needed! • Not yet interoperability: MultipleAdministrative Domains • 2 important aspects : • Computational + Data Grid

4. Example Challenge: Scale up • 50 Machines, 1.5 year of computation • 5000 Machines, with only 50 % Efficiency • ==> 10 days • Applications: • Research for an urgent vaccine • Forecast a bush-fire path • Forecast in real time a flooding consequences ... • etc.

5. Ubiquitous: some numbers • PCs in my lab (INRIA Sophia) : ~ 1500 French Riviera : 1.3 Millions • France : 25 Millions Europe : 108 Millions USA : 400 Millions • Chile: ?? Millions South America: ?? Millions • World : 1 Milliard in 2002 (25 ans) Forecast: 2 Milliards in 2008 • France : • 36 Millions of cellular phones • 2.2 Millions of laptops • 630 Thousand PDA (sources: ITU, Gartner Dataquest, IDC, 02-03, )

6. The multiple GRIDs • Scientific Grids : • Parallel machines, Clusters • Large equipments: Telescopes, Particle accelerators, etc. • Enterprise Grids : • Data, Integration: Web Services • Remote connection, Security • Intranet and Internet Grids, (miscalled P2P grid): • Desktop office PCs: Desktop Intranet Grid • Home PC: Internet Grid (e.g. SETI@HOME)

7. Internet Apache Servlets EJB Databases Enterprise Grids

8. Large Equipment Internet Parallel Machine Clusters Scientific Grids

9. Internet Job management for embarrassingly parallel application (e.g. SETI) Internet Grids

10. The multiple GRIDs • Scientific Grids • Enterprise Grids • Intranet and Internet Grids Strong convergence in process! At least at the infrastructure level, i.e. WS

11. Grid: from enterprise ... to regional Very hard deployment problems … right from the beginning

12. Grid: from regional ... to worldwide • Communication France - Santiago: 70 ms Light Speed • Challenge: Hide the latency ! Define adequate programming model

13. Distributed Objects and ComponentsProActive

14. SMP LAN Clusters ProActive:A Java API + Tools for the GRID Parallel, Distributed, Mobile, Activities, across the world ! Desktop • Model: • Remote Objects • Asynchronous Communications, with automatic Futures • Group Communications, Migration (computation mobility) • Environment: • XML Deployment, dynamic class-loading • Various protocols: rsh, ssh, LSF, Globus, BPS, ... • Graphical Visualization and monitoring: IC2D

15. A V Creating AO and Groups A ag = newActiveGroup (“A”, […], VirtualNode) V v = ag.foo(param); ... v.bar(); //Wait-by-necessity Group and Asynchrony are crucial for the GRID Typed Group Java or Active Object

16. Mobility • Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) • Safe migration (no agent in the air!) • Local references if possible when arriving within a VM • Tensionning (removal of forwarder)

17. Mobility • Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) • Safe migration (no agent in the air!) • Local references if possible when arriving within a VM • Tensionning (removal of forwarder)

18. Mobility • Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) • Safe migration (no agent in the air!) • Local references if possible when arriving within a VM • Tensionning (removal of forwarder) direct

19. Mobility • Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) • Safe migration (no agent in the air!) • Local references if possible when arriving within a VM • Tensionning (removal of forwarder) direct direct

20. Mobility • Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) • Safe migration (no agent in the air!) • Local references if possible when arriving within a VM • Tensionning (removal of forwarder) direct forwarder direct

21. Mobility • Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) • Safe migration (no agent in the air!) • Local references if possible when arriving within a VM • Tensionning (removal of forwarder) direct forwarder direct

22. Mobility • Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) • Safe migration (no agent in the air!) • Local references if possible when arriving within a VM • Tensionning (removal of forwarder) direct forwarder direct

23. Mobility • Same semantics guaranteed (RDV, FIFO order point to point, asynchronous) • Safe migration (no agent in the air!) • Local references if possible when arriving within a VM • Tensionning (removal of forwarder) direct forwarder direct

24. Parallel, Distributed, HierarchicalComponentsfor the Grid

25. Component interface My Business Component Facets Receptacles OFFERED REQUIRED Event sinks Event sources Attributes A CORBA Component Courtesy of Philippe Merle, Lille, OpenCCM platform

26. Building CCM Applications =Assembling CORBA Component Instances Provide + Use, but flat assembly

27. Controller Content The Fractal model:Hierarchical Component Defined by E. Bruneton, T. Coupaye, J.B. Stefani, INRIA & FT

28. Controller Content Interface = access point

29. Controller Content Hierarchical model : composites encapsulate primitives, which encapsulates Java code

30. Controller Content Binding = interaction

31. Controller Content Binding = interaction

32. Component Identity Content Controller LifeCycle Controller Binding Controller Controllers : non-functional properties Controller Content Component = runtime entity

33. D C ProActiveComponents for the GRID 1. Primitive component Java + Legacy An activity, a process, … potentially in its own JVM 2. Composite component Composite: Hierarchical, and Distributed over machines 3. Parallel and composite component Parallel: Composite + Broadcast (group)

34. P A B A A C B D C Group proxy Group proxy Groups in Components A parallel component! Broadcast at binding, on client interface At composition, on composite inner server interface

35. Migration Capabilityof composites • Migrate sets of components, including composites

36. Migration Capabilityof composites • Migrate sets of components, including composites

37. Co-allocation, Re-distribution • e.g. upon communication intensive phase

38. Co-allocation, Re-distribution • e.g. upon communication intensive phase

39. Co-allocation, Re-distribution • e.g. upon communication intensive phase

40. Environments

41. Large Equipment Internet Internet Internet Apache Servlets EJB Databases Job management for embarrassingly parallel application (e.g. SETI) Parallel Machine Clusters How to deploy on the Various Kind of Grids ?

42. Abstract Deployment Model • Problem: • Difficulties and lack of flexibility in deployment • Avoid scripting for: configuration, getting nodes, connecting, etc. • A key principle: Virtual Node (VN) + XML deployment file • Abstract Away from source code: • Machines • Creation Protocols • Lookup and Registry Protocols • Protocols and infrastructures: • Globus, ssh, rsh, LSF, PBS, … Web Services, WSRF, ...

43. JVM on the current Host JVM started using SSH Mapping Virtual Nodes: example (1) • <processDefinition id="linuxJVM"> • <jvmProcess class="org.objectweb.proactive.core.process.JVMNodeProcess"/> • </processDefinition> • <processDefinition id=”sshProcess"> • <sshProcess class="org.objectweb.proactive.core.process.ssh.SSHJVMProcess" • hostname="sea.inria.fr"> • <processReference refid="linuxJVM"/> • </sshProcess> • </processDefinition> Infrastructure informations

44. Definition of LSF deployment, … Globus Mapping Virtual Nodes: example (2) • <processDefinition id=" clusterProcess"> • <bsubProcess class="org.objectweb.proactive.core.process.lsf.LSFBSubProcess" • hostname=”cluster.inria.fr"> • <processReference refid=”singleJVM"/> • <bsubOption> • <processor>12</processor> • </bsubOption> • </bsubProcess> • </processDefinition> Infrastructure informations

45. XML Deployment (Not in source) VNa VNb VNc = VN(a,b) C C A B A B Separate or Co-allocation

46. IC2D: Interactive Control and Debugging of Distribution

47. Monitoring of RMI, Globus, Jini, LSF cluster Nice -- Baltimore ProActive IC2D: Width of links proportional to the number of com- munications

48. Application

49. A Parallel Object-Oriented Application for 3D Electromagnetism • Visualize the radar reflection of a plane, medicine on head, etc. • Pre-existing Fortran MPI version: EM3D • Jem3D: • Sequential object-oriented design, modular and extensible (in Java) • Sequential version can be smoothly distributed: --> keeping structuring and object abstractions • Efficient distributed version, large domains, Grid environment

50. The interface