1 / 70

The Legion Project

The Legion Project. Adam Belloum Computer Architecture & Parallel Systems group University of Amsterdam adam@science.uva.nl. Legion requirements. Security Global name space Fault tolerance Accommodating heterogeneity Binary management and application provisioning Multilanguage support

kamala
Download Presentation

The Legion Project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Legion Project Adam Belloum Computer Architecture & Parallel Systems group University of Amsterdam adam@science.uva.nl

  2. Legion requirements • Security • Global name space • Fault tolerance • Accommodating heterogeneity • Binary management and application provisioning • Multilanguage support • Scalability • Persistence • Extensibility • Site autonomy • Complexity management

  3. Legion design principles • Provide a single-system view • Provide transparency as a means of hiding detail • Provide flexible semantics • Reduce user effort • Reduce “activation energy” • Do no harm • Do not change host operating systems

  4. Legion architectural details • Legion started out with the top-down premise  “ that a strong architecture is necessary to build an infrastructure of this scope.” • This architecture is based on communicating services, which are implemented as objects. • An object is a stateful component in a container process.

  5. Legion is a System • Object based meta-system, providing a single integrated infrastructure • All components are objects (unlike GT) • Data abstraction, encapsulation, inheritance, polymorphism • API to core services • Core object types • Classes/metaclasses: managers and policy makers • Host objects: abstractions of processing resources (one or many) • Vault objects: persistent storage • Implementation objects and caches: “exectuables” • Binding agents: maps objects to physical addresses • Context objects: naming of objects

  6. Tasks involved in the design • 3-level naming, binding, and communication scheme. • Remote invocation based on dataflow graphs. • Security layer to every instance of communication between any services. • Fault-tolerance layers. • Layers for resource discovery through • naming and mechanisms for caching name bindings

  7. Much of the later development in Legion has been in terms of adding new kinds of services • two dimensional files, • specialized schedulers, • firewall proxies, • file export services, • etc. • or tools that invoke methods of services (running an application, running parameter space applications, using a Web portal [41], etc.).

  8. Architecture • Object-based Integrated System • What Legion does not do: • require that Legion run as “root” • replace host operating systems • legislate changes to Internet

  9. Legion Objects • Address-space disjoint • Member functions • Non-blocking calls • Arbitrary metadata • Active or Inert

  10. Some Object types • Host • Abstraction of a processing Unit • Vault • Abstraction of a persistent storage • Class Manager • Responsible for object creation, Responsible for object creation, monitoring, and lifetime management monitoring, and lifetime management

  11. Legion Object Identifier (LOID) • Context Name Context Name • Human Human-readable readable • Legion Object ID (LOID) • Location Location-independent independent • Object Address Object Address • Machine Machine-and and-communication communication-specific

  12. Legion Object Identifier (LOID) • Object name • Location-independent • String of bits • Fields • Type • Domain • Class ID • Instance ID • Public Key

  13. What’s a name date • LOIDs are necessary • immutable identifier of an object • Context names are convenient • human-readable • OAs are actual • machine name, port number LOID

  14. Object Naming • Human-readable • translation • Legion Object ID • Biding • Location-specific • Communication-Specific Context Name Legion Object ID Obbject Address

  15. Object Address • Incorporates physical addresses (IP, XTP) • Contains multiple addresses for various forms of multicast and group communication • Can find “best” address dynamically • Bindings are <LOID, OA, timeout> tuples

  16. Unified Console TTY File Program produces stdout, stderr User creates tty object Prog. User shares tty LOID User starts running program Legion passes tty LOID to program User shares tty LOID

  17. TTY Object • Redirect run-time output to central (or multiple) consoles • Connect and disconnect dynamically • Debug quickly and simply • Monitor status, errors, easily • Share console with others legion_tty <ttyobj>

  18. Context Space / • Unix-like legion_ls legion_pwd legion_cd legion_cat ... hosts home users mach1 mach2 mydir you me subdir prog file1 tty

  19. The Legion Architectural view

  20. Legions Two lowers layers • consist of • location-independent abstract names called Legion object identifiers (LOIDs) • and object addresses specific to communication protocols,e.g., an IP address and a port number. • The binding between a LOID and an object address can and does change over time.

  21. Security Layer • The security layer implements the Legion security model for authentication, access control, and data integrity (e.g., mutual authentication and encryption on the wire).

  22. The core object layer • The core object layer addresses: • method invocation, • event processing (including exception and error propagation on a per-object basis ), • interface discovery, • and the management of metadata. • Objects can have arbitrary metadata, such as the load on a host object or the parameters that were used to generate a particular data file.

  23. The core services • Implement object instance • management (class managers) • abstract processing resources (hosts), • and storage resources (vaults). • Core services are represented by base classes • that can be extended to provide different or enhanced implementations.

  24. Higher-Level System Service • Collection of higher-level system service types and enhancements to the base service classes. • These include classes for object • replication for availability • Message logging for accounting or postmortem debugging • firewall proxy servers for securely transiting firewalls, • enhanced schedulers • databases, • job proxy managers that “wrap” legacy codes for remote execution.

  25. Application support layer • Application support layer • contains user centered tools for parallel and high-throughput computing, data access and sharing, and system management. • High-performance tool set for wrapping legacy codes • legion_register_program • legion_run • Legion MPI tools support cross-platform, cross-site • Fortran support tools (Fortran programs for running on a grid).

  26. Legion basic file type • a basic file type (BasicFile) supports the usual functions: read, write, stat, seek, etc. • All other file types are derived from this type. • for example, piped into tools that expect sequential files.

  27. Legion basic file type • There are two-dimensional files that support • read/write operations on columns, rows, and rectangular patches of data (both primitive types as well as “structs”). • There are file types to support • unstructured sparse data, • parallel files in which the file has been broken up and decomposed across several storage systems.

  28. Data movement on the Grid • Data can be copied into the grid • in which case Legion manages the data • decides where to place it, • how many copies to generate for higher availability, • where to place those copies. • Data can be exported into the grid. • When a local directory structure is exported into the grid, it is mapped to a chosen path name in the global name space (directory structure).

  29. Data movement on the Grid • For example, a user can map • data/sequences in his local file system into /home/grimshaw/sequences using the legion_export_dir command, • legion_export_dir data/sequences /home/grimshaw/sequences. • Subsequent access from anywhere in the grid (read/write) is done directly against the files in the users file system (subject to access control).

  30. data grid can be accessed via a daemon that implements the Network File System (NFS) protocol. • The entire Legion namespace, including files, hosts, and so forth, can be mapped into local OS file systems. • Shell scripts, Perl scripts, an user applications can run unmodified on the Legion data grid. • The usual UNIX commands work, as does Microsoft Windows Exploring

  31. Security in Legion Overview of Security Standards in the Grid, CSE 225 High Performance and Computational Grids Spring 2000 kwalsh@ucsd.edu

  32. Security in Legion (1)Design Principals • As in the Hippocratic Oath, do no harm! • Caveat emptor - let the buyer beware. • Small is beautiful.

  33. Security in LegionStandards • X.509 ? • Keberos ?

  34. Security in LegionLegion Security Model • responsible for securing the communications between Legion objects • responsible for access control • determines what objects/ users are allowed to call a particular object’s methods. • relies on the message layer for some services.

  35. Security in Legion (2)Basic Concepts • Every object provides certain known member functions • MayI, CanI, Iam, and Delegate. • Two objects associated with each operation: • a responsible agent (RA) • and a calling agent (CA) • Every invocation of member function is performed in the context of • a certificate which contains the LOID. • Certificate digitally signed by maker

  36. Security in Legion • Object A invokes method B.foo • call is passed to B, and the Legion run-time system automatically invokes B.MayI • If B.MayI is true, then foo is invoked with the arguments • If not, then an exception is raised and passed back the information necessary to make such a decision

  37. Security in Legion • Legion users are responsible for own security. • Object might trust that the CA is correct. • Policies defined by objects themselves. • Every class defines a special member function, MayI. • MayI defines the security objects for a class. • Every member function invocation permitted only if MayI sanctions it.

  38. Security in LegionAutomatic invocation of outgoing calls • Object A invokes B.foo, • the compiler has interposed code so that A.CanI is automatically invoked before the call leaves A. • If CanI returns true, then the method call proceeds. • If CanI returns false, an exception is raised.

  39. Security in Legion • Authentication aided by use of Legion certificates • - based on public-key cryptography by default. Must know private key to authenticate. • MayI functions can code their own authentication protocols • Every Legion object required to supply special member function Iam for authentication purposes.

  40. Security in Legion • Login establishes user identity and creates responsibility agent for user. • Login is building block for authentication and delegation. • Object can delegate new certificate to delegate rights. • Delegation policy defined by object.

  41. Security in LegionFuture Work • Legion does not specify any particular encryption. Future standardization? • Legion eschews distinguished trusted objects - centralized key management server • Composition of a security policy

  42. Web-based portal interface

  43. User portal and system management tools • the user portal and system management tools • add and remove users, • add and remove hosts, • and join two separate Legion grids together. • Web-based portal interface • for access to Legion • a system status display tool that gathers information from system-wide metadata collections and makes it available via a browser

  44. Web-based portal • Web-based portal allows an alternative, graphical interface to Legion. • Using this interface a user can submit • An Amber job (a 3D molecular modeling code) to NPACI-net (a University of Virginia Legion network) • and not care at all where it executes.

  45. Portal view • user can copy files out of the running simulation • and in which a three dimensional molecular visualization is being used to display the intermediate results.

  46. Legion job status tools • Using these tools users can determine the status • of all of the jobs they have started from the Legion portal • and access their results as needed.

  47. Legion accounting system • The portal interface to the underlying Legion accounting system. • Legion keeps track of who used resource (CPU, application, etc.), • with exit status • with how much resource consumption. • Data is loaded into a relational database, and various reports can be generated

  48. Originally a research project at U. Virginia • commercialised via Applied Meta (now Avaki Corp) • Some successful applications • used to perform a metacomputing run of an ocean model • ran a task-farm of computational chemistry calculations • Grid Object Model not really found favour • in development phase, Grid must run existing applications • hard to do this within an integrated environment • major applications are not themselves object oriented • however cf. OGSA Rob Baxter A Complete History of the Grid (abridged)

  49. Context space

  50. Context Space / • Unix-like legion_ls legion_pwd legion_cd legion_cat ... hosts home users mach1 mach2 mydir you me subdir prog file1 tty

More Related