1 / 21

SAGA : A Simple API for Grid Applications

SAGA : A Simple API for Grid Applications. Shantenu Jha Centre for Computational Sciences, UCL SJ also acknowledges GridNet funding [EPSRC]. Outline Motivation for SAGA Use Cases, Requirement Analysis, Strawman API Status, Implementation(s) and Roadmap Is it Underweight or Overweight?

ringo
Download Presentation

SAGA : A Simple API for Grid Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SAGA : A Simple API for Grid Applications Shantenu Jha Centre for Computational Sciences, UCL SJ also acknowledges GridNet funding [EPSRC] Lightweight Grid Computing Worskhop, May 2006

  2. Outline Motivation for SAGA Use Cases, Requirement Analysis, Strawman API Status, Implementation(s) and Roadmap Is it Underweight or Overweight? Quick connections with what has been said here.. An “application developers” pespective Ignorant of implementation details Early days compared to others.. Lightweight Grid Computing Worskhop, May 2006

  3. Simple API for Grid Applications: Motivation • Q: Why are there so few grid applications used routinely? • - Need to hide underlying complexities, heterogenities and changes from application program(er) • Lack of simple, stable and uniform high-level programming interface that integrates [common look-and-feel across functional areas] the most common grid programming abstractions. • e.g., Globus job-launch, file transfer etc. exist. SAGA operates at the level of scientific applications - end user applications. Lightweight Grid Computing Worskhop, May 2006

  4. Simple API for Grid Applications: Motivation (2) Parallel Programming Analogy Revisited: SAGA is to the grid application developer, what MPI is to the Parallel Program developer (message passing model) Conception and trajectory somewhat similar to MPI… Measure(s) of success: Does SAGA enable the rapid prototyping of “new” grid applications? Does it implement complex functionality using significantly less code? Need to change code. Retrofit at the minimum. Lightweight Grid Computing Worskhop, May 2006

  5. SAGA: Philosophical Roots • Projects came together at the GGF’s APPS Umbrella RG • ReG Steering and GAT (GridLab) • User typically needs to just learn a simple API • API does not change much - stability • Broader in scope to both ReG steering library & GAT Asynchronous versions of nearly all calls • - CoG Kit: • Globus has API, no use to AD! same set capabilities • “exposes features of Globus at a level of abstraction natural to AD” • Don’t reinvent the wheel (yet again..); work with other GGF groups/standards: e.g. DRMAA, JSDL, GridRPC, Lightweight Grid Computing Worskhop, May 2006

  6. SAGA Strawman API Design Decisions • Developed and specified using SIDL - Scientific Interface Definition Language (Babel, CCA projects) • - Language independent specification • Object-oriented preferred design - easier to adapt to procedural languages than vice-versa • Each subsystem is independent (almost). Facilitates independent development and implementation of API Lightweight Grid Computing Worskhop, May 2006

  7. SAGA: Requirement Analysis Document Summary Derived from 17 (10 “uncorrelated”) Use Cases. - Projects: E.g., coreGrid, ReG, GridLab - Applications: E.g., Coastal-modelling, visualization - Functionality: GridRPC Asynchronous operations should be supported Bulk operations should be supported Error support for the API should allow for application level error recovery strategies Auditing and logging should not be exposed at the API Workflows and QoS (as of V1.0) do not require explicit support at the API Lightweight Grid Computing Worskhop, May 2006

  8. SAGA Strawman API (draft) Details 143 page document and growing… Functional Areas: Job mgmnt, Resource mgmnt, Data mgmnt & access Logical files Streams Non-functional Areas:Bulk Operations Async QoS No code snippets. See me for details/URL Lightweight Grid Computing Worskhop, May 2006

  9. SAGA: Status (pre-GGF17, 2006) & Roadmap SAGA-RG formally chartered between GGF11/12 SAGA Use Case 2nd half 2004-early 2005 SAGA-CORE WG agreed at GGF16 (first meeting GGF17) SAGA-RG refine, extend & new functional areas: gridRPC, gridCPR, steering Use Case Analysis - GFD.70 Requirement Analysis Submitted to GGF Process - April 2005 Draft V1.0 API (Strawman) to be submitted to GGF May/June 2006 Lightweight Grid Computing Worskhop, May 2006

  10. SAGA-RG [1] and SAGA-CORE-WG [2] Currently Active contributors: Andre Merzky [2], Thilo Kielmann [1] (VU, Amsterdam) Hartmut Kaiser [2] (LSU) Pascal Kleijer (NEC, Japan), SJ [1] (UCL) Additional contributions by: Stephen Pickles (Manchester) Tom Goodale [2] (Cardiff/LSU) , C. Lee (Aerospace) J Shalf, D Konerding (LBNL), H Rajic (Intel) Gregor von L (ANL), C Smith (Platform) Lightweight Grid Computing Worskhop, May 2006

  11. SAGA: “Specification” versus “Standard” Implementations GGF Standard: Need two independent implementations of the specification to become a standard SAGA (Draft) Specification Implementation: - OMII (Java) [full] - Vrije/LSU (C++) [full] - DEISA (Java) [partial] Lightweight Grid Computing Worskhop, May 2006

  12. SAGA: Strawman Architecture Diagram • SAGA Engine resides on user machine. • Complexity in the SAGA engine and adaptor(s). OK as long as not difficult to compile and run?? • Calls appropriate adaptors. • Adaptor developer support is critical!! Lightweight Grid Computing Worskhop, May 2006

  13. Is SAGA Lightweight? - “Simple” API for Grid Applications Simple at which level?? Grid environment is complex; can’t remove the complexity entirely. Best layer to place/park/hide it ? - Application, Application, Application: SAGA is not middleware! SAGA is for coding against middleware and not for coding middleware Lightweight Grid Computing Worskhop, May 2006

  14. Is SAGA Lightweight? (2) Lightweight in scope & use not in implementation - scope: 80:20 Satisfies 80% of use cases with 20% complexity 80% functionality with 20% of the API calls - implementation: (pg 26, From Req Analysis Doc) “For the SAGA API Specification, simplicity of use should be more important that simplicity of implementation” Lightweight Grid Computing Worskhop, May 2006

  15. Application SAGA API Bound at link time Packages (Proxies) Stream File/Dir SAGA Engine SAGA CPI Bound at runtime Adaptors File/Dir Stream Vrije/LSU: Overall Architecture CPI = Capability provider interface Lightweight Grid Computing Worskhop, May 2006

  16. Vrije/LSU: Overall Architecture Application binds to SAGA engine and packages at link time Adaptors are loaded at runtime by the engine Adaptors get selected based on CPI group and preferences Packages get connected to adaptors on call by call basis Lightweight Grid Computing Worskhop, May 2006

  17. SAGA: Applications Plans (2006-2008) Let us assume that the MPI for Grid now exist. Allows us to begin “possibly” creating SG Grid Applications Applications created ab-initio for grid environments Example: Using SAGA to create distributed Replace Exchange Methods (REM) using MD exploiting lightpaths proteinfolding@home - single proc, limited sys size MPP - inefficient waiting for swap configuration Lightweight Grid Computing Worskhop, May 2006

  18. Lessons from the workshop Opportunities for Developers of Middleware Lessons from SAGAs extensive use case repo Bulk Job Submission? DEISA Job Submission - JRA7 (DESHL) Logical file (Replica systems) interface? Feedback from Application Developers Read the specification. Does it meet your requirements? If yes, say yes! If no, say why? SAGA template use cases? Lightweight Grid Computing Worskhop, May 2006

  19. Lessons from the workshop (2) Opportunities/Lessons for SAGA: AHE? GridSAM? GROWL ?(SABRE) SAGA does not have a security model (yet) Suggestions on how to keep SAGA independent of any security model? Issue is more than that of certificates Lightweight Grid Computing Worskhop, May 2006

  20. SAGA: Recap & Concluding Remarks • Aims to find right application level abstractions. • Derive a simple, usable and consistent API. Common look-and-feel across functional areas. • Define “grid primitives” for (HPC) scientific Grid applications. Recap pre-“MPI” for parallel applications; pre-SAGA for grid applications.. Lightweight Grid Computing Worskhop, May 2006

  21. SAGA-OGSA or OGSA-SAGA? SAGA vs OGSA: Complementary not competitive - OGSA defines interfaces at the service and middleware level; SAGA at the application level - OGSA is primarily an architecture; SAGA is API - SAGA implementations interface OGSA-compliant services - OGSA spec is for middleware developers; SAGA is for App developers Lightweight Grid Computing Worskhop, May 2006

More Related