270 likes | 346 Views
The Gateway System. This project is a collaborative effort between Northeast Parallel Architectures Center (NPAC) Ohio Supercomputer Center (OSC) Aeronautical Systems Center (ASC) MSRC. Goals.
E N D
The Gateway System This project is a collaborative effort between Northeast Parallel Architectures Center (NPAC) Ohio Supercomputer Center (OSC) Aeronautical Systems Center (ASC) MSRC T. Haupt
Goals • To provide a problem-oriented interface (a Web portal) to more effectively utilize HPC resources from the desktop via the Web browser. • This “point & click” view hides the underlying complexities and details of the HPC resources and creates a seamless interface between the user’s problem description on his/herdesktop system and theheterogeneous computing resources • TheseHPC resources include supercomputers, mass storage systems, databases, workstation clusters, collaborative tools, and visualization servers. T. Haupt
Design Issues • Support for a seamless access (security) • Support for distributed, heterogeneous Back-End services (HPCC, DBMS, Internet, ...) managed independently from Gateway • Variable pool of resources: support for discovery and dynamical incorporation into the system • Scalable, extensible, low-maintenance Middle Tier • Web-based, extensible, customizable, self-adjusting to varying capacities and capabilities of clients (humans, software and hardware) front end T. Haupt
Towards a complete solution ... Problem description:I need to model the surface damage due to the impact of laser to harden the material bulk. I need access to models including material bulk properties and interaction with intense electromagnetic fields. Task description: I need 64 nodes of SP-2 at Argonne to run my MPI-based executable “a.out” you can find in “/tmp/users/haupt” onmarylin.npac.syr.edu. In addition, I need any idle workstationwith jdk1.1 installed. Make sure that the output of my a.out is transferred to that workstation Middle-Tier: map the user’s task description onto the resource specification; this may include resource discovery, and other services Resource Specification Resource Allocation: run, transfer data, run
Front-End Services User Modules Back-End Resources Three-Tier Architecture • Tier 1 is a high-level, browser-based Front End • for visual programming (including selection of applications, • generation of input data sets, specification of resources, • post-processing and visualizations) • Distributed, object-based, • scalable, and reusable • Web server, Object broker • and Resource Manager • Middleware forms Tier 2 • Back-End services • comprise Tier 3. T. Haupt
Abstract Task Specification Middle-Tier Resource Specification Target Architecture Problem Solving Environment CTA specificknowledgedatabases Visual Authoring Tools User and Group Profiles ResourceIdentificationand Access Visualizations Collaboration WebFlow Back-End Resources
Portal Page • Provides initial access to the Gateway. • After mutual authentication of the user and the Gateway server, creates a user context, and returns a (signed) control applet. T. Haupt
User and Group Profile • Controls the user/group environment • file access • job monitoring • ... • Allows for customization • preferences • users with disabilities • ... • History of actions • Scientific notebook T. Haupt
Control Applet • The control applet is responsible for maintaining the session, and direct communication with the middle-tier. • Direct communication is the most efficient, but since it is buried into an applet, this mechanism is not readily customizable. • The generic services, such as file service (upload, download, edit, copy, move, delete) and job services (show current jobs/show queues/kill jobs) will be supported this way. [combination of the user context and a query] • The Gateway will also support a non-direct communication with the middle-tier through servelts. T. Haupt
Screen Dump of the Control Applet T. Haupt
Application Domain DataBasesPSE Example: CCM 1. Enter the Gateway system 2. Define your problem 3. Identify resources (software and hardware) 4. Create input file 5. Run your application 6. Analyze results Ken Flurchick, http://www.osc.edu/~kenf/theGateway T. Haupt
Selecting a Predefined Task T. Haupt
Visual Authoring Tools • Allows for composition of the computational task from components (reusable modules) • Different tools to support various programming models such as data parallel, task parallel, data flow, object oriented • No assumption on granularity • Metadata about components and support for archiving and mining the components • Support for instrumentation and steering T. Haupt
Example: Data Flow T. Haupt
Example: DARP T. Haupt
Navigate and choose an existing applicationto solve the problem at hand.Import all necessary data. Example of a portal Select host Select model Set parameters Run Retrieve data Pre/post-processing Run simulations
Processing Task Specification Front-End Applet XML A visual representationis converted into a XML document parse Middle-Tier Web Server save XML service ApplContext Publishes IOR Generates Java code to add modules to ApplContext
Other toolboxes • Visualizations • Collaboration • Scientific notebook • ... T. Haupt
Mesh of WebFlow Servers implemented as CORBA objects that manage and coordinate distributed computation. Front End CORBA Based Middle-Tier Gatekeeper Authentication Authorization T. Haupt
WebFlow Server WebFlow server is given by a hierarchy of containers and components WebFlow server hosts users and services Each user maintainsa number of applicationscomposed of custom modules and common services User 1 User 2 Application 1 App 1 App 2 Application 2 WebFlow Services
WebFlow Context Hierarchy Master Server (Gatekeeper) Slave Server Proxy Slave Server Application Context Module Slave Server User Context T. Haupt
Module functionality • May act as a client for a back-end service such as Globus GRAM or a database • May invoke other Gateway Middle Tier services such as File Services or Resource Discovery • May implement the desired functionality internally (say, in Java) [thus not a proxy] • May interact with other modules and the Front End through events T. Haupt
Browser based Front-End BrowserbasedFront-End Middle-Tier modulesserve as proxies ofBack-End Services User Space Definition and Task Specification Services User Modules Metacomputing Services T. Haupt Back-End Resources
Back End Services • Access to HPCC (via Globus) • Access to distributed databases (via JDBC) • Access to mass storage • Access to the Internet resources • Access to desktop application and local data • Access to code repositories T. Haupt
In order to run WebFlow over Globus there must be at least one WebFlow node capable of executing Globus commands, such as globusrun Jobs that require computational power of massively parallel computers are directed to the Globus domain, while other jobs can be launched on much more modest platforms, such as the user’s desktop or even a laptop running Windows NT. Bridge between WebFlow and Globus WebFlow over Globus T. Haupt
Security Model (Keberos) Front End Applet SECIOP Layer 1: secure Web delegation Layer 2: secure CORBA Gatekeeper SECIOP Layer 3: Secure access to resources authentication & authorization GSSAPI GSSAPI HPCC resources Policies defined by resource owners