1 / 15

GRAPPA

GRAPPA. Part of Active Notebook Science Portal project A “notebook” like GRAPPA consists of Set of ordinary web pages, viewable from any browser Editable execution scripts in Python/Jpython Provides direct access to COG, GDK, VDT tools

meryl
Download Presentation

GRAPPA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GRAPPA • Part of Active Notebook Science Portal project • A “notebook” like GRAPPA consists of • Set of ordinary web pages, viewable from any browser • Editable execution scripts in Python/Jpython • Provides direct access to COG, GDK, VDT tools • Flexible, powerful language with bindings to other languages • Notebook can be run without browser • Satisfies O-O fundamentalists, but usable by real scientists • Input forms that feed parameters to scripts • Notebooks use MyProxy server for credentials, Tomcat • ANSP notebooks can be run in “personal” mode

  2. GRAPPA • Notebook “sessions” are live XML documents • Can be revisited, rerun later • Can be copied/modified for new run • Archived and shared with a collaborator (including live object references, allowing another user remote method invocation) • Notebook goals are to provide • Scripting, launching of jobs via GRAM, Condor, Condor-G, ssh, exec • Interface to information subsystems • Ability for end user to attach arbitrary data to a session: graphs, notes, links, ....

  3. GRAPPA • Application components run by ANSP are standalone executables • Can read/write files • Send event messages (event in CS-talk, not HEP-talk) • Provide DoE Common Component Architecture “port interfaces” • Peers that can keep running when other parts of distributed comp fail ...

  4. Fortran Application (a.out) Component App Manager InputFiles OutFile1 InputFiles InputFiles GRAPPA Component App Manager Model • Application may be untouchable (no source code, etc) • Idea is to make it appear as a fully-enabled component • Create a proxy that manages app and framework comm. exec Start running

  5. Component App Manager Model • App manager is responsible for • making sure all input files are ready before running • event notifications to GRAPPA and other interested parties • publishing file locations, and moving files via a file manager component • When output file of one component is needed as input file of another, receiver is responsible for file move. • App manager can use user's CA to use VDT tools for file transfers, app execution. • App manager can (via resource manager) actually run app on a different machine • Communication about progress, status is primarily via publish/subscribe event messages

  6. File Management • File Manager: component that provides low-level, simple metadata about files • Uses JDBC interface to relational DB • Starts running with a handle to a DB server (may be a “personal mode” DB) • Multiple file managers, with handles to different DB's, can be running in a single application. • File managers use RMI for interchange of data • Likely to be replaced by VDT tools as they come available

  7. File Management • Application developer provides a description of each file an application component reads or writes: <filename>matstruct.gif</filename> Filename the app “opens” <direction>output</direction> Input, output, both <termination>total</termination> Can be streamed or not <format>binary</format> ASCII, binary, or other <mimetype>image/gif</mimetype> Optional; provide if known <description>This is an image of the sparsity structure of the matrix being analyzed; it is part of the overall matstruct.html file </description>

  8. File Management • Notebook developer provides additional information for each file, things which are outside the scope of individual application • Whether file is to be locally archived, remotely archived, or is volatile • Whether file should be cleaned up after/between runs • What kind of compression should be used (if any) • Naming convention for archived files • basename.machine.timestamp.suffix • Location to archive files (machine, directory) • Notebook must provide user with easy, coherent picture of the files • Notebook must also provide for additional files: user notes, etc.

  9. GRAPPA: Overall Picture Users interact via Web browser or scripting Authorization is hidden in server in this picture

  10. GRAPPA: Overall Picture Server accesses a notebook DB for user's sessions

  11. GRAPPA: Overall Picture User's script is part of stored session Components are started through a job launch tool that consults with resource manager Request might be accompanied by requirements

  12. GRAPPA: Overall Picture Executables run under application manager Python scripts

  13. GRAPPA: Overall Picture Application managers publish event messages to file manager about file events

  14. GRAPPA: Overall Picture Application mgr, Grid resource tools publish events to event channels Filtering event channels allow notebook to select events to record, present

  15. GRAPPA/GriPhyN Generalities • Not intended as competitor with VDT, but a user-level interface to VDT tools • Standalone mode is critical • Federated model of information/data management is highly desirable • Handles some security/privacy issues • Event message channel model provides some robustness • Components run more as peers • Components can continue when framework crashes

More Related