1 / 21

OGCE Workflow and LEAD Overview

OGCE Workflow and LEAD Overview. Suresh Marru, Marlon Pierce. September 2009. The LEAD Gateway. LEAD Pioneering Technology. LEAD Dynamic Adaptive Infrastructure . Storms Forming. Forecast Model. Streaming Observations. Data Mining. Instrument Steering. Refine forecast grid.

miracle
Download Presentation

OGCE Workflow and LEAD Overview

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. OGCE Workflow and LEAD Overview Suresh Marru, Marlon Pierce September 2009

  2. The LEAD Gateway

  3. LEAD Pioneering Technology

  4. LEAD Dynamic Adaptive Infrastructure Storms Forming Forecast Model Streaming Observations Data Mining Instrument Steering Refine forecast grid

  5. LEAD Scientists and Educational Interactions Lowering the barrier for using complex end-to-end weather technologies Democratize Empower Facilitate End Users Developers Researcherss

  6. Open Grid Computing Environments Generalize, Harden, Build Test

  7. Flexible Layered Service Oriented Architecture User Interactions Web Portal XBaya GUI Other Clients XBaya Core Event Bus Middleware Services Workflow Engine (ODE) XRegistry GFac Services XMCCat Metadata Catalog Compute & Data Resources Computational Cloud Computational Grids Local Lab Resources

  8. Application Wrapper Framework c Application Factory • Scientific Applications are wrapped into web services by filling in a web-form. • The Application Factory generates a web service for each application with I/O interfaces. • Registers WSDL for the service with a registry • Each service generates a stream of notifications that log the service actions back to the XMCCat Metadata Catalog, user monitoring, and provenance tracking tools App Service Run program & publish events

  9. Application Wrapper Architecture

  10. Service Monitoring via Events 1 2 3 4 5 6 • The service output is a stream of events • I am running your request • I have started to move your input files • I have all the files • I am running your application • The application is finished • I am moving the output to you file space • I am done • These are automatically generated by the service using a distributed event system(WS-Eventing / WS-Notification) • Topic based pub-sub system witha well known “channel” Application Service Instance Notification Channel x x publisher listener

  11. Workflow Suite Architecture

  12. Interoperable Xbaya Architecture BPEL 1.1 BPEL 2.0 SCUFL Abstract DAG Model Composition and Monitoring Python Dynamic Enactor/Interpreter Jython Based Enactor GPEL Engine Apache ODE Engine Taverna Python Runtime Message Bus

  13. Simple Recovery Architecture Portal BPEL Workflow Engine Application Performance Models Resource Reliability Models Fault Tolerance/ Recovery Service Application Service OVP/ RST/ MIG NWS, MDS BQP Notification Service Deadline & Success Probability 14 14

  14. Workflow Composer

  15. Example: Assimilation Workflow

  16. Science Independence Example Ensemble-type processing (minimal network reqs) Capacity-type computing Parallel processing Capability-type computing • Domain webs of large genomes • Input list of amino acid sequences • Identify all known domains • Construct webs

  17. Biology Gateway

  18. Polar Grid Example • Based on data input Matlab have different filer configure to run different data set and run it to create data images to show different projections • Matlab filer program is command line appication to run image creation Job on Grid computers. • There is a need of middleware to take user request and call the Matlab filter program on different grid machines and send the response back to the user.

  19. Example Gateways using OGCE Workflow Suite • LEAD • Lower entry barrier to using weather analysis tools • Improve detection, analysis & prediction of mesoscale weather • Motif-Network • Transformation of sequenced genomes to “domain-space” • Cyber-Infrastructure Evaluation • Performance evaluation of future supercomputer architectures • ADAM • Algorithms for feature extraction, data normalization, classification and normalization • GridChem • Molecular Chemistry Grid helping researchers run chemistry applications on Grid Environment

  20. Live Demo & Questions?

More Related