1 / 30

The Knowledge-based Workflow System for Grid Applications

The Knowledge-based Workflow System for Grid Applications. Ladislav Hluch ý , Viet Tran, Ondrej Habala II SAS, Slovakia hluchy.ui@savba.sk http://www.ui.sav.sk/. K -Wf Grid: Partners. Fraunhofer FIRST (Berlin, Germany) UIBK (Innsbruck, Austria)

henrik
Download Presentation

The Knowledge-based Workflow System for Grid Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Knowledge-based Workflow System for Grid Applications Ladislav Hluchý, Viet Tran, Ondrej Habala II SAS, Slovakia hluchy.ui@savba.sk http://www.ui.sav.sk/

  2. K-Wf Grid: Partners • Fraunhofer FIRST (Berlin, Germany) • UIBK (Innsbruck, Austria) • IISAS (Bratislava, Slovak Republic) • CYFRONET (Cracow, Poland) • LogicDIS S.A. (Athens, Greece) • Softeco Sismat SpA (Genoa, Italy) http://www.kwfgrid.eu EGEE User Forum

  3. K-WfGrid: Consortiumwww.kwfgrid.net Berlin Kraków Innsbruck Bratislava Genova Athens EGEE User Forum

  4. K-Wf Grid: Objectives • Objectives of K-Wf grid • To enable users to create complex workflows and use grid resources without detailed knowledge of grid • To construct workflows optimized for underlying infrastructure, using its advantages and avoiding its bottlenecks • To (semi-)automatically construct workflows based on user’s requirements, using semantic annotation of services, data, applications and resources • To constantly renew information about the grid by using complex monitoring network – to learn from experience • To provide simple, easy-to-use interface to K-Wf Grid services EGEE User Forum

  5. Knowledge-based Workflow System for Grid Applications – K-Wf Grid Execute workflow Monitor environment Construct workflow Analyze information Reuse knowledge Capture knowledge EGEE User Forum

  6. K-Wf Grid: Advantages • Composition of workflow from a set of services • System composes the workflow for you – just tell him what you want to get at the end • System uses services which are available at the time and which are expected (based on past experience) to provide good results (good = what you want) • Usability • Less grid language, more application domain language • Integrated collaboration interfaces • Reuse of components • K-Wf Grid is based on respected standards EGEE User Forum

  7. Abstract Operation Web Service Classes Web Service Operations Grid Resources K-Wf Grid: Workflow Refinement “I want this data” “There is my data!” User Request EGEE User Forum

  8. K-Wf Grid: Pilot applications • PAs as domain-related instances of a typical K-Wf Grid-based application system • Common horizontal features / Specific vertical customization • Flood Forecasting Simulation Cascade (FFSC, PA1) • Chain of simulations, targeting hydraulic simulation of a flooded area • Stateful WSRF services implementation in Java, using Globus Toolkit 4 • Enterprise Resource Planning (ERP, PA2) • Stateless web services in Java (Tomcat/Axis) • Coordinated Traffic Management (CTM, PA3) • Traffic simulation in Genoa, Italy • Developed as stateless web services in Perl EGEE User Forum

  9. K-Wf Grid: PA1 – FFSC • Flood Forecasting Simulation Cascade • a cascade of simulation models for weather forecasting, hydrological and hydraulic simulations, able to produce predictions of possible floods in target areas • Input • map of area, precipitation, river water level etc. • Output • water flow in target area • Suggestions / user support • applicability of different models to specific scenarios (QoS) • comparison between stored predictions and real data • future dispersion forecasting EGEE User Forum

  10. Flood Forecasting Application(PA1-FFSC) • Flood forecasting based on a series of simulations • Several simulation models for • Meteorology • Hydrology • Hydraulics • 2D/3D visualization • Implementation using GT4 WSRF • Being extended • More models • More services – towards risk management EGEE User Forum

  11. Replica location Metadata catalog Central services Informationindex EGEE User Forum

  12. Monitoring service Replica location Metadata catalog Central services Informationindex EGEE User Forum

  13. Monitoring service Knowledge store Information processing Workflow management Replica location Metadata catalog Central services Informationindex EGEE User Forum

  14. Monitoring service Knowledge store Information processing Workflow management Replica location Portal Metadata catalog Central services Informationindex EGEE User Forum

  15. Monitoring service Knowledge store Information processing Workflow management Replica location Portal Metadata catalog Central services Execute workflow Informationindex Monitor environment Construct workflow Analyze information Reuse knowledge Capture knowledge EGEE User Forum

  16. Monitoring service Knowledge store Information processing Workflow management Replica location Portal Metadata catalog Central services Execute workflow Informationindex Monitor environment Construct workflow Analyze information Reuse knowledge Capture knowledge EGEE User Forum

  17. Monitoring service Knowledge store Information processing Workflow management Replica location Portal Metadata catalog Central services Execute workflow Informationindex Monitor environment Construct workflow Analyze information Reuse knowledge Capture knowledge EGEE User Forum

  18. Monitoring service Knowledge store Information processing Workflow management Replica location Portal Metadata catalog Central services Execute workflow Informationindex Monitor environment Construct workflow Analyze information Reuse knowledge Capture knowledge EGEE User Forum

  19. Monitoring service Knowledge store Information processing Workflow management Replica location Portal Metadata catalog Central services Execute workflow Informationindex Monitor environment Construct workflow Analyze information Reuse knowledge Capture knowledge EGEE User Forum

  20. Monitoring service Knowledge store Information processing Workflow management Replica location Portal Metadata catalog Central services Execute workflow Informationindex Monitor environment Construct workflow Analyze information Reuse knowledge Capture knowledge EGEE User Forum

  21. Porting to gLite • In K-Wf grid, applications are implemented as WSRF services • In gLite, the services are replaced by a set of worker jobs that are submited by workflow manager via gLite WMS • When started, the worker jobs will contact to workflow manager for executing tasks EGEE User Forum

  22. Monitoring service gLite WMS Knowledge store Information processing Workflow management Replica location Portal Metadata catalog Central services Execute workflow Informationindex Monitor environment Construct workflow Reuse knowledge Capture knowledge EGEE User Forum

  23. Monitoring service gLite WMS Knowledge store Information processing Workflow management Replica location Portal Metadata catalog Central services Execute workflow Informationindex Monitor environment Construct workflow Reuse knowledge Capture knowledge EGEE User Forum

  24. Int.eu.grid • Goals • Deploy and operate an interoperable production-level e-Infrastructure for demanding interactive applications • Distributed (MPI-) parallel interactive computing and storage at Tera level • User friendly access through a interactive Grid desktop • Support for virtual organizations at all levels: • Setup, Collaborative environment, Grid enhancement of applications, Execution and monitoring tools • Project details • Project start: May 1, 2006 • Project duration: 24 months • Contribution of UI SAV • Operation of computer cluster with 32 CPUs and 32GB RAM as part of the project production testbed • User support, organization of tutorials and users’ trainings • Support for environmental applications and their users • Implementation of interactivity to flood simulation application • Support for gridification and interactivity of radiocative pollution simulation aplication (owned by MicroStep-MIS) http://www.interactive-grid.eu/ EGEE User Forum

  25. Int.eu.grid - Partners • BIFI, Zaragoza/Spain • CESGA, Santiago de Compostela/Spain • CSIC-IFCA, Santander/Spain • CYFRONET, Cracow/Poland • FZK, Karlsruhe/Germany • GUP, Linz/Austria • HLRS, Stuttgart/Germany • ICM, Warsaw/Poland • LIP, Lisbon/Portugal • PSNC, Poznan/Poland • TCD, Dublin/Ireland • UAB, Barcelona/Spain • UI SAV, Bratislava/Slovakia EGEE User Forum

  26. Migrating Desktop Remote Access Server Computing Element Job Submission Services gLogin Worker node 0 Worker node 1 … Worker node N Int.eu.grid - Interactivity CrossBroker JDL Gatekeeper Interactive input LRMS Interactive output Job control Interactive channels EGEE User Forum

  27. Migrating Desktop GWUI GWUI Output/visualizations /state Control Interactive job MM5 MM5 DAVEF DAVEF HSPF HSPF GWES GWES EGEE User Forum

  28. GWES Migrating Desktop GWUI Ontology of modules and data Construction of workflows Output/visualizations Control Execution of workflows Interactive job Monitoring of workflows MM5 DAVEF HSPF GWES EGEE User Forum

  29. Flood Interactivity in Int.eu.grid • Job is internally controlled by a workflow manager • Job steps can be added/removed during runtime • Can be paused/restarted/modified • Job outputs/visualized data are available through the MD interface • Job may be cloned • State = workflow state + intermediate data, may be easily transferred • Good for parameter studies EGEE User Forum

  30. Conclusion • Current research focuses on SOKU (Service Oriented Knowledge Utilities) • K-Wf Grid is one of the early implementations of SOKU concepts • How to adapt infrastructure research to this shift in paradigm? • Application developers & end users need easy access to grid infrastructure • SOKU is the way to achieve this • How to extend gLite towards SOKU? EGEE User Forum

More Related