1 / 16

Workflow sharing and integration services by the ER-flow project

Workflow sharing and integration services by the ER-flow project. k itti . v arga@sztak i.mta . hu on behalf of the ER -flow consortium. EGI Community Forum, Manchester, 8-12 April, 2013. Outline. Motivations SHIWA  ER-flow Technology services SHIWA Simulation Platform

marcie
Download Presentation

Workflow sharing and integration services by the ER-flow project

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Workflowsharing and integration services by the ER-flow project kitti.varga@sztaki.mta.hu on behalf of the ER-flow consortium EGI Community Forum, Manchester, 8-12 April, 2013

  2. Outline • Motivations • SHIWA  ER-flow • Technology services • SHIWA Simulation Platform • Integrated workflow engines • Example • Integrating Moteur & WS-PGRADE workflows • User support services

  3. Motivation Isolation by technology • Many different workflow systems exists  We expect no change • Technological choice isolates users and user communities • Technological choice selects DCI/middleware (WS, gLite, Globus, …) Benefits oftheSHIWA Simulation Platform • Share your own workflow, re-use workflows of others • Create and execute ‘meta-workflows’: built from smaller workflows that use different workflow languages/technologies • Combine workflows and DCIs SHIWAER-flow • SHIWA: FP7 R&D project. Created the SHIWA Simulation Platform (2010-2012) • ER-flow: FP7 support action project. Disseminates the SHIWA technology (2012-2014).

  4. Generic use-cases SE CE DCI #2 SE CE DCI #1 DCI – distributed computing infrastructure; CE/SE – computing/storage element

  5. Generic use-cases find and re-use combine WFs with own data SE CE on own CE? DCI #2 SE CE DCI #1 DCI – distributed computing infrastructure; CE/SE – computing/storage element

  6. ER-flow project Technology providers + User support: MTA-SZTAKI, UoW, CNRS, EGI.eu (through the NGIs) Research Communities: Astro-physics INAF Computational Chemistry LMU + TUD Heliophysics TCD + UCL Life Science AMC + WE ARE LOOKING FOR NEW COMMUNITIES Duration: September 2012 – August 2014

  7. Aims, services, actors Aims: • Provide a platform for research communities that enables seamless execution of workflows of different systems • Investigate data interoperability issues in the workflow domain and propose solutions Services: • SHIWA Simulation Platform • Dissemination and outreach • Training for workflow developers, workflow users Key actors: • Workflow engine developers • Workflow developers • Researchers

  8. ER-flow technology:SHIWA Simulation Platform

  9. A portal/desktopto integrate, parameterize and run these applications Supercomputer grids (PRACE, XSEDE) Access to a large set of ready-to-run scientific workflowapplications Access to various workflow engines and DCIs where these WF applications can be run SHIWA Repository SHIWA Portal Cluster grids (EGI, OSG, …) Desktop grids (BOINC, Condor, etc.) Clouds Grid systems Supercomputers Distributed Computing Infrastructures (DCIs) Local clusters What does the developerof a (meta-)workflow need? + Workflow engines Web services

  10. Facilitatespublishingand sharingworkflows Supports: Abstract workflows with multiple implementations of over 10 workflow systems Concrete workflows with execution specific data Available: From the SHIWA Portal http://ssp.shiwa-workflow.eu Standalone interface: http://repo.shiwa-workflow.eu SHIWA Repository

  11. SHIWA portal SHIWA Repository Connected workflow engines Example: Create meta-workflow fromMoteur and WS-PGRADE workflows Access to Moteur WF Moteur WF engine DCI 1 gLite DCI 2 Cloud DCI n ARC 12

  12. An example: SHIWA solution for LINGA Multi- Workflow Management Sub-Workflows

  13. Computational Chemistry Applications in ER-flow Separate talks later today about Astro & Heliophysics workflows!

  14. ER-flow user support • SHIWA User Forum • www.erflow.eu/shiwa-user-forum • Workshops: • Workflow Application Porting Workshop, London 19-22 March 2013 http://www.erflow.eu/19-22-march-2013-application-porting-workshop

  15. Conclusions • Scientific workflow landscape is fragmented • Many workflow system with specialised & small user base • Many workflows with specialised & small user base • SHIWA Simulation Platform enables integration • Within disciplines • Across disciplines • ER-flow project provides user support • Consultancy & Workshops • User forum • Open for new communities

  16. Questions? www.erflow.eu/shiwa-user-forum www.erflow.eu

More Related