Vision of UNICORE – UniGrids and beyond

1. Vision of UNICORE – UniGrids and beyond Daniel Mallmann d.mallmann@fz-juelich.de 2nd CoreGRID Summer SchoolBonn, Germany24th – 28th July 2006

2. Objectives • Develop Unicore/GS • Based on Unicore and the Open Grid Services Architecture • Compliant with the Web Services Resource Framework • Develop generic software components • Visualisation and steering of simulations • Device monitoring and control • Access to distributed data and databases • Validate UniGrids concepts and demonstrate the application of Unicore/GS in Industrial and scientific domains

3. Duration, Funding • 6th Framework Programme of the European Commission • 2nd Call of the IST 2003-2004 Workprogramme • Objective: GRID-based systems and solving complex problems • Instrument: STReP • Duration: 24 months • Start: July 2004 • Effort: 297 PM (12 developer + manager) • Costs: 3.0 M€ • Grant: 1.9 M€

4. GRIDCOORDBuilding the ERA in Grid research K-WF GridKnowledge basedworkflow & collaboration inteliGRIDSemantic Grid based virtual organisations SIMDATGrid-based generic enabling application technologies to facilitate solution of industrial problems OntoGridKnowledge Services for the semantic Grid NextGRIDEU-driven Grid services architecture for businesS and industry AkogrimoMobile Grid architecture and services for dynamic virtual organisations DataminingGridDataminingtools & services HPC4UFault tolerance,dependabilityfor Grid CoreGRIDEuropean-wide virtual laboratory for longer term Grid research-creating the foundation for next generation Grids ProvenanceTrust and provenance for Grids Specific support action Integrated project Network of excellence Specific targeted research project European Context UniGrids Extended OGSAImplementation based on UNICORE UniGrids Extended OGSAImplementation based on UNICORE Courtesy: Max Lemke, European Commission

5. SSAs supporting NESSI and the Grid community SSAs supporting NESSI and the Grid community Grid services, Grid services, Grid services, Grid services, Nessi Nessi Nessi - - - Grid Grid Grid Degree Degree Degree Challengers Challengers Challengers business models business models business models business models trust, security trust, security trust, security trust, security GridCoord GridCoord GridCoord Grid Grid Grid @ @ @ Asia Asia Asia GridEcon GridEcon GridEcon GridTrust GridTrust GridTrust Edutain Edutain Edutain @ @ @ AssessGrid AssessGrid AssessGrid ArguGrid ArguGrid ArguGrid Grid Grid Grid Provenance Provenance Provenance SIMDAT SIMDAT SIMDAT BeinGrid BeinGrid BeinGrid BeinGrid A - Ware Sorma industrial industrial industrial business business business business simulations simulations simulations experiments experiments experiments experiments platforms, user platforms, user data, knowledge, data, knowledge, environments environments BREIN BREIN BREIN NextGRID NextGRID NextGRID Akogrimo Akogrimo Akogrimo semantics semantics XtreemOS XtreemOS agents & agents & agents & service service service mobile mobile mobile Grid4all Chemomen semantics semantics semantics Linux based KnowArc architecture architecture architecture services services services tum Grid QosCosGrid Gredia operating K - WF Grid InteliGrid CoreGRID system g - Eclipse GridComp virtual laboratories Datamining OntoGrid Grid UniGrids UniGrids HPC4U Specific support action Integrated project Network of excellence Specific targeted research project Courtesy: Max Lemke, European Commission European Context

6. Partner Research Centre Jülich (Project manager) Consorzio Interuniversitario per il Calcolo Automatico dell’Italia Nord Orientale Fujitsu Laboratories of Europe University of Warsaw Intel GmbH University of Manchester T-Systems SfR

7. Resource Broker Resource Database Target System A Network Job Supervisor Unicore Gateway Unicore Client Target System B Target System C User Database Technology Foundations • Develop Unicore Services • Job and file management • User authorisation • Dynamic resource discovery and management • Extend security to include explicit trusted delegation

8. Hosting Environment Build on Jetty Server and Axis SOAP engine WS Resource Framework ResourceProperties Lifetime BasicProfile ServiceGroup WS Notification NotificationProducer NotificationConsumer SubscriptionManager WS Addressing Address URL Resource Disambiguator Technology Foundations

9. Technology Foundations • Web Service Unicore Gateway • HTTP and HTTPS connector • Support for other protocols • Dispatches incoming SOAP messages based on • WS-Addressing headers or • URL information from the HTTP transport layer • Web Service Unicore User Database • Supports different storage formats – default XML file • Interface for secure, remote administration • Single UUDB can be used by multiple services

10. Technology Foundations Client • Multisite Job Internet Gateway Gateway Usite A Usite B Vsite A1 Vsite B1 Vsite B2 NetworkJobSupervisor NetworkJobSupervisor NetworkJobSupervisor TargetSystemInterface TargetSystemInterface TargetSystemInterface

11. Job SSL SSL SubJob Technology Foundations • Multisite Job • Consigner: entity that consigns a job or sub-job • Expressed by use in SSL connection • Endorser: entity that authorises the tasks to be performed • Expressed by signing of serialized AJO direct acyclic graph Primary Network Job Supervisor Client Secondary Network Job Supervisor = User certificate = NJS certificate

12. Technology Foundations Internet Client • Explicit Trust Delegation Gateway Usite A Portal Vsite A1 NetworkJobSupervisor TargetSystemInterface

13. SSL Job User: name SSL Technology Foundations • Explicit Trust Delegation • User: new role besides consignor and endorser • Entity (user) on whose behalf tasks will be performed • Trusted Agents (Portal) • Added to the UUDB explicitly • Allowed to endorse AJO on behalf of users WS- Client(Browser) Portal Network Job Supervisor = User certificate = Portal certificate

14. UniGrids Atomic Services Target System Factory TargetSystemService JobManagementService StorageManagementService FileTransferService Interoperability • UniGrids Atomic Services • Job Management • Storage Management • File Transfer • Target System • Standardisation • OGSA-BESBasic Execution Services • OGSA-DMIData Movement Interface GridClient GridPortal Unicore Globus CGSP

15. UniGrids Atomic Service Interfaces Globus Toolkit 4 Hosting Environment Globus Security Target System Factory TSF Target System Service TSS Job Mgmt Service JMS Storage Mgmt Service SMS File Transfer Service FTS GRAM/TSI GridFTP/GASS Interoperability • UniGrids Atomic Services for GTK4 • Available as Open Source (BSD license) at SourceForge (gpe4gtk.sourceforge.net)

16. ETD GSI Unicore/GS GTK4 Interoperability • Security • Client to server • Security provider concept allows to plug in security model on client side • GTK4 server to Unicore/GS server • Extract original user certificate from proxy certificate chain,pass to UGS (reduced security model) • Unicore/GS server to GTK4 server • UGS could use proxy certificate from ETD-enabled MyProxy server • Generic, standards-based VOMS preferable

17. Interoperability • Clients and GridBeans • GPE Clients work on top of Unicore/GS, GTK4 or ChinaGrid • Only requirement: UniGrids Atomic Services Interfaces • GridBeans are portable without modifications • Client API hides underlying infrastructure details GB GB GB GB GB GB GB Application Client Portal Client Expert Client

18. Interoperability • Grid Ontology • Cooperation with OntoGrid • Encoded in OWL, now published on the UniGrids web site (http://www.unigrids.org/ontology.html) • Permits reasoning and thus can fill in gaps in manually entered descriptions

19. Grid Economics • Economic Service Infrastructure • Full WSRF-Based Resource Broker • Use of extensible Economic terms for resource selection • Resource consumption feedback from Job Execution Engines • Resource Accounting System • Publication through WS Resource Properties • Payment System • Based on MCS Payment protocol • Uses broker as agent for user and/or their VO

20. Higher-level Services • Visualization and application steering • VISualization Interface Toolkit (VISIT) • Light-weight library that supports bi-directional data exchange between visualizations and parallel applications • VISIT GridBean • Manage participants of visualization sessions via VISIT / GS family • Framework works with all VISIT - enabled applications • Unique Unicore/GS feature for interactive scenarios

21. Higher-level Services • Unicore/GS Streaming Framework (UGSF) • Components • Web service • Streaming server • Client library • Architecture • Streaming Service is astand-alone application that performs streaming • Client uses dedicated protocol to connect to the streaming service and obtains real data stream • Web service uses RMI to communicate with streaming service and setup streams

22. Higher-level Services • File transfer service (including data filter) • Exploiting GridFTP • Client – Server transfers • Server – Server transfers • Including SRB • Client GridBean available

23. Higher-level Services • Access to distributed data, access to databases • Interface to OGSA-DAI resources • WSRF Data services deployed in Unicore/GS Container • OGSADAI GridBean • OGSADAI Client

24. Applications • Deploy test beds • Prove UniGrids concepts inindustrial and scientific domains • Oil reservoir modelling • Bio-molecular, genomic and proteomic • Nuclear risk management • Re-engineerindustrial applications using Unicore/GS WSRF approach

25. Standardisation • Influence standardisation process for selected Grid standards • JSDL • WS-RF • OGSA-BES • Promote European research results • Leverage partner participation in standards organisations • SAGA-RG • WS-N • OGSA-RSS

26. Beyond UniGrids • Grid projects exploiting UniGrids results • D-Grid (www.d-grid.de) • DEISA (www.deisa.org) • OMII-Europe (omii-europe.com) • A-WARE (www.a-ware.org) • Chemomentum (www.chemomentum.org) • EGEE-II (www.eu-egee.org) • UNICORE 6 Release Schedule • Alpha release ≡ UniGrids results: July 2006 • Beta release: mid 2007 • Final release: end 2007

27. A-WARE • Objectives • Simplify usage of Grid • Workflow management • User focused workflow orchestration • Repository to store workflows • Fill an existing gap • Between middleware and portals

28. Chemomentum • Objectives • Provide an integrated Grid solution for workflow-centric, complex applications;focus on the end users • Provide Grid-enabled applications, data services and knowledge management solutions • Risk assessment • Toxicity prediction • Drug design • Test-drive developed services in the context of the European regulatory initiative “Registration and Evaluation of Chemicals” (REACH) • Aimed at reducing animal testing, by developing in silico, Grid-based tools

29. OMII Europe • Objectives • Provide key software components for building e-infrastructures • Common interoperable services: • Database Access • Virtual Organisation Management • Portal • Infrastructure integration • Initial EGEE/UNICORE/Globus interoperability • Interoperable security framework • Accounting • Job Submission • Job Monitoring

30. Summary • UniGrids built Unicore/GS • Based on Unicore and the Open Grid Services Architecture • Compliant with the Web Services Resource Framework • UniGrids achieved interoperability • UniGrids developed generic software components • UniGrids results are the base of UNICORE 6 • UniGrids results are exploited by several Grid projects