D-Grid Marcel Kunze Steinbuch Centre for Computing (SCC)

1. D-GridMarcel KunzeSteinbuch Centre for Computing (SCC)

2. The Three Waves of D-Grid 2003 - 2008 2008 - 2010 2010 - 2015 The Research Wave The Industry Wave The Consumer Wave Technology, Prototypes Grid-Enabled Products Commodity Virtual Organizations Enterprise Solutions IT Utility, Web2.0 Standards Interoperability Integration OGF, IETF, OASIS OGF, IETF, OASIS Legal, Ethical, Political Orgs Finally, Grids will “disappear”

3. D-Grid in a Nutshell Building a national e-Infrastructure for research and industry • 01/2003: Pre-D-Grid Working Groups  Recommendation to Government • 09/2005: D-Grid-1: early adopters, ‘Research Grids’ • 07/2007: D-Grid-2: new communities, ‘Service Grids’ • 04/2008: D-Grid-3: Service Grids for research and industry • D-Grid-1: 25 MEuro > 100 Orgs > 200 researchers • D-Grid-2: 35 MEuro > 100 addl Orgs > 200 addl researchers and industry • D-Grid-3: Call expected in the comíng weeks (~20 MEuro)

4. D-Grid-1 Integration Project User-friendly Access Layer, Portals Astro-Grid C3-Grid HEP-Grid IN-Grid MediGrid Textgrid WISENT ONTOVERSE WIKINGER Im Wissensnetz . . . . . . Generic Grid Middleware and Grid Services

5. D-Grid-1, -2, -3 Integration Project User-friendly Access Layer, Portals Astro-Grid C3-Grid HEP-Grid IN-Grid MediGrid Textgrid WISENT ONTOVERSE WIKINGER Im Wissensnetz Business Services, SLAs, SOA Integration, Virtualization . . . . . . Knowledge Management Generic Grid Middleware and Grid Services

6.  D-Grid in 2010  User-friendly Access Layer, Portals Astro-Grid C3-Grid HEP-Grid IN-Grid MediGrid Textgrid WISENT ONTOVERSE WIKINGER Im Wissensnetz Business Services, SLAs, SOA Integration, Virtualization . . . . . . Virtual Competence Centers for Resources, Middleware,Support, Training, Knowledge Services Level Agreements Generic Grid Middleware and Grid Services

7. D-Grid Middleware User Application Development and User Access GAT API GridSphere Plug-In UNICORE Nutzer High-levelGrid Services SchedulingWorkflow Management Monitoring LCG/gLite Data management Basic Grid Services AccountingBilling User/VO-Mngt Globus 4.0.1 Security Resourcesin D-Grid DistributedCompute Resources NetworkInfrastructur DistributedData Archive Data/Software

8. Current D-Grid Infrastructure (~20.000 Cores, 5 PB Storage)

9. HEP-Grid: p-p collisions at LHC at CERN Event rate Level 1 Trigger Rate to tape Crossing rate 40 MHz Event Rates: ~109 Hz Max LV1 Trigger 100 kHz Event size ~1 Mbyte Readout network 1 Terabit/s Filter Farm ~107 Si2K Trigger levels 2 Online rejection 99.9997% (100 Hz from 50 MHz) System dead time ~ % Event Selection: ~1/1013 Luminosity Low 2x1033 cm-2 s-1 High 1034 cm-2 s-1 Data analysis: ~1PB/year “Discovery” rate

10. AstroGrid

11. C3 Grid: Collaborative Climate Community Data and Processing Grid Climate research moves towards new levels of complexity: Stepping from Climate (=Atmosphere+Ocean) to Earth System Modelling Earth system model wishlist: Higher spatial and temporal resolution Quality: Improved subsystem models Atmospheric chemistry (ozone, sulfates,..) Bio-geochemistry (Carbon cycle, ecosystem dynamics,..) Increased Computational demand factor: O(1000 -10000)

12. InGrid: Virtual Prototyping & Modeling in Industry Fluid-Structur/ Magneto-Hydro- dynamic Interaction Molding Metal Forming Fluid Processes Groundwater Transportation Gridspezifische Entwicklungen Methods and models for solving engineering problems in Grids Grid-specific developments Knowledge-based support for engineering-specific decision support Distributed simulations-based product & process optimization Support for engineering-specific Workflows Security and trust models Cooperation and business models Integration project AP 3 AP 4 AP 2

13. Raw Data Cell Organ/Tissue Patient Population Illness Molecule Metadata Metadata Metadata Metadata Metadata Metadata Access Control Search, Find, Select Homogenization Target data Correlate, Process, Analyze Resulting Data Presentation Final Result MediGrid: Mapping of Characteristics, Features, Raw Data, etc

14. TextGrid • Modular Platform for distributed cooperative scientific • text processing for the humanities • Workbench for Publishing, Text Processing, Text Retrieval, Linking, and Workflow • Focus of TextGrid is on: - Quantity: Full text instead excerpts, text and pictures - Speed of reprography: stability of text, volatility of medium - Precision: Maximum requirements on correctness - Availability: international open standards

15. WISENT, Knowledge Network for Energy Meteorology • Charakterisation of energy fluctuation from solar and wind energy • Results through interdisciplinary collaboration (Meteorology) • Transformation of wind and solar energy (physics) • Flexible distributed systems (computer science)

16. D-Grid-2, Start June 2007 • ‘Horizontal’ Service Grids:professional Service Providers for heterogeneous user groups in research and industry • ‘Vertical’ Community Service Grids using existing D-Grid infrastructure and services, supported by Service Providers • D-Grid extensions, based on a D-Grid 1 gap analysis - Tools for operating a professional grid service - Adding business layer on top of D-Grid infrastructure - Pilot service phase with service providers and ‘customers’

17. D-Grid-2 Projects • BauVOGrid: Grid-based platform for VOs in the Construction industry • FinGrid:Financial Business Grid: A service grid architecture for the financial service industry • AeroGrid: Grid-based collaboration among Aerospace research & industry • ProGrid:Automotive: Cooperative product design and development in simulation and production data management • BIS-Grid: Grid-based Enterprise Information Systems, integration & orchestration in commercial IT systems • Biz2Grid:Enterprise Grids; Acct, Pricing, Billing, Metaschedlg; ERP/SAP • PartnerGrid:Cooperation in industrial VOs (SME, supplier, svc providers) • GDI-Grid:Geographical Data infrastructure for providing and processing data and simulation for catastrophes, noise, and navigation • F&L-Grid: Grid-based IT services for research and education • D-MON: Horizontal integration of resource and service Monitoring • SUGI: Grid support for small institutions and SMEs

18. BauVOGridGrid-based platform forvirtual organisations in the construction industry

19. BIS-Grid Overview Utilisation of Grid technologies to integrate decentralised business information systems

20. BIS-Grid Outcome Horizontal Service Grid • New collaboration and business models for Grid Providing and utilisation • Especially aiming on SMEs • Utilisation of WS-BPEL to orchestrate stateful WSRF-conform Grid Services • Development of a Grid Service-enabled WS-BPEL engine • Integrated with UNICORE 6 • Evaluation of results in partner-specific application scenarios

21. BIS-Grid Application Scenario Enterprise Application Integration using Grid technologies • Different sites • Extensive amounts of data and legacy data • Interrelations of data from different sources/databases

22. Financial Business Grid 22

23. Research Approaches & Research Objectives Research of market acceptance and willingness for diffusion Development and implementation of diverse pricing models for efficient resource allocation in Grid-Environments Specification of service-orientedGrid-Implementation approaches Requirement profile of Business Grid-Applications Requirement profile of Business Grid-Applications • Prototype 3 • Asset-backed Securities Factory • Prototype 2 • Customer Portfolio Performance Measurement • Prototype 1 • Grid Ressource Pricing Tool Design Science Workshops Reports Publications Constructivist System Engineering Approach Development of 3 Grid-Prototypes Development of aGrid-Economy Field Studies Case Studies Positivistic FinGrid Standard Project Presentation Release: 2007-08-10 FinGrid 23

24. Geo-Data Infrastructure Grid • Enhancement of existing GDI applications with Grid technologies: • meet enormous need for processing power • allow access to distributed data sources and huge data storage • combine current base technologies of spatial data infrastructures (SDI) with Grid middleware (GT4) • enable seamless processing of spatial data in Grid infrastructures • Development of generic services for the • integration • fusion • management and • processing of spatial data on the D-Grid infrastructure • Survey of user requirements, feasibilityand marketable potential  Spatial Data Infrastructure Grid GDI-Grid 24

25. Scenarios and Proof of Concept Three selected scenarios for the evaluation of the implemented GDI-Grid components: • spatial simulation of flood disasters - flood hazard  Spatial Data Infrastructure Grid GDI-Grid 25

26. Scenarios and Proof of Concept Three selected scenarios for the evaluation of the implemented GDI-Grid components: • spatial simulation of flood disasters - flood hazard • noise dispersion simulation for noise pollution predictions  Spatial Data Infrastructure Grid GDI-Grid 26

27. Scenarios and Proof of Concept Three selected scenarios for the evaluation of the implemented GDI-Grid components: • spatial simulation of flood disasters - flood hazard • noise dispersion simulation for noise pollution predictions • route optimization for disaster management  Spatial Data Infrastructure Grid GDI-Grid 27

28. PartnerGrid: Project Goal Development of Grid platform that supports the collaboration within a virtual organisation: • Definition and execution of complex and distributed process flows • User interface for access and use of the Grid via portal technology • Transparent access to distributed data (file systems, data bases) • Management of distributed, virtual organisations and integrated security concepts • Interfaces for integration of complex, application specific analysis modules • Interface to service providers for the execution of compute intensive application • User-transparent use of D-Grid services • Secure and liable procurement of resources to the users considering resource provider and resource user • Resource usage is charged by an accounting and billing function.

29. PartnerGrid Architecture

30. PartnerGrid: Application scenarios Scenario: Simulation of a founding process • Use case optimisation • Usage of Grid-resources with parallelism in two dimensions • Connection of the optimisation-workflows to the grid computing • Use case enterprise scenario • Deployment of Grid computing in a big enterprise • Use case outsourcing • MAGMASOFT-Users can use external hardware-, software- and license resources, that are provided by a independent service provider (MAGMA GmbH) Scenario: Forming process technique • Use case „project processing between customer, supplier and service provider“ • more efficient project work, reduction of the project management cost • Use case „provision of compute resources and support“ • better utilisation of hard- & software-resources, consulting service, extension of the service range

31. D-Grid Summary Building a national e-Infrastructure for research and industry • An attractive infrastructure has been set up • Increasing number of projects and Grid users • Impact of new developments like Cloud Computing? • Important: • Operate a sustainable production grid infrastructure after the end of the funding • Integration of new communities • Evaluating business models (operational models) for grid services

32. Acknowledgments • Slides and information have been provided by • Wolfgang Gentzsch • D-Grid Projects