Service Infrastructure within the ESA GS

1. Service Infrastructure within the ESA GS Pier Giorgio Marchetti – ESA / ESRIN +39 0694180413 Cell. +39 3485628804 Pier.Giorgio.Marchetti@esa.int Ground Segment Department

2. adding value to data…means supporting services Data Environment User Environment Mission Planning Acquisition Order Desk Catalogue Archiving Help Desk Processing Ordering Dissemination Quality Control Service Support Support 4 Service Providers Support 4 users Service Environment

3. Oxygen Ground Segment Model Quality Control User Services Help Desk Web Portal Mission Planning Catalogue Data Ordering CO-ORDINATION CENTRE SERVICES INFRASTRUCTURE Network Monitoring & Control Information Mining (future) Acquisition Archive Dissemination Production FACILITY

4. Services Infrastructure – From … Data Environment Data Environment Data Environment End User End User Service Provider Service Provider Distri-butor Value Adder Value Adder Value Adder User Envir. User Envir. Non EO Data

5. …To - Services Infrastructure Data Environment Data Environment Data Environment End User End User IIM IIM IIM IIM Service Provider Distri- butor Service Provider Value Adder Value Adder Value Adder User Envir. User Envir. Non EO Data

6. Service ???? • ServiceA Service is the repeated, regulated (usually by means of a Service Level Agreement) and agreed (via contract or agreement) or declared (with defined quality parameters) exploitation (for commercial or non-commercial purposes) of a function, processor or application. The output of a service includes data, products, information and solutions. • Web Services Please refer to W3C http://w3c.org

7. Service Infrastructure – the users Service Providers Service Consumers Users

8. Service Infrastructure – the interactions B2B Co-operation B2C Service Exploitation Service Providers Service Consumers Users

9. SI – high level requirements Empower Service Providers: Service Providers maintain full control over their own infrastructure “Neutral” service infrastructure Empower GS harmonisation Simple catalogue service Order service Generic service activation Shortcut data “format” issue: Concentrate on supply chain orchestration and flow control Let Users (and COTS) rule data format issues Online, nrt or offline service execution are all the same Foster long term research & industry growth rely on widely accepted and open standards (see OGC, W3C and ISO TCs) Support requirements from relevant actors and European Programmes EC-DG INFSO, JRC, EUSC GMES, INSPIRE,… B2B Co-operation B2C Service Exploitation Service Providers Service Consumers Users

10. SI – the functional objectives B2B Co-operation B2C Service Exploitation • Service support environment for the integration of Earth Observation and GIS supply chains • Support for data providers, service providers and end-users alike • Supply chain orchestration uses web services and workflow strds • Open and distributed architecture, allowing to add new supply chains • User driven/selected service delivery Service Providers Service Consumers Users

11. SI – the user… Multiple utilisation scenarios Service Provider co-operation Test and fast/cheap deployment of new supply chains Simplified user access … Multiple user types: anonymous user: can only activate free services access via the Global, Regional or Thematic Portal registered user: can activate free or paying services access via the Global, Regional or Thematic Portal service provider: can add new services access via the Global, Regional or Thematic Portal access via workflow client tools helpdesk, administrator access Admin functionality on the Global, Regional or Thematic Portal B2B Co-operation B2C Service Exploitation Service Providers Service Consumers Users

12. SI – the architecture … Global EO Service Infrastructure B2B Co-operation B2C Service Exploitation Service Providers Regional or Thematic Service Infrastructure Service Consumers Users

13. Legacy Service Legacy Service ServiceProvider Catalogue Service Toolbox Regional or Thematic Service Infrastructure ServiceProvider SOAP Catalogue Service SOAP Portal Service Infrastructure SOAP Workflows, service data, order data ... AOI Server HTTP HTTP OGC WMS Server(s) Mobile Client Web Browser User Workflow Editor OGC Gazetteer Server

14. Supply Chain Orchestration … 1 AOI selection(GIM) NPP*Service VITO Format Conversion Service GIM Clipping Service GIM NPP-Europe-HDF NPP-Europe-GeoTIFF NPP-Belgium-GeoTIFF1 NPP input + AOI description + required format (*) NPP or Net Primary Productivity represents repeatable estimates of the net flux of carbon. Product is based on Végétation S10.

15. Supply Chain Orchestration… 2 Combining service • GIS additional way to combine services. • Useful for vulnerability maps, hazard maps, rescue resources map, crisis map etc. Fire stations Roads Community boundaries NDVI Risk Map Fire risk map Digital Terrain Model LST Chaining Overlaying

16. Control flow : control flow : data flow : XML Order A (1) Portal & Workflow Engine Result A (8) MASS User Internet (4) (5) User FTP Server Result A1 Order A1 Result A2 Order A2 (2) (7) (3) (6) Service Provider A1 (e.g. VITO) Service Provider A2 (e.g. GIM) (9)

17. User Interaction • Select service of interest • Identify time and AOI (method of selection is chosen by SP) • Select parameters (e.g. format) • Get RFQ (if applicable) • Issue request • Check status of orders/requests • Get the output (delivery options identified by SP)

18. Selection of time, AOI and parameters

19. Check status of orders/requests

20. Get results (service output)

21. Oil spill detection (Spacetec)

22. Ship detection (Spacetec)

23. Simple Catalogue Service • Objective: Proof and test catalogue service within environment for the orchestration of EO services • Technology: Web Services, Workflow engine • Preliminary results: • Simple catalogue access and interoperability is made easy • Technological homogeneity makes integration of (different) service elements simple and their orchestration straightforward • Exploitation of state of the art OGC standards allows seamless integration into mapping services

24. Spot “search” results (GML) Envisat “search” results (GML) Catalogue search results (textual)

26. Service Provider • Enters in partnership with other service providers • orchestrates the supply chain defining which elementary services are part of it… • authorises users to activate “his” supply chain • publishes the services from “his” infrastructure (over which he maintains the full control) • monitors “his” service execution, state, on the workflow, on his infrastructure • is responsible for • defining and publishing “his” own SLA • monitoring and respecting “his” QoS; invoicing (where applicable)

27. Workflow console to monitor service instances Workflow editorto chain services

28. Interface vs. SP legacy Infrastructure EO Service Infrastructure Service Provider Legacy Infrastructure Toolbox Client SOAP / HTTP Legacy HTTP

29. The Toolbox Concept • Allows easy “publishing” of any service, with minimum assumptions on the SP available infrastructure (including NO infrastructure) • Provide a generic gateway toward service providers’ infrastructure • Generic/Customisable Service Interface: different Back End communication and interactions supported. • Customisation towards the SP infrastructure based on XML scripting

30. TOOLBOX Overview (1) • Based on free and Open Source software • No additional licences are needed • Front-End supporting the communication with MASS. • Based on Java and the SUN JWSDP • TOMCAT • JAXP (XML handling library) • It implements all the MASS interfaces • Order (synchronous and asynchronous) • Request For Quotation (synchronous and asynchronous) • Search (synchronous) • Present (synchronous) • Supports a generic service integration • Automatic WSDL creation

31. TOOLBOX Overview (2) • Back-end supporting the most common “communication mechanism” • FTP file transfer • HTTP • File exchange • Email notification • JDBC • Command and script invocation • SOAP over HTTP • Products delivery supported by an internal built-in FTP server

32. Service Integration on the Service Provider Side -1 • Service Configuration and test • Possibility to create and configure a service by means of a Web Application • Access control • A Simple SOAP client is provided for test purposes. • It includes a Catalogue test page. • On-line Service Definition Script validation.

33. Service Integration on the Service Provider Side - 2 • Service Execution • Triggered by SOAP messages coming from EO Service Infrastructure • Support for both synchronous and asynchronous communication mechanisms. • Service Monitoring and Control • Possibility to monitor the status of the service by means of a Web Application • Requests cancellation • Requests monitor • Service log Monitoring • Log based on XML • XSLT transformations.

34. Service Testing: SOAP client

35. Service Test: Simple Catalogue

36. Service Monitoring: Log

37. Service Definition Panel 2 - define interaction methods 1 - select service category 3 - describe service 4 - here the techs! 5 – register!

38. Operational Demonstration • Operational Demonstration : • Starts in October 2004 • >30 services • >15 service providers • Countries already involved: Belgium, France, Germany, Italy,(Norway), UK, Switzerland, The Netherlands, Portugal, … others • Open to any user and additional service providers

39. Conclusions… • Service Infrastructure facilitates: • Integration of wide range of heterogeneous EO and GIS services including catalogues (open to geospatial local/global data/services) • contacts between service providers and users more coherent (e.g. single service responsibility) • Working relationships among service providers via the flow control (basic cooperation environment) • SI empowers service providers • enabling to combine and chain service elements within the supply chain • Allows re-use generic or basic services • Efforts potential service providers limited to specific skills and added value • Allows to broaden and improve existing services • SI empowers users • offering services that are closer to the user’s expectations.

40. Conclusions… • Future work • Consolidate ongoing work on simple catalogue access and order management Interface Control Documents discussed with CNES, DLR, ASI, + others • Get feedback on open source TOOLBOX • Expand institutional cooperation at national and international level EC-DGINFSO, EUSC, JRC, … • Involve federating agencies EEA, … • Extend collaboration with standardization bodies OGC, ISO, … • Consolidate working relationships with relevant programmes GMES, GSE, … • ICD Documents & TOOLBOX available at http://earth.esa.int/rtd

41. Image Information Mining Coordination GroupASI, CNES, CNR, DLR, EC-IST, ESA, ETHZ, EUSChttp://earth.esa.int/rtd/IIMCG • IIMCG terms of reference • Charter • For Image Information Mining (IIM) applied to Earth Observation images, the IIMCG shall: • Promote European research and development of IIM techniques • Interface with European and National programmes in the field, like the O2 (Open and Operational) ESA initiative, the EC FP6 activities, etc. • Foster, within the members, the possibility to: • Share scientific and technical experience, data, information and applications • Identify relevant scientific and technical issues • Suggest technical directions • Suggest solutions for funding and coordination of activities • Provide a forum where interested bodies may: • Constantly be kept updated on ongoing activities in the field and on relevant scientific and technical issues • Exchange information (including gathering of suggestions and requirements) • Be informed on available demonstrators, prototypes, products, etc. • Identify and suggest / promote appropriate standards • Facilitate and promote the use of resulting products

42. Image Information Mining Coordination GroupASI, CNES, CNR, DLR, EC-IST, ESA, ETHZ, EUSChttp://earth.esa.int/rtd/IIMCG IIMCG terms of reference • Establish and maintain liaison with other interested bodies and with new sensors’ research and design (in particular for the identification of short / medium / long term needs) • Identify research and technological issues • Define possible strategies and solutions • Suggest activities and coordination methods • Suggest standards and interfaces for possible technologies