Computational engineering services for all

1. Computational engineering services for all LNEC’s experience as a INCD/IBERGRID/EOSC user Anabela Oliveira (LNEC)

2. Overview • LNEC’s activity in a e-infrastructure context • The INCD project: goals and opportunities for researchers • INCD’s E-Services for the engineering communities: a fast overview • OPENCoastS - Coastal circulation on-demand forecast platform • WorSiCa - Water mOnitoRingSentInel Cloud platform • Future directions and how to participate

3. Field monitoring Scalemodels LNEC’s activity: tools and expertizes • Develops service-oriented research, integrating multiple tools • Supportsdecision-making, intertwinning science and engineering Numericalmodels DecisionSupportSystems Strong requirements: • Computational power • Storage

4. The INCD project: goals and opportunities for researchers • INCD e-infrastructure provides computing and storage services to the national scientific and academic community in all areas of knowledge. • (some of) INCD’s project goals: • Maintain and enhance a world-class computing distributed infrastructure • Support flagship science and the participation in large international projects • Build and operate research community-wide services - pilot activities together with user communities

5. The INCD project: e-services for engineering Thematic services Model-based services Data-based services In between... Conda-based Data Science framework WorSiCa Water mOnitoRing SentInel Cloud platform SWMM + Basement integrated modelling in urban systems and rivers Coastal biogeochemical observatories Adaptation tool for climate change impacts in cities INCD-ERNZEH – energy efficiency in hotels (EnergyStar) AI integration for Bridge modeling and monitoring

6. The OPENCoastS platform in a nutshell • Implement forecast systems for a system chosen by the user, using a browser-based, user-friendly, interface • Allow the choice of the processes, model and forcings • Allow the replication and change of forecast systems • Avoid the need of a large team to generate forecast systems • Take advantage of the European Open Science Cloud (EOSC) to provide the required computational resources (EOSC-hub project) • Access: https://opencoasts.ncg.ingrid.pt/

7. What do we get with the service OPENCoastS? • Advantages • Easy to apply to all (no need for IT, coastal processes ormodeling experts…) • Maintenanceandextentionofforecastsis free • Everytimeweupdatetheplatform (newmodels, newfeatures,…), itbecomesavailable to all • Provisionofcomputationalresourcesisavailable (INCD, Ibergrid, EOSC,…) • AutomaticcomparisonwithEMODnetPhysics’s data is embedded in the platform features • EOSC(-hub) makes this service available at its marketplace: • https://marketplace.eosc-portal.eu/services/opencoasts-portal

8. OPENCoastS: data needs and link to global and regional services DATA ATMOSPHERE OCEAN • WIFF EMODnetPhysics FES2014 NOAA / GFS CMEMS METEO-FRANCE / ARPEGE DGT PRISM2017 - Uses WIFF framework - Water Information Forecast Framework - Predictionsmadewithmodeling suite SCHISM (community, open sourcemodelwith a largeworldwidecommunity) - Developed to accommodateothermodels in the future Computationalgrid Parameters Riverflow USER

9. OPENCoastS: architecture Theuseronlyinterateswiththe web interface, sothiscomplexityistransparent!

10. Web interface 3 components: configure, manage and view results • Configuration assistant: building a deployment step by step • Outputs viewer: • visualize results (maps, time series) • Adding data/model points on the fly • Saving time series and model outputs in your PC • Compare time series from several deployments • Forecast manager : what can we do with our forecasts: check status, check details, clone it, freeze/restart it, delete it,…

11. Configuration assistant • Step 1: Selectthemodel • Step 2: Uploadandvalidategrid • Step 3: Specifyboundaryconditions • Step 4:Define output stations • Step5: Define physicalandnumericalparameters • Step 6: Define space-varyingparameters • Step 7: Reviewandsubmit

12. Step 2: Uploadandvalidatecomputationalgrid • Grid format: SCHISM/SELFE/ADCIRC • WGS84 (long-lat) recommended. If not, then the EPSG must be selected • Vertical reference: MSL

13. Step 3: Specifying the boundary conditions • Elevation at sea boundaries, river at freshwater boundaries • After defining boundaries, the provider for ocean and atmosphere forcings is chosen from a list • Global forcings are provided for worldwide service coverage, regional inputs are also available for Western Europe for higher accuracy

14. Step 7: Reviewandsubmit • Download input files • Possibility to go back to previous steps and make changes • 5 simultaneous deployments per user are provided Goes to manager

15. Forecast manager: tasks • Check status and configuration of each deployment • Clone– duplicate a deployment for scenarios or parameter/b.c. sensitivity analysis • Re-activate a pauded system or eliminate it • Go back to the configuration assistant and finish the deployment

16. Viewer • Example: elevation and velocity maps and time series in La Rochelle • New time series on-the-fly • Compare time series from several deployments • Download output files and run logs

17. WorSiCa - Water mOnitoRingSentInel Cloud platform • Web platform for Earth-Observationwater-relatedservices • Twomainservices are included: • WADI - water leak detection service • CorSiCa – CoastalmonitoringSentInelCloudplAtform Automatic detection of water leaks in large water distribution networks Automatic detection of inundation areas

18. E-Service WADI Racionale: • Water losses in large distribution networks constitute a major concern • Large water losses occur due to long leak detection times • Difficulty to pinpoint the leak location Goals: • surveillance service for leaks information on water infrastructures outside urban areas, using manned and unmanned aerial platforms • Reliability service: satellite data is used to confirm a possible leak detection An innovative and reliable water leak detection service supported by data-intensive remote sensing processing Ricardo Martins, Alberto Azevedo, André Fortunato, Elsa Alves, Anabela Oliveira LNEC – National Laboratory for Civil Engineering (www.lnec.pt) Alexandra Carvalho EDIA - Empresa de Desenvolvimento e Infraestruturas do Alqueva (www.edia.pt) INTERNATIONAL CONFERENCE ON COMPUTATIONAL SCIENCE (ICCS) Faro, June 12-14, 2019 This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 689239

19. LNEC’s Leak-detection reliability procedure • Take advantage of historical, high-frequency Sentinel-2 satellite images • Possible leak analysis is performed on water indexes computed from the remote sensing images. • The reliability procedure is being validated in the Alqueva infrastructure (Portugal) and will be also applied to the Societé Canal de Provence (France) • Can be applied anywhere Water index image, computed from the remote sensing imagery ICCS, Faro, June 12-14 2019

20. Methodology for the water leak detection – Step 1: Climatology Workflow • automatically check for available Sentinel-2 imagesets that cover all or part of the user-specified AoS, and processes them. For each imageset: • Download itfrom ESA repository • Convert to L2A (ifnecessary) • Cropimagesetsand compute water indexes • calculate climatology and generate final product Select Area of Study Climatology CPU+storage Process all available imagesets for that area Available Imagesets 1) Download 3) Compute 2) Convert Merge processed products Calculate climatology The processed MNDWI products for the same day (e.g: 08/08) of each year and its climatology. ICCS, Faro, June 12-14 2019

21. Methodology for the water leak detection – Step 2: Leak Detection Workflow 1) Process water indexes: User selects a Sentinel-2 image by specific date to determine leak presence, service automatically process water indexes (exactly the same procedure as described previously) 2) Calculate anomaly: Difference between the image for the user-selected date and the climatology image for the closest day available in the climatology dataset 3) Calculate Laplacian: Apply the second derivative to anomaly to determine possible leaks, which is estimated by finite differences 4) Identify leaks The leaks’ locations are then confronted with the water network location CPU+storage ICCS, Faro, June 12-14 2019

22. Web interface for the reliability service • A web-portal interface was developed to facilitate the usage of the reliability methodology. • Developed in Django, a Python web framework to create web applications with a frontend and a backend. ICCS, Faro, June 12-14 2019 22

23. Web interface for the reliability layer - Architecture • Frontend: • send user requests to backend • Backend: • processes the frontend requests and checks the ESA’s repository for Sentinel-2 image sets. • embedded raster layer service to load and store the processed images as rasters. • Processing service: • is asynchronous (uses Celery) and interacts with the backend to provide the processing status • use toolbox for image processing: Sen2Cor, GPT, GDAL ICCS, Faro, June 12-14 2019 23

24. Conclusionsand future work • Availabilityof core e- infrastrutures are fundamental assets to developcommunityengineering e-servicesbothatEuropean (EOSC), Iberian (Ibergrid) or nacional (INCD) levels • OPENCoastSisoperationalandavailable to allat: https://opencoasts.ncg.ingrid.pt/ • Remotesensingservicewillbeintegrated in EOSC-Synergy for more applications (inundationareasdetection) and open to all • Otherengineeringservices are providedby INCD (energyefficiency in hotels; cityclimateadaptationservices,…) • Ifyou are interested in receivingupdatesorparticipate in thedevelopingefforts: aoliveira@lnec.pt

25. Teams LNEC: JoãoRogeiro, Ricardo Martins, Joana Teixeira, Alberto Azevedo, André Fortunato, Marta Rodrigues, Pedro Lopes LIP: Jorge Gomes, Mário David, João Pina, Samuel Bernardo Université de La Rochelle: Xavier Bertin, Laura Lavaud Universidad de Cantabria: Sonia Castanedo, Fernando Mendes