1 / 89

Grid-based System for Flood Forecasting

Grid-based System for Flood Forecasting. Ladislav Hluchy Institute of Informatics SAS in co-operation with Water Research Institute, Vah River Authority and Slovak Hydrometeorological Institute Slovakia hluchy.ui@savba.sk. Outline. Introduction Flood Forecasting

zora
Download Presentation

Grid-based System for Flood Forecasting

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Grid-based System for Flood Forecasting Ladislav Hluchy Institute of Informatics SAS in co-operation with Water Research Institute, Vah River Authority and Slovak Hydrometeorological Institute Slovakia hluchy.ui@savba.sk Cracow Grid Workshop ’03,27-29.10.2003

  2. Outline • Introduction • Flood Forecasting • Grid infrastructure for Flood Forecasting • Use cases • Grid-based Implementation • Results • Conclusion Cracow Grid Workshop ’03,27-29.10.2003

  3. Flood Forecasting one of the Geospatial Applications • Applications that use data from Geographic Information System (GIS) • Typical applications: flood forecasting, fire simulations, environmental risk management etc. Cracow Grid Workshop ’03,27-29.10.2003

  4. Flood Forecasting • Topical problem: floods have caused widespread damages in the recent years • Common interest: many countries threatened • Many potential users: governments, flood crisis teams, insurance companies, public, • Requires Grid technology Cracow Grid Workshop ’03,27-29.10.2003

  5. ANFAS ANFAS Architecture Cracow Grid Workshop ’03,27-29.10.2003

  6. ANFAS Integration to ANFAS core Machine 1 ANFAS Core Server currently hosted by EADS-MS&I Use of the i-cluster in the ANFAS system Web Machine 2 hosting RPS Controller i-cluster LAN Cracow Grid Workshop ’03,27-29.10.2003

  7. ANFAS SMS/FESWMS • FESWMS has been developed under funding by the U.S. Federal Highways Administration (FHWA) • FESWMS is specifically suited for modeling regions involving flow control structures, such as are encountered at the intersection of roadways and waterways. Specifically, the FESWMS model allows the user to include weirs, culverts, drop inlets, and bridge piers into a standard 2D finite element model. • As there is highway planned at the Vah River pilot site in Slovakia, the choice of FESWMS model is important • SMS provides graphical tools for defining these structures and controlling analysis using the FESWMS model. Both pre- and post-processing capabilities are included in the interface. Cracow Grid Workshop ’03,27-29.10.2003

  8. ANFAS Solution schema Nonlinear solver Input files Finite element Generating matrix Linear solver Nonlinear solver write solution to the file Update solution OK Solution file Detailed FESWMS structures Cracow Grid Workshop ’03,27-29.10.2003

  9. ANFAS Generating partial matrix Generating partial matrix Generating partial matrix Updating solution Updating solution Updating solution Parallel matrix generation PARALLEL LINEAR SOLVER Cracow Grid Workshop ’03,27-29.10.2003

  10. ANFAS Implementation issues Real flood modeling software is much more complicated than its mathematical model: • Mathematical model of flood is well-known (partial differential equations  finite elements  nonlinear solver  linear solver) • Real software has to deal with • Input processing: different types input data, different variations of each type, different formats of each variation • Special cases: wetting/drying, raining/evaporation, special constructions (bridges, dams, culverts), wind effect, … • Calibration of results • Graphical user interface (GUI), visualisation • Error checking, documentation As the result, source code of real software may be hundreds times longer than source code of mathematical model Cracow Grid Workshop ’03,27-29.10.2003

  11. ANFAS Remote processing Processing input data Pre-processing Parallel computational kernel Post-processing Save solutions Remote processing Cracow Grid Workshop ’03,27-29.10.2003

  12. ANFAS Planned highway in the Váh pilot site Cracow Grid Workshop ’03,27-29.10.2003

  13. ANFAS Main part affected by highway Bytca city Predmiervillage LIDAR+highway position Cracow Grid Workshop ’03,27-29.10.2003

  14. ANFAS Predmier village in orthophotomap Cracow Grid Workshop ’03,27-29.10.2003

  15. ANFAS Predmier village in LIDAR Cracow Grid Workshop ’03,27-29.10.2003

  16. ANFAS TIN network at Predmier Cracow Grid Workshop ’03,27-29.10.2003

  17. ANFAS Scenario: Water level for current terrain situation (Q-100-year) Water depth Cracow Grid Workshop ’03,27-29.10.2003

  18. ANFAS Scenario: Water level for highway without bridges (Q-100-year) Water level is about 70cm higher than for situation without highway Water depth Cracow Grid Workshop ’03,27-29.10.2003

  19. ANFAS Scenario: Water level for highway with 2 bridges (Q-100-year) Water level is about 30cm higher than for situation without highway Water depth Cracow Grid Workshop ’03,27-29.10.2003

  20. Why Grid? • Cooperation: requires cooperation between many organizations (meteorological institutes, river authorities) from many countries • Data management: needs large amount of data of different sources, different owners, different countries, different access right • Computation power: forecasting require large computational power for modeling and simulation Cracow Grid Workshop ’03,27-29.10.2003

  21. Virtual Organization • Purpose • Shared data and computational power for flood forecasting • Cooperation between users for flood forecasting • Requirements • Identify and define clear relationships between users • Authentication: certificate authorities • Authorization: access right for each data/resources • Collaborative tools • Security Cracow Grid Workshop ’03,27-29.10.2003

  22. Data sources meteorological radars • External sources of information • Global and regional centers GTS • EUMETSAT and NOAA • Hydrological services of other countries surface automatic meteorological and hydrological stations systems for acquisition and processing of satellite information Storage systems databases High performance computers Grid infrastructure meteorological models hydrological models hydraulic models Users Flood crisis teams • river authorities • energy • insurance companies • navigation • meteorologists • hydrologists • hydraulic engineers • media • public Flood Forecasting VO Cracow Grid Workshop ’03,27-29.10.2003

  23. Data sources GIS Fuel type - vegetation - canopy cover GIS Topography -elevation -slope Meteorological data source Weather - wind direction, speed - temperature, rel. humidity FIRE MODEL Descriptive Numerical Parameters Grid infrastructure Users Fire Management Creation Decision Support system, prevention -Terrain, resources, - capacities Fire suppression authorities - training - operation mode Ecosystem authorities Universities, Insurance companies Virtual Organization for Fire Simulation High performance computers Storage systems databases Fire Modelling System FARSITE Cracow Grid Workshop ’03,27-29.10.2003

  24. Data management • Typical data: satellite images, radar images, measured data from hydrological stations, topographical data, historical data, simulation results • Different formats, different quality, different owners, different access right • Metadata server:data description, security, replication Cracow Grid Workshop ’03,27-29.10.2003

  25. Data sources meteorological radars • External sources of information • Global and regional centers GTS • EUMETSAT and NOAA • Hydrological services of other countries surface automatic meteorological and hydrological stations systems for acquisition and processing of satellite information High performance computers meteorological models hydrological models hydraulic models Storage systems Users Flood crisis teams • river authorities • energy • insurance companies • navigation • meteorologists • hydrologists • hydraulic engineers • media • public Grid infrastructure Databases FloodVO data transfer Cracow Grid Workshop ’03,27-29.10.2003

  26. DataGrid • EDG Replica Manager • EDG Local Replica Catalogue • EDG Replication Metadata Catalogue • EDG Replica Optimization Service Cracow Grid Workshop ’03,27-29.10.2003

  27. Storage control EDG RM Metadata EDG RMC EDG ROS Storage Element Storage Element Storage Element EDG LRC EDG LRC EDG LRC DataGrid (cont.) Cracow Grid Workshop ’03,27-29.10.2003

  28. Grid computing • Many multidisciplinary simulations are needed for flood forecasting • For critical situations, short response times are very important • Numerical simulations are computationally intensive • Grid can offer the necessary computational power Cracow Grid Workshop ’03,27-29.10.2003

  29. Visualization • Data are stored in many different formats • Unified visualization tools may simplify the user-interface • Many data for flood forecasting has spatial character => GIS software may be used as the unified visualization tool Cracow Grid Workshop ’03,27-29.10.2003

  30. Portal • The unified user-interface • Allow users access to the VO remotely • Simple requirements on clients - based on standard Web technologies Cracow Grid Workshop ’03,27-29.10.2003

  31. 3 current portals • Based on GridPort using Globus grid toolkit • Based on Jetspeed portal framework using DataGrid/CrossGrid services • Migrating Desktop - java fat client using DataGrid/CrossGrid services Cracow Grid Workshop ’03,27-29.10.2003

  32. GridPort • A set of Perl scripts that enable Perl based portal (its CGI scripts) to use grid services of underlying Globus toolkit • Wraps Globus’ command line tools • Provides session management • Provides no additional portal infrastructure Cracow Grid Workshop ’03,27-29.10.2003

  33. Resource 1 User’s web browser Globus toolkit (GSI, MDS, JobManager, GridFTP, …) Portal (Apache web server) GridPort toolkit … Resource 2 User’s web browser … Storage Resource n Storage & Portal Machine Architecture of GridPort based portal Cracow Grid Workshop ’03,27-29.10.2003

  34. GridPort screenshot Cracow Grid Workshop ’03,27-29.10.2003

  35. Job submission in Flood-VO XML file (parameter description) Config. file (default values of parameters) New config file globus-job-submit machine job_script config_file Job script file Cracow Grid Workshop ’03,27-29.10.2003

  36. Flood-VO: Job list Cracow Grid Workshop ’03,27-29.10.2003

  37. Time Value 00:00:00 102.00000 cm 01:00:00 126.00000 cm 02:00:00 103.00000 cm 03:00:00 80.00000 cm 04:00:00 70.00000 cm 05:00:00 65.00000 cm SHMI II SAS RDBMS Flood-VO: Field data Cracow Grid Workshop ’03,27-29.10.2003

  38. Jetspeed • Portal framework • Server-side Java based engine (application server) • Client services are plugged using software components called portlets. • User can arrange portlets – position, size, visibility Cracow Grid Workshop ’03,27-29.10.2003

  39. Jetspeed - architecture Cracow Grid Workshop ’03,27-29.10.2003

  40. Application portal screenshot(Jetspeed) Cracow Grid Workshop ’03,27-29.10.2003

  41. Application Portal Cracow Grid Workshop ’03,27-29.10.2003

  42. Migrating Desktop (MD) • Java application (applet) running at the client computer • Provides interface to all basic grid services (authentication, job management, file management) • Application specific job parameter input and job submission is supported via application plug-ins • has built-in viewer for common picture formats (jpeg, gif, png) and text files, advanced visualization of results via application specific visualization plug-in • Being developed in the context of the CrossGrid project Cracow Grid Workshop ’03,27-29.10.2003

  43. Migrating Desktop Screenshot of MD with Job submission wizard dialog Cracow Grid Workshop ’03,27-29.10.2003

  44. Use case: Cascade simulation Data sources Meteorological simulation Hydrological simulation Hydraulic simulation Portal Cracow Grid Workshop ’03,27-29.10.2003

  45. Model characteristics • ALADIN (meteorological model) • Limited area model • Operated by 13 Euro-Mediterranean countries • ALADIN/SLOVAKIA operated by SHMI • More than 1M lines of source code (mainly F90) • Developed for 64 bit big-endian architecture • Proprietary - requires nondisclosure agreement Cracow Grid Workshop ’03,27-29.10.2003

  46. Model characteristics • ALADIN (meteorological model) • Type: MPI parallel task, possible parameter studies – multiple executions • CPU time: approximately one hour on 8 processors • I/O size: 33/180 MB per run • Scalability: on fast Ethernet up to 8 processors • Input data: boundary conditions • Output data: quantitative precipitation forecast, temperature Cracow Grid Workshop ’03,27-29.10.2003

  47. Model characteristics • HSPF (hydrological model) • Type: sequential task, multiple executions (high throughput computing) • CPU time: very small (seconds - minute) • I/O size: 1-10 MB • Scalability: HTC • Input data: quantitative precipitation, temperature, topographical data • Output data: hydrograph Cracow Grid Workshop ’03,27-29.10.2003

  48. Model characteristics • FESWMS (hydraulic model) • Funded by US Federal Highway Administration • Distributed in commercial package SMS by EMS-I • Source code available (direct cooperation with developer) • Optimized and parallelized by II SAS Cracow Grid Workshop ’03,27-29.10.2003

  49. Model characteristics • FESWMS (hydraulic model) • Type: MPI parallel task, multiple executions with different input data • CPU time: 10min to several hours per a task • I/O size: 10-100 MB • Scalability: good for smaller number of processor (to 16). • Input data: inflow, topographical data • Output data: water levels and velocities Cracow Grid Workshop ’03,27-29.10.2003

  50. DaveF model • A time-explicit finite-volume model from the same developers as FESWMS. It is considered as the complement of FESWMS and it is best suitable for unsteady state with critical or super-critical flow (dam-breaking, flash flood, flood with wetting/drying in large expanses) • DaveF uses the same graphical environment like FESWMS (SMS) and similar input/output format =>can be easily added into ANFAS system • Parallel version of DaveF has been developed for clusters by II-SAS and shows good results Cracow Grid Workshop ’03,27-29.10.2003

More Related