150 likes | 257 Views
Cluster Programming Technology and its Application in Meteorology. Computer and Automation Research Institute Hungarian Academy of Sciences. Hungarian Meteorological Service. Silicon Computers Ltd. Background.
E N D
Cluster Programming Technology and its Application in Meteorology Computer and AutomationResearch InstituteHungarian Academy of Sciences HungarianMeteorologicalService Silicon Computers Ltd.
Background • Hungarian Meteorological ServicedevelopedMEANDER (MEsoscale Analysis Nowcasting and DEcision Routines) • crucial task in the protection of life and property (storm warning at Lake Balaton, weather warnings for aviation...) • based on incoming meteorology information and computational intensive methods • MTA SZTAKI developed P-GRADE parallel programming environment • efficient, graphical support for the entire life cycle of parallel program development • Cluster programming technology and its applications in meteorology • joint project of MTA SZTAKI, Hungarian Meteorological Serviceand Silicon Computers Ltd. • supported by Research & Development Division, Ministry of Education
MEANDER Program Package • Goal: Analysis of all the available meteorology information • producing parameters on a high resolution regular mesh (10km--> 1km) • ultra-short range forecast (up to 6 hours) • Application: Forecasting dangerous weather situations (storms, fog, etc.) • Meteorology information: surface level measurements, high-altitude measurements, radar, satellite, lightning, results of previous computed models, etc. • Basic parameters: pressure, temperature, humidity, wind, … • Derived parameters: type of clouds, visibility, …
Structure of MEANDER First guess data ALADDIN SYNOP data Satellitedata Radar data Lightning CANARI DELTA analysis decode Basic fields: pressure, temperature, humidity, wind Radar to grid Rainfall phase Satellite to grid Derived fields: Type of clouds, visibility, etc. Visibility Overcast BASIC GRID Type of clouds Visualization “Present” weather For meteorologist:HAWK For users: GIF
Processing of satellite images Receiving image Satellite raw image 36000 km Transformation & Interpolation & Processing Processing:altitude of clouds Transformation Interpolation to basic GRID
THE PROBLEM MEANDER Parallelversion PC clasterSGI Origin 2000 SUN E10000 Parallelisation? MEANDER Sequentialcode C, C++ Fortran Debugging? Performance?
The solution: P-GRADE development environment GRP_LBload balancer PROVE performancevisualizationtool GRED graphicaleditor TLC modelchecker GRP_MMmigration module Macrostep debugger GRP_CHKPT GRMmonitor DIWIDE distributed checkpointingtool GRAPNELgraphical language/ GRP2C pre- compiler distributeddebugger PVM or MPI message passing library Unix-like operating systems &C, C++, Fortran programming libraries
2D analysis MEANDER: 3D FIELDS Results: temperature and wind at 850 hPa level Radardata Satellitedata Delta 3D analysis ...computes the basic meteorological fields: pressure, temperature, humidity, wind velocity and direction for a high resolution 3D mesh (10km -1km)
Implementation of DELTA analysis in P-GRADE Fortran seq. code
P-GRADE version of MEANDER for clusters & supercomputers 25 x 25 x 4 x 25 x 10 x 20 x
Live demo (5th DataGRID conference) P-GRADE PERL-GRID 11/5 Mbit Dedicated job ftp.met.hu netCDF 34 Mbit Shared job netCDF input GRM TRACE & Results GRM TRACE &Results 512 kbit Shared PERL-GRID CONDOR-PVM Parallel execution and GRM
Resource requirements of P-GRADE Edit, debugging Performance-analysis Testing,Execution
Advantages of P-GRADE environment • Efficient support for each stage of parallel program development • Fast parallelisation of existing algorithms • Reusability of sequential code • Hiding of low level communication functions • Unified and integrated graphical concept • Predefined communication templates • Support for hierarchical design • Even non-professional programmers can use it (steep learning curve) • Portability from supercomputers to PC clusters
Support for cluster computing III. Load balancing& migration • New facilities in P-GRADEfor long-running parallelapplications: • Distributed checkpointing • Process migration • Dynamic load balancing • Fault-tolerance execution More information: www.lpds.sztaki.hu I. Execution withoutload balancing IV. Result:Faster execution II. Checkpointing