NOWCASTING WITH NEURAL NETWORK USING REFLECTIVITY IMAGES OF METEOROLOGICAL RADAR

1. Indaiatuba/SP, may,24-05, 20:30UTC – F3 NOWCASTING WITH NEURAL NETWORK USING REFLECTIVITY IMAGES OF METEOROLOGICAL RADAR Palmital/SP, may,25-04, 17UTC – F3 C. A. Thompson (2) R. Machado (1,2) R. V. Calheiros (1) Lençóis Paulista/SP, may,25-04, 20UTC – F2 (1) Meteorological Research Institute (IPMet)/UNESP, Bauru, SP, 17033-360, Brazil (2) Polytechnic Institute (IPRJ)/UERJ, Nova Friburgo, RJ, 28601-970, Brazil INSTITUTO DE PESQUISAS METEOROLÓGICAS

2. OBJECTIVES & STATUS • GENERAL : SUPPORT TO OPERATIONAL NOWCASTING IN CENTRAL SÃO PAULO • SPECIFIC : IMPLEMENT A NEURAL NETWORK APPROACH TO IPMET’S OPERATIONAL FORECASTING PRACTICES • AS OF NOW: PRELIMINARY TESTS OF NETWORK PERFORMANCE RUN FOR DISTINCT COMPUTATION OF AVERAGES ON STATISTICAL TEXTURE DESCRIPTORS INSTITUTO DE PESQUISAS METEOROLÓGICAS

3. DATA & AREA • PRODUCT: REFLECTIVITY CAPPIS AT 3,5 KM HEIGHT AGL, TO A 240 KM RANGE FROM THE BAURU RADAR (BRU) • PERIOD: SUMMERS OF 2002/2003 & 2003/2004 • DATA SET: 300 IMAGES GATHERED IN INTERVALS OF 2 HOURS EACH, COMPOSING TWO SUB SETS: CHARACTERIZED BY (1) RAIN AND (2) NORAIN SITUATION AT THE END OF THE TIME INTERNAL • DATA SAMPLE: • TARGET AREA: 15 KM RADIUS CIRCLE AROUND THE RADAR INSTITUTO DE PESQUISAS METEOROLÓGICAS

4. PROCESSING • A) • STATISTICAL TEXTURE DESCRIPTORS (ATTRIBUTES), I.E. MEAN, STD DEVIATION, SKEWNESS AND KURTOSIS WERE COMPUTED FOR EACH IMAGE • AVERAGES OF THE ATTRIBUTES WERE CALCULATED FOR ALL IMAGES WITHIN EACH 2 H INTERVAL • RESULTED TWO SETS OF ATRIBUTE VECTORS: ONE CORRESPONDING TO THE PROCESSING OF EACH IMAGE, AND THE OTHER FOR THE AVERAGE VALUES OF EACH ATTRIBUTE (SEE TABLE 1) INSTITUTO DE PESQUISAS METEOROLÓGICAS

5. TIME EVENT ATTRIBUTE 12:53 13:23 13:53 14:23 AVERAGES TABLE 1 : ATTRIBUTES FOR A SAMPLE EVENT 30-JAN-2004 FROM THE DATA BANK ( ALL TIMES LT = UTC – 3) MEAN 0.1529 0.2059 0.2629 0.2675 0.2223 STD DEVIATION -1.5224 1.8106 1.9487 2.0532 1.8337 SKEWNESS -0.1604 -1.3625 -1.9867 -1.8567 -1.3415 KURTOSIS 9.1118 8.1791 6.2289 5.2513 7.1927 OBS.: STATISTICS WERE COMPUTED ON THE IMAGE PIXELS IN mm.h-1 DERIVED WITH Z = 300R1,4 INSTITUTO DE PESQUISAS METEOROLÓGICAS

6. m ö S æ P a = d ÷ > 0 ç ø i è = i 1 • B) • ATRIBUTE VECTORS WERE USED AS INPUTS TO A NEURAL NETWORK CONSTITUTED OF 2 LAYERS, WITH 4 NEURONS IN THE HIDDEN LAYER AND 1 NEURON IN THE OUTPUT LAYER • LINE COMMAND WAS: net2=newff(minmax(p),[4,1],{‘logsig’,’logsig’},’traingda’); • Newff = NETWORK WITH BACK-PROPAGATION • [4,1] = TWO LAYERS, 4 NEURONS IN THE HIDDEN LAYER AND 1 NEURON IN THE OUTPUT LAYER; AND • logsig = TRANSFER FUNCTION OF EACH NEURON, DIFFERENTIABLE, WITH OUTPUT BETWEEN 0 AND 1. • RESULTS OF RUNS WITH DIFFERENT NEURAL NETWORKS ARE DEMONSTRATED FOR TWO OF THEM • TRAINNING WAS EFFECTED FOR OUTPUT VALUES BETWEEN O AND 1, AS:IF OUTPUT ≥ 0.5 → RAIN, OUTPUT< 0.5 → NO-RAIN • (OBS. AN AREA PROBABILITY OF RAIN, a, IF δi IS AN INDICATOR VARIABLE EQUAL TO 1 WHEN RAINS OCCURS AT A POINT 1, AND ZERO, IS • FOR BRU, ECHO STATISTICS INDICATES a ~ 0.55 FOR SUMMER). INSTITUTO DE PESQUISAS METEOROLÓGICAS

7. FIRST NETWORK • TRAINING: PERFORMANCE RESULT INSTITUTO DE PESQUISAS METEOROLÓGICAS

8. SIMULATION: ATTRIBUTES FOR EACH IMAGE WERE USED. THE FIRST 15 IMAGES WERE KNOWN TO RESULT IN RAIN, AND THE LAST 15 IMAGES IN NO-RAIN, TABLE 2 – RESULTS AT THE OUTPUT OF THE FIRST NETWORK FIRST 15 IMAGES (VALUES SHOULD BE ≥ 0.5) 0.8305 0.7978 0.789 0.7882 0.2381 0.2316 0.3901 0.421 0.897 0.7415 0.5778 0.6096 0.895 0.8603 0.8477 LAST 15 IMAGES (VALUES SHOULD BE < 0.5) 0.1595 0.1132 0.24 0.322 0.2745 0.2205 0.2051 0.193 0.2726 0.3201 0.303 0.2559 0.3032 0.3114 0.2616 ERRORS (VALUE IN RED) = 4/30 ≅ 13%  87% OF SUCCESS INSTITUTO DE PESQUISAS METEOROLÓGICAS

9. SECOND NETWORK (TRAINING NOT SHOWN) SAME ATTRIBUTES AND CRITERIA FOR RAIN (≥ 0.5) AND NO-RAIN (< 0.5 ) AS THE FIRST NETWORK, BUT USING AVERAGES OF THE ATTRIBUTES TAKEN OVER EACH 2H INTERVAL. TABLE 3 – RESULTS AT THE OUTPUT OF THE SECOND NETWORK FIRST 15 IMAGES (VALUES SHOULD BE ≥ 0.5 0.7375 0.6675 0.8093 0.8564 0.4218 0.6752 0.8411 0.9048 0.566 0.8166 0.972 0.8733 0.9503 0.9561 0.9485 LAST 15 IMAGES (VALUES SHOULD BE < 0.5 0.3749 0.348 0.0304 0.166 0.0223 0.0519 0.2077 0.1444 0.054 0.0261 0.2231 0.108 0.081 0.2816 0.263 ERRORS (VALUES IN RED) = 1/30 ≅ 3%  97% OF SUCCESS INSTITUTO DE PESQUISAS METEOROLÓGICAS

10. CHI – SQUARE TEST • TWO SAMPLES (2 H INTERVALS) TAKEN. FOR THE FIRST x1=22, nI =30 AND FOR THE SECOND: x2=79, n2=90, WHERE xi=1,2 = RAIN, n i=1,2 = SAMPLE SIZE. • NULL HYPOTHESIS IS FORMULATED, I. E. RAIN/ NO - RAIN RELATIONS ARE TRUE. x i = 1,2 IS A RANDOM VARIABLE. MODELING ITS BINOMIAL DISTRIBUTION BY A NORMAL DISTRIBUTION, THE PROPORTION OF THE SAMPLE TAKEN BY x(θ = X/ n) IF UNKNOWN, IS • FOR THE MODELED NORMAL DISTRIBUTION , • IF THE RANDOM INDEPENDENT VARIABLES z1 AND z2 HAVE STANDARD NORMAL DISTRIBUTION, THEN (y= z2 + z2) HAS A CHI-SQUARE DISTRIBUTION WITH m DEGREES OF FREEDOM. • COMPUTING Y WITH THE ABOVE NUMBERS : θ = 0.84 y = 3.495 • FOR α = 0.05 AND υ(m) = 2 DEGREES OF FREEDOM = 5.991(>3.495) • NULL HYPOTHESIS IS SATISFIED TO 95% OF CONFIDENCE, I. E., FORECASTS ARE NOT BIASED. ^ ^ ^ ^ INSTITUTO DE PESQUISAS METEOROLÓGICAS

11. NEXT STEPS • IMPROVEMENT OF PERFORMANCE • A . 1) ADD NEW TECHNIQUES, E. G. • GABOR FILTERING AS A FIRST LAYER IN THE NETWORK SYSTEM TO EXTRACT TEXTURAL FEATURES, WHICH WILL FEED THE INPUT LAYER OF THE FORECASTING NETWORK • (GABOR FILTERING HAS SHOWN TO IMPROVE THE PERFORMANCE OF NEURAL NETWORKS) • FUZZY LOGIC, DUE TO THE FACT THAT THERE IS NO CLEAR SEPARATION BETWEEN SEASONS, DAILY INTERVALS, AND OTHER STRAFICATION FACTORS. • GENETIC ALGORITHMS, WHICH USE TECHNIQUES OF BIOLOGICAL DERIVATION THAT COULD BE APPLIED TO RAINFALL CONFIGURATIONS SUCH AS : HERITAGE ( RAIN AT T0 IS RELATED TO T0 – 1), MUTATION ( RAIN PATTERNS CHANGE STRUCTURE IN TIME), NATURAL SELECTION (PREFERENTIAL DEVELOPMENT CONDITIONS EXIST), AND RECOMBINATIONS ( RAIN CELLS SPLIT AND MERGE IN TIME) INSTITUTO DE PESQUISAS METEOROLÓGICAS

12. A.2) ADD NEW (METEOROLOGICAL) ATTRIBUTES, E. G. • DOPPLER RADAR WINDS • SATELLITE IMAGES (VIS, IR, WV & MW) INDIVIDUALLY OR IN COMBINATIONS TO INFER, E. G. RAIN/NO - RAIN THRESHOLD. • VARIABLES, LIKE TEMPERATURE, PRESSURE, HUMIDITY. INSTITUTO DE PESQUISAS METEOROLÓGICAS

13. B) VERIFICATION/VALIDATION COMPARISONS WITH OTHER NOWCASTING TECHNIQUES EITHER IN TESTS OR OPERATIONAL, OR IN CONSIDERATION FOR OPERATIONAL USE, AT IPMET FORECASTING SECTOR. B.1) TITAN (THUNDERSTORM IDENTIFICATION, TRACKING, ANALYSIS AND NOWCASTING) PREDICTING ECHO CENTROID POSITION EVOLUTION. STATUS: UNDER OPERATIONAL EVALUATION B.2) KAVVAS (ADAPTIVE EXPONENTIAL METHOD) PREDICTING SHORT-TERM EVOLUTION (15 MIN. TO 2 H) OF CENTROID, BASED ON REFLECTIVITY AND VELOCITY (DOPPLER). STATUS: UNDER STUDY B.3) VIL (VERTICALLY INTEGRATED LIQUID WATER CONTENT) PREDICTOR IS WATER COLUMN FROM GROUND TO 12 KM AGL COMBINED WITH PRESENCE OF 45 dBZ ABOVE 3 KM. STATUS : OPERATIONAL INSTITUTO DE PESQUISAS METEOROLÓGICAS

14. CONCLUSIONS • NEURAL NETWORK APPROCH TO RADAR BASED NOWCASTING IN CENTRAL SÃO PAULO HAS SHOWN CLEAR POTENTIAL. • STATISTICAL TEXTURE DESCRIPTORS HAVE PROVEN A VALID INPUT TO THE NOWCASTING WITH NEURAL NETWORK IN CENTRAL SÃO PAULO. • IMPROVEMENTS RESULTING FROM AVERAGING DESCRIPTOR VALUES INDICATES THAT EVEN RELATIVELY MINOR OPERATIONS ON IMAGE CHARACTERISTICS CAN SIGNIFICANTLY IMPACT NETWORK PERFORMANCE. • FURTHER IMPROVEMENTS SHOULD BE PARTICULARLY EXPECTED FROM TEXTURE CLASSIFICATION THROUGH GABOR FILTERING. INSTITUTO DE PESQUISAS METEOROLÓGICAS