590 likes | 659 Views
L2OS threshold optimisation 20 June 2014, PM26. JL Vergely, J. Boutin, P. Spurgeon ACRI-ST,LOCEAN, ARGANS. RFI/outlier detection. Aim : To improve the thresholds of the L2OS processor to be applied on TB measurements in order to remove outliers. About thresholds :
E N D
L2OS threshold optimisation 20 June 2014, PM26 JL Vergely, J. Boutin, P. Spurgeon ACRI-ST,LOCEAN, ARGANS
RFI/outlier detection Aim : • To improve the thresholds of the L2OS processor to be applied on TB measurements in order to remove outliers. About thresholds : • should be independent on L1c quality products
Thresholds to be tested test for outlier detection (dwell test) nsig test for out of range TB detection (FOV test) Tm_out_of_range_affov Tm_out_of_range_eaffov Tm_out_of_range_stokes3_affov Tm_out_of_range_stokes3_eaffov Tm_out_of_range_stokes4_affov Tm_out_of_range_stokes4_eaffov test for oscillation TB detection (FOV test) Ts_std Ts_std_stokes3 Ts_std_stokes4 Other tests : max of iteration
Tests conditions |X_swath| < 400 km Coast : 1000 km PCT_var < 80 -40° < lat < 40° SSS ref: Coriolis global NRTOA (MyOcean) Day : 1,2,3,4,5/5/2013, L1C v550 L2OS proc : v600 (CATDS processing chain)
Indicators / SSS quality filter • chi2P : good TB fit if chi2P high. Chi2P > 0.05 in current processor. Warning : Dg_chi2P in L2OS processor = 1-Chi2P • SSS error < 1.4 psu • mean(SSS SMOS – SSS Coriolis) and std(SSS SMOS – SSS Coriolis) • X = (SSS SMOS – SSS Coriolis)/SSS_error. X should be close to a Gaussian law with mean(X)=0 and std(X)=1. Does not depend on SSS accuracy (close to the ratio between empirical error and theoretical error).
Chi2P and RFI Percentage of RFI contamination : january 2012, asc Chi2P, 5/5/2013, asc
nsig : full ocean Outlier detection. Dwell test. TB removed if : |TBsmos – TBmodel – DA| > nsig.rad_noise DA = mean dwell correction Current value : 5 Tested values : 2, 3, 4, 5
nsig full ocean 4 sigmas test Queue distribution : outliers Expected distribution Centred reduced variable
nsig full ocean Mean and std(X) nsig = 2 No specific filter Chi2p > 0.05 & sigSSS < 1.35
nsig full ocean Mean and std(X) nsig = 3 No specific filter Chi2p > 0.05 & sigSSS < 1.35
nsig full ocean Mean and std(X) nsig = 4 No specific filter Chi2p > 0.05 & sigSSS < 1.35
nsig full ocean Mean and std(X) nsig = 5 No specific filter Chi2p > 0.05 & sigSSS < 1.35
nsig full ocean nsig=2: Many outliers at 4 sigmas
nsig coast Expected distribution
nsig coast Mean and std(X) nsig = 2 No specific filter Chi2p > 0.05 & sigSSS < 1.35
nsig coast Mean and std(X) nsig = 3 No specific filter Chi2p > 0.05 & sigSSS < 1.35
nsig coast Mean and std(X) nsig = 4 No specific filter Chi2p > 0.05 & sigSSS < 1.35
nsig coast Mean and std(X) nsig = 5 No specific filter Chi2p > 0.05 & sigSSS < 1.35
nsig coast nsig=2: Many outliers at 4 sigmas nsig=2 : very biased !!
Tm_out_of_range_affov or eaffov (polar X,Y,3,4) Snapshot removed if at least one TB is an outlier : |TB smos – TB model| > Tm_out_of_range Problem because the test is applied directly on the TBs and not on the TBs normalised by the radiometric noise X and Y from short and long integration time Current value : 50 K for affov and 100 K for eaffov Tested value : 10, 20, 30, 40 K
Tm_out_of_range_affov full ocean Tm = 10K No specific filter Chi2p > 0.05 & sigSSS < 1.35
Tm_out_of_range_affov full ocean Tm = 40K No specific filter Chi2p > 0.05 & sigSSS < 1.35
Tm_out_of_range_eaffov full ocean Tm=10K : Little bit better but lost of accuracy No significative change (with Tm_out_of_range_affov = 12)
Tm_out_of_range_stokes3_affov full ocean Tm=6K : Little bit better -> try to work in dual pol mode ? No significative change (with Tm_out_of_range_affov/eaffov = 12/18)
Tm_out_of_range_stokes3_eaffov full ocean No significative change (with Tm_out_of_range_stokes3_affov = 8)
Tm_out_of_range_stokes4_affov full ocean No significative change (with Tm_out_of_range_stokes3_affov/eaffov = 8/16)
Tm_out_of_range_stokes4_eaffov full ocean No significative change (with Tm_out_of_range_stokes3_affov/eaffov = 8/16 & Tm_out_of_range_stokes4_affov = 10)
Ts_std thresholds Snapshot is removed if : rms((TB smos –TB model)/ra) > Ts_std rms((TB smos –TB model)/ra) is expected to be close to 1 (if OTT well applied) Current value = 2.5
Ts_std full ocean Ts_std=1.5 : Little bit better
Ts_std_stokes3 full ocean Ts_std=0.5 or 1 : Too low Too biased No significative improvement
Ts_std_stokes4 full ocean Ts_std=1 : Too low Too inaccurate No significative improvement
Comparison current configuration and configuration without thresholds
Comp current/without thres. full ocean current conf Chi2p > 0.05 & sigSSS < 1.35
Comp current/without thres. full ocean without filter No signicative change Chi2p > 0.05 & sigSSS < 1.35
Comp current/without thres. full ocean A little bit better without thresholds
Comp current/without thres. coast Without thresholds : better for 4 sigmas SSS
5 day processing 1,2,3,4,5/05/2013; nsig = 2, 3, 4, 5 Dwell test. TB removed if : |TBsmos – TBmodel – DA| > nsig.noise DA = mean dwell correction Current config : nsig = 5
Iteration number 2 modes !!
Iteration number Signature TEC ? RFI or island ? Hot spot
Global improvement using iterMax Small global effect. What about specific area ?
4 zones with RFI/coast contamination Pacific + coast Pacific Atlantic Indian ocean
4 zones with RFI/coast contamination Pacific + coast Pacific Atlantic Indian ocean