Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2

1. Interfaces EO data with Atmospheric and Land Surface Model: Progress report 2 Liang Feng, Paul Palmer

2. Project Tasks Reference OSSE system High-resolution model CO:CO2 ratios

3. Observing System Simulation Experiment Tool I. Progress overview Overall Aim:Developing a reference OSSE simulation for community researchers to evaluate impacts of space-borne atmospheric composition measurements on surface flux estimates. Outputs: 1). OSSE framework with flexiable module and libraries. Status: Done. 2). One complete example OSSE systemfor 1)simulating OCO-like XCO2 observations; and 2) estimating regional CO2 fluxes by assimilating XCO2 observations using an Ensemble Kalman Filter (EnKF). Status: Done . 3. Detailed documents. Done. 4. EOF and Visualization Tools. Done.

4. II. Download and Installation • Codes and documents: • 1) Packages for PyOSSE and EOF visualizations can be downloaded from http://xweb.geos.ed.ac.uk/~lfeng • 2) Installation guides has also been provided, covering how to: • unzip archive; • set python search path; • build shared libraries; • test codes. • 3) documents and references can be found on-line or from local • directory. • Data • Being required mostly by the example OSSE system. • Covering climatology for cloud, aerosol PDF , satellite orbit, instrument averaging kernel, and GEOS-Chem model outputs. • Being stored as ASCII text files, netCDF files and BPCH files in www.esa-da.org/data/otool_data.tar.gz.

5. III. PyOSSE framework • Data flow and directory structure • Classes • IO Modules • Configuration files

6. Surface flux Ensemble CTM Ensemble forecasts (3-D fields) Obs operator Model Obs Ensemble • Data flow and directory structure Flux Forecasts Observation Simulation EnKF assimilation CTM Forecast (3-D concentrations) Prior + error Posteriori + error Obs operator Obs +random errors ETKF

7. PyOSSE package has over 90 python/fortran modules, stored in 8 subdirectories: util (lib) atmosphere (CTM outputs) surface (fluxensemble) observation (sample/convolve) instrument (ORB, AVK. CLD …) enkf (stv and etkf solvers) example (OSSE for OCO)

8. Classes: • We have defined about 40 python classes as containers for data and functions of model objects Atmosphere Transport Flux model inventory Met files Surface flux Top-down estimate Observation Measure

9. Example: class for state vector class ens_stat_cl: Class for state vector used to assimilate observations Members: (total of 44) ------------------------------------------------------------- 1. attr_dict:<dict>: attributes 2. step_tag_lst:<list>: ID for run step # variables for current assimilation window 3. wnd_mean_x0:<array, (wnd_nx)>: prior coefficient values for lag window 4. wnd_mean_x:<array, (wnd_nx)>: posterior coefficient values for lag window 5. wnd_dx:<array, (wnd_nx, wnd_ne)>: perturbations ensemble for coefficients 6. wnd_inc_m:<array, (wnd_ne>: increment matrix for ETKF data assimilation 7. wnd_xtm: <array, (wnd_nx, wnd_ne)>: transform matrix for ETKF data assimilation 8. wnd_xinc: <array, (wnd_nx)>: increment for ETKF data assimilation # current analysis increments 9. inc_m:<array, (wnd_nx)>: increment matrix for current step 10.xtm: <array, (wnd_nx, wnd_ne)>: transform matrix for current step 11. xinc: <array, (wnd_nx)>: increment for ETKF data assimilation Functions (total of 14) ========================================================== 1. __init__: initialization 6. construct_tcor_matrix: Create temporal correlation matrix 7.add_new_x_to_window: Add new apriori to assimilation window 10.do_assim: assimilate observations and update state

10. Configurable IO modules • PyOSSE relays on many pre-defined data sets such as satellite orbit, instrument sensitivity, and cloud PDF etc. • PyOSSE needs mechanism to exchange data with CTM. • Data is usually given in different formats, and often needs pre-processing. • Configurable IO modules are designed as bridges between host classes and data files. • These modules are easy to be re-configuredor even be replaced. Host class IO module Disk files Var-list Var-dict fdesc fopen fclose fwrite fread data Def-Var-List Def-Var-dict Create-fdesc Openfile Readfile writefile Closefile .

11. Configuration files • We have developed a new type of text-based menu files to: • Numeric or string parameters as function inputs (traditional). • Object as parameters such as list, functions (new) • Object creater(new) Example: vob_def.cfg for observation simulations allows users to choose classes for satellite orbit, averaging kernel, cloud/AOD PDF , and sampling etc #MENU (cloud) #================================================ name|cloud path|$DATAPATH$/clim_dat/ flnm|meancloud_frac_XMONTHX_landsea.dat dict|__load:$MDPATH$.cloud_file_m:cld_varname_dict fopen|__load:$MDPATH$.cloud_file_m:open_cloud_file fread|__load:$MDPATH$.cloud_file_m:read_cloud_file fclose|__load:$MDPATH$.cloud_file_m:close_cloud_file fget|__load:$MDPATH$.cloud_file_m:get_cloud_data keywords|__dict:Nil fclass|__load:$MDPATH$.cloud_m:cloud_cl #MEND

12. Menu term (sample) in vob_def.cfg #MENU (data random sampling) #======================================================= name|sample # cloud sampling fcld_sample|__load:$MDPATH$.cloud_m:cloud_sample fcld_penalty|None fcld_keywords|__dict:Nil # AOD faod_sample|__load:$MDPATH$.aod_m:aod_sample faod_uplimit|0.3 faod_keywords|__dict:Nil # landcover flc_sample|__load:$MDPATH$.landcover_m:sample_lc flc_stype_lst|__load:$MDPATH$.ak_file_m:ak_lc_stype_lst flc_keywords|__dict:Nil #MEND

13. Menu term (averaging kernel ) in vob_def.cfg #MENU (averaging kernel) #===================================================== name|ak path|$DATAPATH$/oco/oco_ak/ flnm|aknorm_XSURFACEX.XEXTX viewmode_dict|__load:$MDPATH$.ak_file_m:def_ak_viewmode_dict stype_dict|__load:$MDPATH$.ak_file_m:def_ak_surf_type_dict fopen|__load:$MDPATH$.ak_file_m:open_ak_file fread|__load:$MDPATH$.ak_file_m:read_ak_file fclose|__load:$MDPATH$.ak_file_m:close_ak_file fget|__load:$MDPATH$.ak_file_m:get_ak_data keywords|__dict:Nil fclass|__load:$MDPATH$.ak_m:ak_cl #MEND ……

14. IV. OSSE Example system • Observation simulation • Inversion

15. Observation simulation (example/obs_simulation): Generate dummy observation by using gen_dummy_obs.py (and vod_def.cfg) Generate model observation by sampling actual model outputs (gen_sat_obs.py and sob_def.cfg)

16. GEOS-Chem outputs can easily be replaced by user’s CTM simulations by changing sob_def.cfg #MENU (class for model profile) #================================================ name|fc_prof # model output path|$DATAPATH$/gc_std/ # file name (empty to use defaults in ) flnm|'ts_satelliteXEXTX.XYYYYXXMMXXDDX.bpch' fdiaginfo|'diaginfoXEXTX.dat' ftracerinfo|'tracerinfoXEXTX.dat' vname|single_profile ext|'.ST001.EN0001-EN0002' # output access fopen|__load:ESA.util.gc_ts_file_m:open_ts_file fread|__load:ESA.util.gc_ts_file_m:setup_daily_profile fget|__load:ESA.util.gc_ts_file_m:get_mod_gp fpres|__load:ESA.util.gc_ts_file_m:get_mod_pres fclass|__load:ESA.atmosphere.ts_slice_m:gc_slice_cl #MEND

17. Inversion (example/obs_simulation): 1. Settings: default settings are defined in osse_def.cfg, configuring classes for state vector, flux, perturbation ensemble, prior flux, observations, model forward simulation, and model outputs sampling etc. Example: CTM defined in osse_def.cfg #MENU (CTM for single tracer run) name|ctm runpath|$FCRUN_ROOT$ datapath|$FCRUN_DATAPATH$ tra_st|1 tra_end|2 finput_gen|__load:ESA.example.enkf_oco.input_geos_gen:create_new_input_file ext|ST001.EN0001-EN0002 runtype|3 runscript|rungeos.sh fclass|__load:ESA.example.enkf_oco.run_geos_chem:geos_chem_cl #MEND

18. 2) Results: We have a rich set of outputs for inversion results as well as diagnostic data For example: Posterior fluxes

19. For example: Posterior model XCO2

20. V. EOF and visualization • The pyeof module • Visualization package

21. pyeof • Pyeofdefines a class eof_cl to find EOFs for temporal and spatial data sets. • It uses SVD technique to find eigenvalues and eigenstate for covariance matrix. • It is easy to use, and also provides functions for visualization of EOFs and PCs EOF1

22. Visualization • We have developed a wxpython-based GUI application for quick visualizations including 2D plots and animations. • It also has a python shell to use a rich set of existing visualization tools. • We have also provided functions for common data processing.

25. VI Conclusion • OSSE tool and visualization package are ready for public release. • This framework is simple and flexible, due to the adopted algorithm (EnKF), and the way to implement it (classes, configurable IO modules, and enhanced menu functions …). • It has been successfully used to digest GOSAT observations. • We are making further developments.

26. B. Hi-resolution modeling

27. I. Motivation • Interpret and digest future satellite (like OCO-2) observation of atmospheric CO2 concentrations: • Model errors (transport and representation errors) have significant impacts on top-down estimates of surface fluxes. • Higher model resolution usually means better transport modelling, and smaller representation errors. • Provide benchmarks for coarser simulations or nested model simulations. • Itself is one by-product of our efforts to develop EnKF approach based on nested GEOS-Chem transport model. • In-cooperate other process-based models (such as plume model) into GEOS-Chem (Gonzi et al, University of Edinburgh). • Due to the finer temporal and spatial resolutions, It is more sensitive to the details of ‘small scale’ processes.

28. II. Challenges: • Availability of High-resolution emission inventories • 1x1 monthly fossil fuel emissions (Oda et al). • 1x1.25 3-hourly CASA biospheric CO2 exchanges (R. Kawa, J. Collatz, and D. Liu) • 1x1.25 daily biomass burning (R. Kawa , J. Collatz, and D. Liu) • 4x5 monthly oceanic surface CO2 flux (Takahashi et al)

29. Programming issues Memory: In particular, stack size is limited by most of fortran compiler. Modifications have been made over GEOS-Chem v8.02 to limit the use of common block and avoid static arrays in modules. Parallelisation: Mainly we have to reschedule parallelisation in tpcore modules, and change functions in dao_mod.f . Minor issues: Mainly they are associated with diag outputs.

30. Results • Comparison with standard GEOS-Chem 4x5 simulations CO2 /ppm(ML1-8) CO2/ppm (ML1-8) Range: 374 – 415 ppm Range: 377 ppm – 403 ppm

31. Comparison with nested simulation Nested 0.5x0.666 Global 0.5x0.666 CO2 /ppm(ML1-4) CO2 /ppm(ML1-4) Nested model being used in UK GAUGE project

32. Comparison with GEOS-Chem drived by ECMWF winds GEOS-5 winds ECMWF winds CO2 /ppm(ML1-8) CO2 /ppm(ML1-8) ECMWF Met fields may have stronger vertical transport across boundary layer.

33. Comparison to averaged aircraft profiles --- GEOS-5 PBL TABATINGO SANTAREM N = 7 N = 11 ALTA FLORESTA RIO BRANCO N = 11 N = 11

34. Summary • Native resolution simulation provides: • More realistic descriptions of observed variations • Benchmark for lower-resolution model • Framework for including detailed process models • Our comparisons also reveal some possible model errors. • We are now working on flux inversions for selected regions • We have done some OSSEs for Astrium Project • We are working on digesting ICOS data.

35. Spare slides

36. Data Assimilation Module (Ensemble Kalman Filter) • Top-down Optimal surface flux estimation model gain observation Priori Posteriori x: regional surface fluxes yobs: measurements of atmospheric concentrations. H: Jacobian (CTM model). K=PfHT(HPfHT+R)-1– Kalman gain matrix. Pf: a priori uncertainty matrix R: observation error matrix

37. Key features: • (Feng et al., 2009 ; 2011) • Represent a-priori uncertainties by an ensemble of flux perturbations. • )T • Use a lag windows to limit computational costs. • Any emission will only be constrained by observations within a following limited time period. After that period, it is considered to be well-known. • Project the ensemble of perturbations together with prior estimates into the observation space using CTM. • Use Ensemble Transform Kalman Filter (ETKF) to determine posterior fluxes, and the associated uncertainties from digesting observations. • Ensemble Approach

38. A. State vector and ensemble representation of its uncertainties Surface fluxes Prior estimates of surface fluxes Basis functions for pulse-like flux perturbations, :coefficients to be estimated: State vector: ,, , …,]; Perturbations:, ,, …,] Error covariance: Full or partial representation: , ,, …,],

39. We use region_map_c.py to defined as 8-day surface CO2 flux perturbations over 144 global regions. We use gen_region_pb_flux_m.py to generate BFs from biosphericGPP map and multi-layer maps for 144 regions.

40. B. Ensemble forecasts • We change IO function in pb_flux_c.py to generate the ensemble of flux perturbations according to the defined basis functions and pre-defined uncertainties. • Functions in ctm_config_c.py and ctm_restart_c.py are override to support GEOS-Chem tagged runs to project flux perturbation ensemble to the ensemble of atmospheric tracer concentrations. • C. Projection of ensemble forecasts to the observation space (ctm_world_c.py)

41. D. Inversion algorithm:EnsembleTransform Kalman Filter (ETKF) • A posteriori and the associated uncertainties are simultaneously calculated using SVD (or sparse-matrix LU) technique. • +R]-1 • T • TTT=[1+-1 • Corresponding updates are made to the ensemble of model 3D concentrations to retain the contributions of fluxes outside the assimilation (time) windows(Feng et al, 2009)

42. Algorithm testing: Comparison with the LSCE 4d-var system In the comparison experiments, we have assimilated the same ACOS-GOSAT B210 XCO2 retrievals over 2009 and 2010, but using different approaches: EnKF (UoE) and 4d-var (LSCE)