Jörg Trentmann

1. Technical Introduction Jörg Trentmann Satellite Application Facility on Climate Monitoring (CM SAF)Deutscher Wetterdienst

2. Overview • Introduction to • CM SAF data • Software tools (cdo, R) • Example scripts • Suggestions/ideas for exercises

3. Introduction Are you planning to use CM SAF data during the CM SAF Event Week? No(but will useCM SAF data later) Yes No(no plans to useCM SAF data)

4. Introduction Are you a registered user of CM SAF? Yes No Have you ever ordered CM SAF data through the Web User Interface (WUI: www.cmsaf.eu/wui)? Yes No

5. CM SAF data files How does a CM SAF data file look like, have you ever seen a CM SAF data file?? Yes No

6. CM SAF data files The file names of CM SAF data follow a certain naming convention:

7. CM SAF data files The file names of CM SAF data follow a certain naming convention:

8. CM SAF data files Example:

9. Data Formats hdf(http://www.hdfgroup.org/HDF5/) • supports additional data compression • commonly used in satellite data community • provides a standard • not very common • only limited software available to access the data • complex to read data

10. Data Formats hdf(http://www.hdfgroup.org/HDF5/) netcdf (http://www.unidata.ucar.edu/software/netcdf • supports additional data compression • commonly used in satellite data community • provides a standard • not very common • only limited software available to access the data • complex to read data • growing number of meteorological / climatological users • wider range of software available to access the data, incl. GIS • standard for climate data available • no data compression, larger files

11. Data Formats hdf(http://www.hdfgroup.org/HDF5/) netcdf (http://www.unidata.ucar.edu/software/netcdf It is recommended to order CM SAF data in netcdf-format via the Web User Interface (WUI)! • supports additional data compression • commonly used in satellite data community • provides a standard • not very common • only limited software available to access the data • complex to read data • growing number of meteorological/climatological users • wider range of software available to access the data, incl. GIS • standard for climate data available • no data compression, larger files

12. Software tools • The free, open-source climate data operators (cdo) and the statistical software tool R are the main tools supported to analyse CM SAF data. • cdo: https://code.zmaw.de/projects/cdo • R: http://www.r-project.org

13. Software tools Have you ever used the climate data operators (cdo) / R? cdo R Yes Yes Very little Very little No No

14. tools: cdo • Climate Data Operators • developed at Max Planck Institute for Meteorology, Hamburg • freely available including extensive documentation:https://code.zmaw.de/projects/cdo • works with multiple file formats, including grib, netcdf etc. • collection of operators for processing climate (model) data e.g. • spatial interpolation • data selection • Subsampling • statistical and arithmetical functions • … • is accessible from the ‘command line’ (‘prompt’) • cdo does currently not include a graphical user interface and can not visualize data

15. tools: R • R • programming language and software environment for statistical computing and graphics: http://www.r-project.org/ • includes an interface to netcdf • has extensive statistical and graphical capabilities, mainly through additional packages • supports the use of scripts • extensive online-documentation available on the official webpage and on other webpages • has been developed for use in statistics (biology, medicine, etc.); • has not been developed for use in meteorology / climatology:-> terminology sometimes not intuitive -> rather slow for large data sets

16. tools: cdo + R Which operating system are you using? Linux/Unix Windows XP 7 MAC OSX 2000 Vista

17. tools: cdo + R Is cdo / R available on your computer system? cdo R Yes Yes No No

18. tools: cdo + R • A collection of software scripts including example CM SAF data is provided on the Course page • scripts are based on Windows, rearranging them for Unix/Linux/Mac-operating systems should be straight forward. • Before running these scripts, installcdo and R on your local computer system (see further instructions on the Course Page)! • Suggestion: install panoply (visualize netcdf-files) and Notepad++ (text-editor). • Have a look at the screencasts (provided on the Course page) that describe the use of the scripts!

19. Step-by-step • Order data in netcdf-format on a regular lon-lat grid from the Web User Interface

20. Step-by-step • Order data in netcdf-format on a regular lon-lat grid from the CM SAF Web User Interface • Combine the individual time steps into one netcdf-file (cdo)

21. Step-by-step • Order data in netcdf-format on a regular lon-lat grid from the CM SAF Web User Interface • Combine the individual time steps into one netcdf-file (cdo) • Extract the time series for a single location (cdo) • Calculate the temporal / spatial means (cdo) • Calculate the multi-year monthly averages (cdo) • Calculate the monthly anomalies (cdo)

22. Step-by-step • Order data in netcdf-format on a regular lon-lat grid from the CM SAF Web User Interface • Combine the individual time steps into one netcdf-file (cdo) • Extract the time series for a single location (cdo) • Calculate the temporal / spatial means (cdo) • Calculate the multi-year monthly averages (cdo) • Calculate the monthly anomalies (cdo) • Plot time series data (R) • Analyse the time series data, e.g., calculate average annual cycles, trends (R) • Visualize the 2D Data, e.g., monthly means / temporal means / anomalies (R)

23. Step-by-step Surface Solar Radiation, Offenbach, Germany Examples Time Series Analysis Regional Plot

24. Now it is your turn! • Use the scripts and the data provided • Order more data that fits your purpose using the WUI • Best way of learning new tools is using them!

25. Now it is your turn! • Use the scripts and the data provided • Order more data that fits your purpose using the WUI • Best way of learning new tools is using them! • Some ideas to start with: • Have a look at your home town: • What is the average annual cycle of cloud cover, solar radiation, humidity? • Is there a trend in solar surface radiation? • Are there temporal/spatial correlations between solar radiation, cloud cover, humidity in your country? • Which was the ‘best’ summer / winter in the last five years?

26. Now it is your turn! • Use the scripts and the data provided • Order more data that fits your purpose using the WUI • Best way of learning new tools is using them! • Some ideas to start with: • Have a look at your home town: • What is the average annual cycle of cloud cover, solar radiation, humidity? • Is there a trend in solar surface radiation? • Are there temporal/spatial correlations between solar radiation, cloud cover, humidity in your country? • Which was the ‘best’ summer / winter in the last five years? • Which region in your country has the highest cloud cover (e.g., in July)? Is this the region with the highest solar radiation?

27. Now it is your turn! • Use the scripts and the data provided • Order more data that fits your purpose using the WUI • Best way of learning new tools is using them! • Some ideas to start with: • Have a look at your home town: • What is the average annual cycle of cloud cover, solar radiation, humidity? • Is there a trend in solar surface radiation? • Are there temporal/spatial correlations between solar radiation, cloud cover, humidity in your country? • Which was the ‘best’ summer / winter in the last five years? • Which region in your country has the highest cloud cover (e.g., in July)? Is this the region with the highest solar radiation? Create your own project, we are curious to learn from you!

28. Now it is your turn! Any Questions? We will be back later today (14 UTC) in Centra for further comments / questions. • See also the course page: • Download CM SAF Sample File including instructions (screencasts). • Forum at Course page for further discussions. Have fun playing with the data!