Science Software – v2.01 Training UCLA September 26, 2007

1. Science Software – v2.01 Training UCLA September 26, 2007

2. Agenda • Agenda • 02:00 Software Status Report D. King • 02:10 Themis Data Analysis Software K. Bromund • 02:30 Graphical User Interface (GUI) D. King • 03:00 Troubleshooting and Q&As D. King

3. Status Report • V2.01 Science Software Status Report • General • Loads, introduces and calibrates all L1 quantities, all instruments • Loads calibrated L2 quantities • FGM • L1, L2 data available since early March 2007 • Spin ripples (power system) cleanup completed, 11Hz noise (ETC related) is TBD • SCM • L1 data available since early March 2007 • L2 frequency spectrograms only (FBK) available now; timeseries in next release • EFI • All L1 data available from TH-C since May 2007, TH-D,E since Jun 7 • Bias currents correct on all since last Thursday, July 19 • L2 frequency spectrograms only (FBK) available now; timeseries in next release • ESA • No L1 data, only L0 data – however, read-in is transparent to user • All data available since ESA turn-on, i.e., mid-March • L2 omnidirectional energy spectrograms, ground moments available now • L2 on-board moments available in the next release • SST • L1 data available since SST turn-on, mid-March • L2 omnidirectional energy spectrograms available now • L2 on-board moments available in the next release

4. Updates Since Last Demo Software upgrades • Cotrans now includes SM system (in addition to GSM, GSE, GEO, GEI) • Routines can read in data of arbitrary coordinate • thm_load_state,probe=‘a’,coord=‘gse’,suff=‘_gse’ • Coordinate transformations include FAC system, minvar system, any rotation • See fac_crib.pro (e.g., libs,’fac_crib’ to find location or look on .html repository) • See mva_crib.pro (e.g., libs,’mva_crib’) • See tvector_rotate.pro (for explanations type e.g., doc_library, ‘tvector_rotate’) • LMN in the works (Jiang Liu code) • Tsyganenko models available • Requires dll installation, and works for: • 1. Linux, 32 bit • 2. Linux, 64 bit • 3. Mac OS X, Intel • 4. SunOS, 32 bit • 5. SunOS, 64 bit • 6. Windows

5. Updates Since Last Demo Software upgrades • Thm_cal_scm optimizations and fixes • Fixed extreme slowness for SCW calibration • Decimate data before spin fit if sample rate > 128 S/s • Use reasonable defaults for convolution kernel size • Fixed error in polarity of AC waves (calibration – convolution kernel was time reversed…) • N step 3, correct for DFB digital filter and the DFB Analog Butterworth filter frequency response, in addition to the antenna frequency response • Filtering in step 5 is now applied to z-axis as well as x and y. • Implemented fast convolution for step 3 and 5 • Spin ripples (power system) and 8/32Hz noise cleanup completed • Thm_cal_efi • Eff_dot0, efp_dot0 and efw_dot0 quantities implemented (z axis e-field measurement is computed based on the assumption that E dot B = 0, using the x and y e-field measurements in conjunction with the FGM measurement of B)

6. THEMIS Data Analysis Software • K Bromund • SP Systems Inc./NASA/GSFC • M Davis, D Larson, H Frey, J Bonnell, J McTiernan, J Lewis, D King, V Angelopoulos – UC Berkeley • K Ramer, P Cruce, H Schwarzl – UCLA • E Georgescu – MPE • P Robert, O LeContel– CETP • B Jackel, E Donovan – Calgary

7. Overview • Software Objectives • Powerful, Flexible Command Line Interface • GUI to provide Easy Access to Key Features • Software Installation • Data Distribution • Key Routines, crib sheets. • Examples Overview

8. Software Objectives • Code is available to everyone, but not required to analyze data. • IDL based (library of routines –but no main program!). • Command driven. • Separates the tasks of: • Reading files. • Manipulating data • Plotting • Platform independent. Works on: • Solaris • Linux • Windows • Mac OS X • Self-Documenting • Auto-generated html help: idl/_tdas_doc.html • DOC_LIBRARY, ’routine_name’

9. Software Design • THEMIS-specific routines (idl/themis) • Instrument-specific routines organized according to ground/spacecraft/state, fields/particles, instrument name. • Loading data • Calibrating data • Transforming data • Examples – crib sheets • GUI – built on top of the command-line routines • General routines (idl/ssl_general) • Library of generic routines useful for building mission-specific load routines • CDF reading/writing routines • File retrieval routines • Miscellaneous routines • Plotting routines • Routines to manipulate/plot tplot variables • tplot variables: strings that associate data together with metadata and plotting parameters. • Data Export routines • Data Processing routines • External Libraries (idl/external) • CDAWlib • IDL_GEOPACK

10. System Requirements • Windows, Solaris, LINUX, PPC Mac or Intel Mac. • IDL 6.2 or higher required • IDL Patch Recommended • http://cdf.gsfc.nasa.gov/html/cdf_patch_for_idl6x_new.html • Required for IDL 6.2 • Required for Intel Mac, regardless of IDL version • For Mac, system configurations are required to run IDL • X11 – may need to be installed. • mouse click-through • one-time X11 configuration necessary for proper operation: defaults write com.apple.x11 wm_click_through -bool true • See User's Guide for full information.ftp://apollo.ssl.berkeley.edu/pub/THEMIS/3 Ground Systems/3.2 Science Operations/Science Operations Documents/Software Users Guides/

11. Installing/Configuring TDAS • For a new installation: • Download and expand the latest TDAS release .zip file. • Create a directory called TDAS into which you will copy the latest software. • Move the tdas_x_xx folder into the TDAS directory you created. • Configure IDL to search the TDAS directory for IDL programs. Details on next slide. For an upgrade of an existing installation of TDAS, installed a per the above 4 steps: • Remove old tdas_x_xx from the TDAS directory. • Download and expand the latest TDAS release .zip file. • Copy the new tdas_x_xx directory into the pre-existing TDAS directory. • Re-start IDL.

12. Path Setup • Set up the IDL path • Windows and IDLDE on any platform: File->Preferences Path Tab Press Insert Browse to find the TDAS folder you created. Check the box preceding the path to 'search subdirectories' • UNIX-like systems (Mac OS X, Linux, Solaris) In .cshrc: setenv IDL_PATH ‘<IDL_DEFAULT>:+/path/to/tdas’ -Or- In .bashrc or .bash_profile: export IDL_PATH=‘<IDL_DEFAULT>:+/path/to/tdas’ • Path to Data Directory • Data directory will be created automatically at • C:/data/themis (Windows) • ~/data/themis (UNIX/LINUX/Max OS X) • Run thm_ui_config at the IDL prompt to change this.

13. Data Definitions • The software operates on Level 1 and Level 2 data. • Data Level Definitions: • Level 0 Data – • Raw files (*.pkt) one per APID. • Only used for loading ESA data. • Level 1 Data - • CDF (Common Data Files) files (*.cdf) • Contain raw, uncalibrated data. i.e. counts, DAC units. • Used primarily by instrument scientists. • Typically requires software to interpret. • Level 2 Data - • CDF files – contain physical quantities • Available for most instruments/data quantities

14. Data / Directory structure • Data Directory structure is large! • Scores of files per day • ~3GB/day for all probes (L1 data) • Directory hierarchy keeps directory sizes manageable • Software performs automatic file retrieval. • Software maintains directory hierarchy. • Behavior of Automatic File Retrieval is configurable • ‘No Download’ mode for stand-alone operation. • ‘No Update’ mode to preserve local modifications. • Root directory of local copy of hierarchy is determined automatically, but configurable. • GUI and Command Line share same local file cache and configurations. • Available configuration methods: • thm_ui_config IDL widget • 'Config' button in the thm_gui IDL widget. • Environment variables

15. Primary Routines • Load Routine Summary Notes: (*) calibration routine available but still under development - data reduction and analysis routines available: see crib sheet

16. Primary Routines • Usage Conventions: Use keywords to determine functionality data level - level 2 is default if it exists. calibration coordinate transformation datatype and probe keywords determine what data is loaded and/or created through calibration process /get_support_data keyword to load support data required by thm_cal* and thm_cotrans routines. Example thm_load_state, probe = 'a', /get_support_data thm_load_fgm, probe='a', coord='gsm', datatype='fgl', level=1, type='calibrated'

17. Primary Routines • Calibration Routines • thm_cal_efi • thm_cal_fbk • thm_cal_fit • thm_cal_fgm • thm_cal_scm • thm_cal_mom • Calibration can be called separately from thm_load routine when raw data is loaded • for other instruments, see examples in crib sheet.

18. Primary Routines • Coordinate Transformations • thm_cotrans • transforms to/from any of the following coordinate systems in a single call • updates metadata in output. • knows coordinate system of input from metadata • Currently Supported Coordinate Systems • SPG Spinning Probe Geometric • SSL Spinning SunSensor L-vectorZ • DSL Despun SunSensor L-vectorZ • GEI Geocentric Equatorial Inertal • GSE Geocentric Solar Ecliptic • GSM Geocentric Solar Magnetospheric • Example • thm_load_state, /get_support_data • thm_cotrans, 'th?_fg?', out_coord='gsm'

19. SPG coordinates: X normal to solar panel 1 Z along probe geometric axis (out of page) Other coordinate systems: x z SSL (Spinning, Sun Sensor, L-vector) DSL(Despun, Sun direction, L-vector)

20. Sun Sensor ESA SST

21. Primary Routines • Plotting • tplot • tplot_names • tlimit • tlimit, /full • tlimit, /last • ctime, t • get_data • store_data • tr = timerange(/current) • etc. Data Processing • tdegap • tdeflag • wav_data • dpwrspec • etc. Exporting • tplot_ascii • makepng • makegif • tplot_save • tplot_restore

22. Variable Names • Probe specification. Example: tha • a – can be one of [a-e] specifies probe • Particle data. Example: tha_peif • p – particles • e – ESA, s – SST • i – ions, e – electrons • f – full, r – reduced, m – moments, b – burst • FGM data. Example: tha_fgl • l – low telemetry rate, h – high telemetry rate, e – engineering decimated high rate, s – spin fit. • Electric Fields and SCM. Example: tha_efs • ef - efi, sc – scm, fb – fbk, ff – fft • s – spin fit, f – full orbit or fast survey, p – particle burst, w – waves burst. • Wildcards accepted in names when plotting and data processing: • th?_fg? • th[ab]_fg[lh] • th?_state*

23. thm_crib_fgm thm_crib_fit thm_crib_gmag thm_crib_mom thm_crib_scm thm_crib_sst thm_crib_state thm_crib_tplot thm_map_examples Crib Sheets Crib Sheets for Loading, Processing and Plotting • thm_crib_asi • thm_crib_dproc • thm_crib_efi • thm_crib_esa_da • thm_crib_esa_moments • thm_crib_export • thm_crib_fbk • thm_crib_fft .run thm_crib_asi or cut and paste or copy and modify for your own use

24. Command Line Example 1 • To load data: • timespan,'6-10-2',2,/days • thm_load_gmag,site='ccnv',$/subtract_average • To plot data: • options,'thg_mag_ccnv',$labels=['Bx','By','Bz'] • tplot_options, 'title', $'GMAG Examples' • tplot,’thg_mag_ccnv’

25. Command Line Example 2 • Split the 3-vector into components: • split_vec,'thg_mag_ccnv' • Compute transform of one component • wav_data,'thg_mag_ccnv_x',/kol $ ,trange=tr ,maxpoints=24l*3600*2 • Set color limits (log scale) • zlim,'*pow', .0001,.01,1 • Plot it. • tplot,'*ccnv_x*',trange=tr • Wavelet transform on an interval of interest • Define and display the interval • Tr = ['2006-10-2/16:00','2006-10-3/05'] • timebar,tr

26. Command Line Example 3 • thm_crib_esa_da • data analysis • of ESA electron • spectra and • distributions:

27. J. McTiernan 23-Jul-2007 http://ssl.berkeley.edu/~jimm/themis/thm_gui.ppt The THEMIS Graphical User Interface

28. IDL> thm_gui • Called from IDL command line. • Gives the ability to load and plot data, and also some data processing tasks. • Calls various TPLOT routines • IDL functions are available from command line when widget is up. • Find this document: http://ssl.berkeley.edu/~jimm/themis/thm_gui.ppt

29. Main Widget:

30. Choose Data to Load: Click here for FGM data

31. Data choice window: Nothing will happen until you click here

32. Choices show up in History Window: Click here to clear choices.

33. Load Data: Click here to load.

34. Oops, Nothing happens unless you choose a time range Click Here to choose times.

35. Time Range Window: You can type here. Initially times are set to 1970-01-01/00:00:00 Times show up in history as you type. Click here for a time selection widget

36. Time selection widget:

37. Selection shows up in the History: Now you can load data.

38. After loading, All data is “active”. State data has been loaded automatically for FGM.

39. Setting Active Data: Type a string, or click to set data to “active”

40. What next? Plotting. Data Processing Coordinate transform

41. Coordinate Transform: Click one of these to choose output coordinates. Shows current transform Click here to do the deed. Message window: this will show warnings, errors, and progress.

42. Clicked on GEI, and Transform Button:

43. Back on the main widget: New “active” data. Remember – only active data is processed.

44. Plot Widget: If there is active data, plot pops up first.

45. Plot Widget: Setting “PNG” Click here Click here File shows up in working dir. Remember to click on SCREEN to return to screen plots

46. Check History: These are the IDL commands used.

47. Data Processing (dproc) widget:

48. Data Processing (dproc) widget: Deletes Active Data.

49. After dpwrspec button: (and time limits button) *For some time ranges, this crashes in the IDL POLY_FIT routine, due to a singular matrix detected.

50. Note new “active” data: These processes reset the “active” data.