XMM data reduction with SAS

1. XMM data reduction with SAS Simon Vaughan (original notes by Tim Roberts)

2. Overview • Context & basics • Obtaining XMM-Newton data and identifying useful files • Setting up the analysis environment • Reducing and cleaning EPIC data • Producing images, light curves and spectra using XMMSELECT XMM data reduction

3. What you are aiming for… Cas A supernova remnant Jupiter’s X-ray aurora The nearest star in X-rays Silicon Continuum Iron Relativistic Fe line emission from close to a black hole X-rays from the Cen A radio jet Hot gas in the Coma cluster of galaxies XMM data reduction

4. Obtaining X-ray data • Astrophysical source of X-rays • Intervening absorption (e.g. Galactic neutral gas) • X-ray optics (e.g. grazing incidence mirrors) • X-ray detectors (e.g. CCDs) XMM data reduction

5. Peculiarities of X-ray data • X-ray detectors are photon counting (as opposed to measuring incoming flux) • X-ray data composed of lists of events and their attributes (time, energy etc…) • X-ray data is usually photon limited – products have few or no counts in many bins • Requires specific data analysis techniques and statistical approaches XMM data reduction

6. X-ray data products • Event list – time-tagged events, with a position (detector/sky space) and energy • Detector attributes e.g. CCD pixel pattern for event – allows rejection of “poor” events • Filter event list then project in 1-, 2-D to give data products • Images, energy spectra, light curves • Calibration essential in interpreting data • e.g. exposure maps + PSF, response matrices XMM data reduction

7. XMM-Newton • XMM-Newton instruments: • EPIC – pn, MOS × 2 • RGS × 2 • OM X-ray telescopes X-ray Detectors Optical monitor XMM data reduction

8. Obtaining XMM-Newton data • Write your own proposal • Unfortunately, a high risk process where success may not be rewarded for up to 1.5 years • Use the archive • XSA provides access to all datasets beyond the 1-year proprietary period • Accessed via: http://xmm.vilspa.esa.es – java interface best run via netscape XMM data reduction

9. EPIC set-up XMM data reduction

10. Select object of interest… …then execute query XMM data reduction

11. Select data… … before going to check out ..then move it to your basket (will need to log in)… XMM data reduction

12. Request multiple then highlight both ODF and PPS… …then submit request, and follow instructions XMM data reduction

13. Data receipt • E-mailed ftp instructions – follow, unpack tar files • Two types of data • ODF – observation data files – telemetry data reformatted to FITS files • PPS – pipeline processing system – top-level science products including event lists, images, source lists, catalogue cross-correlations • TIP: load INDEX.HTM into browser – summary info XMM data reduction

14. XMM data reduction

15. PPS files • Good for “first look” at data • Specific naming convention: • Where: PiiiiiijjkkaablllCCCCCCnmmm.zzz aa – detector: pn, m1, m2, r1, r2, om CCCCCC – file ID e.g. P(M)IEVLI – pn (MOS) imaging event list, IMAGE_n – image (n gives band ID) zzz – file type: ASC, PDF, PNG, HTM, TAR, FTZ XMM data reduction

16. Should I reprocess? • i.e. are the PPS files sufficiently well formatted and calibrated? • New (proprietary) data • Should be OK – just gone through most recent version of pipeline • Archival data • Whilst reprocessing of archival datasets does occur, perhaps best to adopt “better safe than sorry” approach and reprocess XMM data reduction

17. A manageable directory structure XMM data reduction

18. Setting up the user environment • To run on XROA system: > sas-setup-new initialises latest version > setenv SAS_ODF (path_to_ODF_directory) e.g. /data/05/sav2/xmm/tons180/odf > setenv SAS_CCFPATH /usr/local/ccf > cifbuild >& cifbuild.log builds ccf.cif (= Calibration Index File) > setenv SAS_CCF (path_to_ccf.cif_file) > odfingest >& odfingest.log builds ***SUM.SAS file in ODF directory – ODF summary file necessary for reprocessing XMM data reduction

19. Pipeline processing of EPIC data • Each pipeline (pn, MOS) needs one command > emchain (or emproc) > epchain (or epproc) • NB – may need to set ftools up first > lhea-setup-new • Output is calibrated event lists (*EVLI*) • Caveats – multiple event lists may be formed if more than one exposure in dataset, can take some time to run! XMM data reduction

20. What is the result? • One ‘event’ list file [*EVLI*] per exposure • An ‘event’ is a detection (usually an X-ray) • Each event on a CCD is tagged with: • which detector (camera/CCD) • time (CCDs are ‘read out’ periodically) • position (X,Y) on detector • ‘pattern’ indicating how many pixels are involved • energy (~amount of charge deposited in the pixels) • quality ‘flag’ indicating known good/bad pixels/events • To get a ‘science product’ you need to filter this list all the unwanted times, patterns, image areas, etc… XMM data reduction

21. Event patterns (grades) XMM data reduction

22. The SAS GUI • Run by > sas & • Simply scroll, double-click on utility e.g. emproc PRO: transparency CON: only handles one dataset at a time XMM data reduction

23. What next? • Clean data, and produce science products – XMMSELECT GUI • Can run this from SAS GUI… • …but quicker to start from command line > xmmselect table=P0106860101PNU002PIEVLI0000.FIT%events • NB – can take some time to load large datasets esp. pn data containing considerable background flaring XMM data reduction

24. Logical expression used to filter data • Process: • Edit ranges • Click parameter • Repeat… • Select param(s). for product • Select product Filtering criteria and ranges Product selection XMM data reduction

25. Filtering options • Main choices are: • PI – output energy range for product in eV • Time – portion(s) of the observation to include • Pattern – charge signature on one or more pixels • pn: 0 – 4 • MOS: 0 – 12 • Flag – quality control for events • pn: FLAG==0 • MOS: #xmmea_em XMM data reduction

26. Cleaning data: flare exclusion • Perhaps largest problem with XMM-Newton data: space weather • Enhances background – dilutes source signals • Energy-dependent – reduces effectiveness of spectroscopy • Can remove by identifying periods when flaring at worst and excluding them from products XMM data reduction

27. Do I need to flare-filter? • YES! • Examine PPS MOS images – look for enhancement in background in region visible to sky XMM data reduction

28. Filtering • Do for pn: select events above 10 keV, flag & pattern • Select “time” • Extract “OGIP Rate Curve” • Set output file, bin size (10 sec normally OK) XMM data reduction

29. XMM data reduction

30. Excluding time intervals • For one large flare, or a small number of flares work out time intervals when you want to accumulate data (zoom in grace) • xmmselect: (TIME in (xxx:yyy))&& • For noisy data, create Good Time Intervals (GTI) file >tabgtigen table=pn_rates.fits:RATE expression=‘RATE<1.5’ gtiset=pn_gti.fits timecolumn=TIME XMM data reduction

31. Total 21.9 ks data XMM data reduction

32. Caveats • Energy spectra of flares can vary • Conservative approach is to also check light curve from 0.3 – 10 keV data using GTI filter • MOS & pn may have different start/stop times • If need both instruments on (light curves, some spectra) add extra filter with Tstart, Tstop • OK to use pn GTI file on MOS! • If treated separately – do similar filtering for MOS • Very heavy flaring – create new filtered event file before extracting products XMM data reduction

33. Images • Select energy, flag, pattern, time • Indicate X, Y & extract image • Select image tab – set output name, binning size • pn: x/ybinsize=80 • MOS: binsize=20 XMM data reduction

34. XMM data reduction

35. Individual source products • Next step: extract spectra, light curves for individual sources • Selection of correct source, background regions very important! • Rules of thumb: • Avoid: other sources(!), chip gaps, out-of-time events, diffuse sources (if possible) • Consider: distance from read-out, detector structure, same quadrant (pn) XMM data reduction

36. Al-K Si-K XMM data reduction

37. Regions • 30-arcsec around NGC 1313 X-1 • 45-arcsec background region • Save both regions separately! Read-out direction Set to background using “info” then “property” XMM data reduction

38. Then extract…. • Light curves: • Create for source, background separately • Use “2-D region” button to automatically transfer region to xmmselect selected expression • Select “time”, then “OGIP rate curve” • Choose “withtimeranges=yes” • Set “timemin” and “timemax” same for source, background • But note: using flare filter means light curves broken up (i.e. data gaps present) • Spectra: • Highlight source, background regions in ds9, then… XMM data reduction

39. Especget • Select PI, time filter • Choose “OGIP spectral products” • choice to optimise region • Change “stem” in “filenames” • Run (may be slow!) • Spectrum appears in grace window • RMFs, ARFs produced XMM data reduction

40. Resources • This talk (and others): • http://www.star.le.ac.uk/~sav2/stats/ • XMM-Newton SAS web pages • Via http://xmm.vilspa.esa.es • Particularly useful documentation: • HEASARC ABC guide • SAS user’s guide • Talk to other experienced users! • Once you’ve mastered the GUI, try the command line (more power!) XMM data reduction