Solar Data Analysis

1. F. Breitling, G. Mann, C. Vocks 3nd Workshop of the Solar Key Science Project of LOFAR Astrophysikalisches Institut Potsdam 2010 July 05 Solar Data Analysis Solar Data Analysis F. Breitling

2. OverviewThe three LOFAR data types • Dynamic spectra • Single station imaging • Standard imaging RCU: Receiver unit RSP: Remote station processing board TBB: Transient buffer boards Solar Data Analysis F. Breitling

3. Dynamic spectraData • Beam-formed data (currently) • Container format: Hirachical Data Format 5 (HDF5) • Specification: Interface Control Document 3 lofar-usg-icd-003.pdf • Dynamic spectrum data (maybe in future) • Container format: Hirachical Data Format 5 (HDF5) • Specification: Interface Control Document 6 lofar-usg-icd-006.pdf => both formats are currently under heavy development Ref: http://usg.lofar.org/wiki/doku.php?id=documents:lofar_data_products Solar Data Analysis F. Breitling

4. Dynamic spectraSoftware • Dynspec package • developed by Solar KSP (Frank Breitling) • part of LOFAR User Software (http://usg.lofar.org) • build on Data Access Layer (DAL) • Python scripts • Components • specview.py (can read Fit and HDF5 files) • downsample.py (can also create Fit files) • rebin.py Solar Data Analysis F. Breitling

5. Dynamic spectraFirst solar observations Date: 2009-09-11 10:08:52 (ID108) 2009-09-11 13:09:39 (ID109) Duration: 1h LOFAR stations: CS302-LBA RS307-HBA RS503-HBA Measured param.: voltages of both polarizations Subbands/station: 62 Down sampling: 128 Solar Data Analysis F. Breitling

6. Dynamic spectraFirst results Solar Data Analysis F. Breitling

7. Dynamic spectraIdentified problems • stripe of high intensity in CS302, shifting from subband 30-40 for obs108 to subband 50-60 for obs109 • intensity peaks and drops for subband 0-20 in obs109_CS302LBA • intensity gradient in obs109_RS503HBA Solar Data Analysis F. Breitling

8. Dynamic spectraNext steps • New versions of the beam-formed and dynamic spectra data format has to be finalized • Dynspec software can be adapted • Discussion with Michael Wise and Lars Bähren about their further development plans • Travel to ASTRON is already planed for July 12-16 Solar Data Analysis F. Breitling

9. Single station imagingData • Visibilities from LCU correlator • 96 receiver units (RCUs) x 2 polarizations x 16 byte • Format: Binary • Specification: • Menno Norden <norden at astron.nl> • rspctl --help (running on LCU) Reference: http://usg.lofar.org/forum/index.php?topic=202.0 Solar Data Analysis F. Breitling

10. Single station imagingSoftware • acm2skyimage.m • developed by Stefan Wijnholds <wijnholds at astron.nl> • runs with Matlab and Octave • Extended components by Frank Breitling • SolarAzEl.m to add the Sun • RADec2AzEl.m to add other astronomical objects (also published as Matlab package at http://www.mathworks.com/matlabcentral/fileexchange/25617-right-ascensiondeclination-to-azimuthelevation) • make-movies.sh Solar Data Analysis F. Breitling

11. Single station imagingPotsdam commissioning Solar Data Analysis F. Breitling

12. Single station imagingResults from Effelsberg Solar Data Analysis F. Breitling

13. Single station datacombining with GURT Solar Data Analysis F. Breitling

14. Standard imagingBasics • Basic data analysis software • CASA (Common Astronomy Software Applications) http://casa.nrao.edu/ • Data format • Measurements Sets (MS) from CASA • Data processing: Standard Imaging Pipeline • The LOFAR wiki http://www.lofar.org/wiki/doku.php?id=software:standard_imaging_pipeline • The LOFAR Imaging Cookbook http://www.mpa-garching.mpg.de/~fdg/LOFAR_cookbook/LOFAR_Cookbook_v2.3.pdf Solar Data Analysis F. Breitling

15. Standard imagingPipeline Components • OLAP: Online Application Processing (Blue Gene/P correlator) • DP3/DPPP: Default Pre-Processing Pipeline (for flagging and averaging) • BBS: BlackBoard Selfcal (calibration) • CImager: Imager Control • via parset files Solar Data Analysis F. Breitling

16. Standard imagingFirst solar observations Date: 2010-06-09 10:17:00, 11:41:00, 13:40:00, 15:40:00 Duration: 4 x 10 Min LOFAR stations: 20 CS +3 RS, i.e. RS106HBA, RS306HBA, RS503HBA [CS00{1-7}, CS0{21,30}, CS103] HBA{0,1} Measured param.: complex visibilities, 2 polarizations Frequency range: ~150-220 MHz Subbands /station: 7 Solar Data Analysis F. Breitling

17. Standard imagingData Inspection with casaplotms • a lot of RFI • flagging with flagdata • creation of NDPPP.parset • reprocessing with NDPPP Solar Data Analysis F. Breitling

18. Standard imagingSky model Calibration using BlackBoard Selfcal requires skymodel File contents of 078xx.skymodel: # (Name, Type, Ra, Dec, I, Q, U, V, ReferenceFrequency='60e6', SpectralIndexDegree='0', SpectralIndex:0='0.0', MajorAxis, MinorAxis, Orientation) = format # The above line defines the field order and is required. TauA.neb, GAUSSIAN, 05:34:32.00, +22.00.52.00, 1888.55, 0.0, 0.0, 0.0,81.5e6, 0, -0.299, 420.0, 290.0, 0.0 3C123, POINT, 04:37:04.72, +29.40.15.6, 454.97 3C133, POINT, 05:02:58.58, +25.16.22.7, 44.67 3C131, POINT, 04:53:23.26, +31.29.26.4, 37.54 3C139.2, POINT, 05:24:26.92, +28.12.54.2, 35.88 3C141, POINT, 05:26:43.55, +32.50.02.0, 31.76 3C132, POINT, 04:56:43.00, +22.49.23.7, 24.39 4C29.17, POINT, 05:10:40.10, +29.09.00.5, 19.48 4C29.16, POINT, 05:09:29.50, +29.57.56.3, 17.43 3C138, POINT, 05:21:10.46, +16.38.26.0, 14.63 PKS0531, POINT, 05:34:44.65, +19.27.21.9, 14.52 Taken from “The VLA Low-Frequency Sky Survey” catalog (http://www.cv.nrao.edu/4mass/VLSSlist.shtml) Solar Data Analysis F. Breitling

19. Standard imagingBBS calibration After running the BBS calibration amplitudes and phases are stable Solar Data Analysis F. Breitling

20. Standard imagingmwimager after calibration the master worker imager creates: • PSF • dirty image • clean image Solar Data Analysis F. Breitling

21. Standard imagingDirty image Dirty image not yet sucessful Solar Data Analysis F. Breitling

22. Standard imagingNext steps • investigation of 2s clock offset of CS002-CS007 (found by Olaf Wucknitz) • BBS calibration without short baselines (suggested by James Anderson) • automatic flagging of RFI • travel to ASTRON on July 12-16 and discuss with the experts Ger van Diepen and Ger de Bruyn • take more observations, e.g. new data on July 03 Solar Data Analysis F. Breitling

23. Progress documentation Solar Data Analysis F. Breitling

24. Solar Science Data CenterYearly data volume for solar observations with LOFAR (assuming a maximum observation time of 8h/d) Solar Data Analysis F. Breitling

25. Summary • The SKSP is engaged in 3 types of data analysis • Dynamic spectra can already be taken, next the new format has to be completed • Single station imaging is important for commissioning, single station operations and for our colleagues from GURT • Standard imaging is the newest data for the SKSP and will need a lot of attention in the future • Ultimately we plan for a Solar Science Data Center and work is ongoing Solar Data Analysis F. Breitling

26. References • User Software Group: http://usg.lofar.org/ • LOFAR User Forum: http://usg.lofar.org/forum/index.php • LOFAR Interface ControlDocuments (ICDs) http://usg.lofar.org/wiki/doku.php?id=documents:lofar_data_products • Documentation for the Standard Imaging Pipeline http://www.lofar.org/wiki/doku.php?id=software:standard_imaging_pipeline • The LOFAR Imaging Cookbook http://www.mpa-garching.mpg.de/~fdg/LOFAR_cookbook/LOFAR_Cookbook_v2.3.pdf • Frank Breitling <fbreitling at aip.de> http://www.aip.de/People/fbreitling/ Solar Data Analysis F. Breitling

27. Yearly observation time (Maximum possible) • at Exloo (core stations) • altitude >10° above horizon => tobs=~8 h/d =10x106 s/y Solar Data Analysis F. Breitling