MeerKAT Science Processing

1. MeerKAT Science Processing Jasper Horrell, SKA SA Postgrad Bursary Conference, Stellenbosch – Dec 2010

2. 2014 Vision Its 2014, imagine ….

3. Science Processing (SP) Back to the present: • Renewed 2010 focus on “science processing” with a dedicated team – fairly early days. 6 people currently plus some part-time inputs. • Two main focus areas as reflected in the approved large survey projects: a) imaging; b) non-imaging. Is there anything else?  • Imaging: SP focus on data processing and archive post correlation. Define, design and build the facility systems to support this over next few years. • Non-imaging: Current SP focus on raw data capture, offline correlation, data analysis, pulsars, VLBI capture, SETI. • Interface to the approved large survey project teams re data products, formats, pipeline development, etc. (Meeting earlier today!) • SIMULATIONS, prototypes, involvement in early science projects, etc. A bit more to this than “using CASA for data reduction” 

4. MeerKAT Data Flow

5. KAT-7 – our playground

6. Observation Planning/Tracking

7. Getting there - not quite 

8. Getting there – remote ops!

9. New Cape Town Office (cont.)

10. KAT-7 Early Fringes (2009)

11. KAT-7 Horizon Masks

12. KAT-7 Horizon Masks

13. KAT-7 Images (2010) Moon size 4-dishes

14. KAT-7: Archive To Date • Started archiving visibility data from KAT-7 in Dec 2009 • To date, 1230 data captures saved to archive • Approx 490 GB visibility data so far • On average 40 data captures per week (March to Oct, excluding RFI campaign) • Trivia: Fornax A and Pic A are most observed targets

15. KAT-7: Archive Future • Archive will need more and more attention as data requirements evolve • Already seeing the need to support multiple data sources and formats – Fringe Finder, KAT-7 and holography • Currently reviewing and testing Apache OODT components to assess viability for MeerKAT data catalog and archiving • ASTRON and NRAO also have systems of interest…. • Early days…. • The threat of raw data!!!

16. KAT-7 Data Archive

17. KAT-7: Last week on site • First site test of raw data capture (full band 400 MHz for a few secs plus GPU-based digital down-converter for “indefinite” period) • First VLBI data capture run (HartRAO fringe test) • First SETI data capture for Project Dorothy • 300 GB recorded • Also, “Picture A”

18. The Beast - “Big Gulp?” • 2 x 10 GbE NICs • 2 x GTX 460 2GB GPUs • 2 X 6 Core 3.0 GHz Xeons • 16 x 2 TB HDD • 24 GB RAM

19. Pic A: Hot Off the Press!

20. Pic A: Hot Off the Press!

21. Pic A: Hot Off the Press!

22. Pic A: Hot Off the Press!

23. Pic A: Hot Off the Press!

24. Pic A: Hot Off the Press!

25. Exploring Polarimetric Cal

26. Exploring Polarimetric Cal !! where (even) Oleg fears to tread…!!

27. Exploring Polarimetric Cal

28. Exploring Polarimetric Cal

29. SP: MeerKAT to provide (?) : • Calibrated visibilities • Certain standard pipelines (spectral line image cubes, continuum images) • Storage for visibility data (project disk quotas?) • A 10 PB archive on site plus 3 PB in CT, plus likely European mirror • Archive access and facilities to reprocess from archive • Flexible architecture for “black belt” users plus space in the Array Processor Data Centre for additional user-supplied equipment • Support for some re-use of existing mature packages where possible (support CASA, MeqTrees). • Proposal management etc.

30. SP: In place now • Basic data capture framework to HDF5 file archive format and MS export writing capability (currently being extended for full KAT-7 correlator) • Simple archive for commissioning purposes • SCAPE commissioning package • First imaging software • Simulation and data reduction tests in Meqtrees • Prototype online system based on ROACH sampling, followed by GPUs pre-processing and disk storage. Allows for raw data recording for commissioning, VLBI, software correlation, SETI • Emerging NRAO and ASTRON collaborations • Some great early results on the KAT-7 system • A lot of work to be done over the next few years (we won’t be bored).

31. SP: Next few months • Tests of streaming framework (online system) prototype for KAT-7 with new 16-input KAT-7 correlator (KAT-8 ready!) – now in lab testing phase, site deployment soon’ish • More mature imaging on KAT-7 using both CASA and MeqTrees. Aim for 7-antenna hand-crafted test image using both packages and compare. Also try “pipeline mode” on new data (expect poor initial results). • Coarse grain parallelization investigations (for CASA) • GPU gridding (for CASA) • Simulations for MeerKAT using primary beams (sky de-rotation effects etc). CASA and Meqtrees. Aim for SKA memo. • Beam polarization characterization on ATA and KAT-7 • Continue RFI characterization and mitigation work (RFI database plus GPU flagger) • Ongoing HDF5 scalability testing • Start of some pulsar tools on KAT-7 (port GUPPI to ROACH2 plus GPU?) • More detailed design for MeerKAT (archive, data model etc) • Small VLBI and SETI projects on KAT-7 Engagement with large survey project teams!!

32. Conclusions • Major joint effort needed over coming years to ensure that the science teams hit the ground running • Also need to support commissioning and smaller PI proposals • Technology within reach for MeerKAT scale imaging • Fine tuning (e.g. imaging pipeline) will take time • Technology challenges in pulsar search area • KAT-7/8 will continue to be an excellent engineering and science test bed • Plan re-use software where possible, but expect some custom developments • Lots to do. Technical team will need to grow a bit. Project teams can help (a lot).

33. Questions… Credits (full SP Team): Thomas Bennett Mattieu de Villiers Jasper Horrell Tshaks Namalili Simon Ratcliffe Ludwig Schwardt jasper @ ska.ac.za http://www.ska.ac.za