TDP activities in calibration and processing

1. TDP activities in calibration and processing Athol Kemball for TDP Calibration & Processing Group (CPG) Department of Astronomy, University of Illinois at Urbana-Champaign akemball@illinois.edu

2. . Calgary . . . Cornell . . MIT UIUC UCB . UNM NRL NRAO Current CPG membership • Athol Kemball (Illinois) (Chair) • Sanjay Bhatnagar (NRAO) • Geoff Bower (UCB) • Shami Chatterjee (Cornell) • Jim Cordes (Cornell; TDP PI) • Shep Doeleman (Haystack/MIT) • Joe Lazio (NRL) • Colin Lonsdale (Haystack/MIT) • Lynn Matthews (Haystack/MIT) • Steve Myers (NRAO) • Jeroen Stil (Calgary) • Greg Taylor (UNM) • David Whysong (UCB) • Mark Yashar (UIUC)

3. Primary CPG goal Provide a quantitative cost and feasibility model for calibration and processing elements of the US TDP LNSD SKA design. Contribute to a construction-ready Phase 1 SKA design and proposal.

4. Current CPG activities • Continued progress on Project Execution Plan (PEP) tasks. • Key recent and continuing focus is coordination with needs of DVA-1 project within TDP: • Processing Verification Program (PVP). • Liaison with PrepSKA project and pathfinders. • Graduate students research involvement and engagement. • Outreach at meetings and in community. (L. Matthews)

6. Feasibility: tracing science requirements SKA science requires unprecedented sensitivity and dynamic range. Need too meet thermal noise limit: (Lazio et al. (2009); Chakraborty & Kemball (2009)) • We are considering these requirements on calibration and processing per science use case.

7. Processing verification strategy • Trace calibration and processing requirements to the SKA science cases. • Assess practical community experience in high-dynamic range imaging with current interferometers. • Parallel/simultaneous strategies: • Analytic or semi-analytic approaches. • Numerical simulations. • Pathfinder tests. (3C286; ATA Whysong & Bower)

8. PVP: projecting DVA-1 performance • Antenna and feed design parameters: • Mount type • Reflector: • Size, shape, and manufacturing method • Optics • Feed and illumination • Polarization purity • Net bandwidth • Etc. • Overall system performance verification: • Cost per unit achieved sensitivity as a function of: • Angular distance from center of main lobe: ρ • Polarization: {I,Q,U,V} • Frequency: ω • Feasibility: • Limiting sensitivity in {I,Q,U,V} (ρ ,ω) due to uncorrected systematic errors.

9. Feasibility: practical community experience in high-DR imaging • Community survey of current DR limitations (EVLA, WSRT, GMRT, ATCA) (Chakraborty & Kemball 2009): • Hardware is destiny • Sky-mount vs rotating antennas • Closure errors • Direction-dependent errors: • Pointing • Beam pattern errors • RFI • Computational and software limits • Longitudinal study on 3C273, 1974-2010 (Chakraborty & Kemball, 2010). • DR is tracking thermal sensitivity evolution, but at cost of calibration complexity. Mean DR evolution over time (Chakraborty & Kemball 2010)

10. Numerical simulations • Current focus of numerical simulations: • Calibrating semi-analytic dynamic-range scaling laws derived earlier. • Placing constraints on radiation patterns (e.g. sidelobe levels and stability, pointing) as part of DVA-1 support.

11. Feasibility: wide-band polarization calibration Pathfinder ATA studies (Bower et al) Typical values up to 10%. Smooth changes with frequency.  Rate of ~3%/10 MHz at 1.4 GHz. Spiral and loop patterns seen in real-imag space. Period of loops similar to that of log-periodic feed (e.g., 3/4 of a turn at 1.4 GHz). Leakages are contiguous between adjacent bands.  This suggests leakages originate in the feed. 1.4 GHz leakages for antennas (numbered) in real,imaginary space as a function of frequency. Leakages for two adjacent correlator tunings.  The leakages change smoothly between bands.

12. Feasibility: direction-dependent polarization calibration Pathfinder ATA studies (Bower et al) Leakages are measured in offset pointings from calibrator. Similar leakages throughout the primary beam. Random contribution increases off-axis and toward higher frequencies. Random component of order 3% within half the HP point at 1.4 GHz. Leakages for ant 11x at center and half power points.  Color shows frequency dependence.  Axes show leakage scale. Same leakages, but showing diff with center.

13. Feasibility: primary beam radiation pattern stability (ATA: Harp et al. 2010)

14. W Jinc/top hat function Correlator FOV Shaping: Smart (t, ν) averaging • Multiply the sky by a weighting (window) functionWconvolve the • u-v plane by Fourier transform of the window function, effectively • tailoring the FOV • Can be used to mitigate sidelobes (shape FOV) in high-resolution imaging • Applying single weighting function in (u, v) plane will impose same FOV • on all baselines • Do this for each visibility, during correlation, pre-calibration

15. Using both real (MERLIN) and simulated (MAPS) data, the MIT Haystack • group has been exploring: • - Attenuation of distant sources due to realistic primary beams (from Cortes) • and array synthesis beams (based on Bolton & Millenaar) • - Expected noise contributions from distant sources before and after • convolution • How well various choices of the correlator FOV function reduce the noise • contributions from sources outside the field of interest; impact on • achievable dynamic range • - Requirements for implementation in future correlators

16. No convolution Gaussian Top Hat Top Hat Top Hat 3C343 3C343.1 from Lonsdale et al., in prep.

17. Costing: emerging trends in exaflop power consumption • Exaflop architectures will have extreme parallelism and heterogeneous processing elements. • Energy costs will be a key architectural driver. • Future energy costs lie predominantly with: • Memory • Data transfer/movement costs within system • Both costs will likely dominate processor power. • Many technical innovations under consideration • Proprietary information and variable technological risk • This introduces uncertainty into energy extrapolation. Yashar & Kemball (2009)

18. Current power projections • Considering system power only: • Exascale general-purpose system built out using today’s technology: 150 MW++ • Exascale general-purpose system assuming timely but evolutionary energy efficiency advances: 60-100 MW+ • Very aggressive technological evolution assumed: 30MW+ • Exascale system with limited computational modes (i.e. tuned for modest number of SKA operation types): • Low tens of MW (say 30 MW). • A hardware-software co-design (e.g. a vendor partnership) – or an “off-the-shelf” configurable supercomputer. • A single-mode exaflop system: • < 20 MW, perhaps << 20 MW. (Gara 2008) (Donofrio et al. 2009)

19. Power utilization effectiveness • Cooling (chilling) and circulation sub-systems (+20-35%, but extensive room for improvement). • Optimized flow of coolant or air (at the board and chip level). • Extensive environmental monitoring of system and data center. • Passive environmental cooling. • Minimizing number of voltage down-conversions from HT to chip board voltage. • E.g. variable voltage computer boards. • Processor and board power optimization (slowdown) modes. • A strong industry driver: • You can’t sell an exaflop system that needs 150 MW of power! • Regulatory interventions loom for large data centers. • Example: National Petascale Compute Facility: • LEED Gold certification for green buildings • Power utilization effectiveness (PUE) ~ 1.1 to 1.2 • Opening June 2010 (Cray Ecophlex system document)

20. Liaison with PrepSKA and international SKA project • Participation in key PrepSKA and SKA meetings: • PrepSKA WP2 PY#2 meeting (Manchester UK, Nov 09). • SKA 2010. • PrepSKA WP 2.6 group participation. • TDP is lead institution on: • Calibration and Imaging Techniques for WBSPF (PrepSKA WBS 2.6.3.1) • Exascale Computing & Hardware (PrepSKA WBS 2.6.7) TDP and PrepSKA have close synergies in goals during design and development phase Close engagement with PrepSKA in refining and coordinating cal. & processing work in WP2; TDP funding facilitates US engagement.