1 / 51

SKA Central Signal Processor Architecture

SKA Central Signal Processor Architecture. SKA CSP LMC Peer Review 11.April , 2016, Madrid. Sonja Vrcic. SKA CSP LMC Sub-element Lead. List of topics to address as a part of this review. CSP Architecture CSP Control Context Detail CSP Monitor and Control Architecture

tanner
Download Presentation

SKA Central Signal Processor Architecture

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SKA Central Signal ProcessorArchitecture SKA CSP LMC Peer Review 11.April, 2016, Madrid Sonja Vrcic SKA CSP LMC Sub-element Lead

  2. List of topics to address as a part of this review • CSP Architecture • CSP Control Context • Detail CSP Monitor and Control Architecture • Prototyping - Status • Proposed Design SKA CSP Monitor and Control Peer Review, 11. April 2016

  3. Presentation - Overview and Approach Requirements and functionality for CSP_Low and CSP_Mid are similar but different. 1) This presentation first addresses CSP_MID: • Functions • First level decomposition • Sub-elements – functions and design 2) Discuss similarities and differences between CSP_Low and CSP_Mid. SKA CSP Monitor and Control Peer Review, 11. April 2016

  4. SKA1_MID Telescope Source: Phase1 Baseline Design SKA CSP Monitor and Control Peer Review, 11. April 2016

  5. CSP_MID FunctionsReceive input data from receptors • Receive dual polarisation wideband digitised data from 133 SKA1-Mid receptors and 64 MeerKAT receptors. • Resample and convert the MeerKAT data to a form compatible with SKA1-Mid. • Input sample streams from all 197 SKA1 Mid and MeerKAT antennas are resampled into the corresponding common SKA1 Mid sample-rate. • All the resampled signal streams can be processed by identical signal processing modules, reducing design effort. SKA CSP Monitor and Control Peer Review, 11. April 2016

  6. SKA1_MID and MeerKAT Observing Bands SKA CSP Monitor and Control Peer Review, 11. April 2016

  7. Sub-arrays • SKA1-Mid operates concurrently in imaging mode and non-imaging mode with concurrent operation of between 1 and 16 subarrays. • Each subarray is programmable as a separate conceptual telescope in terms of antenna pointing, band selection and the setting of configurable imaging and non-imaging parameters. • Apart from limited resources (hardware that process input from receptors, forms tied-array beams and the communication links) sub-array are filly independent. SKA CSP Monitor and Control Peer Review, 11. April 2016

  8. CSP_MID FunctionsSupport for sub-arrays Make provision for TM to: • Sub-divide array in up to 16 sub-arrays (group the receptors in up to 16 sub-arrays); • Independently select signal processing mode and parameters for each sub-array; • independently start/operate/stop signal processing in each sub-array. • Assign any receptor to any sub-array; • Assign any number of receptors to any sub-array, including a configuration where all receptors are assigned to the same sub-array; • Perform signal processing concurrently and independently in all sub-arrays. Constraint: A receptor cannot belong to more than one sub-array at a time. SKA CSP Monitor and Control Peer Review, 11. April 2016

  9. CSP_MID FunctionsCorrelate input data received from receptors, transmit visibilities • Simultaneously correlate and integrate input data for all receptors that belong to the same sub-array. • Simultaneously and independently correlate input data in up to 16 sub-arrays. • Send the visibilities resulting from the correlation to the SDP Element. • The purpose of the correlation is to measure the level of signal correlation between all antenna pairs at different frequencies across the observing band. SKA CSP Monitor and Control Peer Review, 11. April 2016

  10. CSP_MID Functions Correlation (continued) • Includes coarse delay compensation, channelization in frequency, RFI flagging and excision, fine delay compensation, cross multiply and accumulate. • Correlation is performed for each channel and each pair of antennas, including auto-correlation (where input from a single antenna is correlated with itself). • The result of this operation is commonly called the cross power spectrum, or visibility set and is a representation of the measured signal power from the sky as a function of frequency. SKA CSP Monitor and Control Peer Review, 11. April 2016

  11. CSP_MID Correlator Output Products (Visibilities) • Number of channels: 64,000 • Number of products: Nsub‐arrays x Nbeams x Nchannels x Nbaselines x Npolarisations • For a sub-array where number of antennas N=197: 19,503*64,000*4 = 4,992,768,000 • Integration time (requirement): 0.14 – 1.4 seconds • Total correlator output 64 * 100 Gbps links. SKA baseline design: • The CSP‐Mid shall send and the SDP‐Mid shall receive visibility data at a maximum rate of 2.97Tbps. SKA CSP Monitor and Control Peer Review, 11. April 2016

  12. Concurrency with other observing modes • For the purpose of calibration, visibilities must be produced in all non-idle sub-arrays, but the number of channels may be reduced when correlation is performed concurrently with other modes. • For example: Concurrency with VLBI beamforming is supported, but if performed in the same sub-array results in a reduced number of spectral-channels. SKA CSP Monitor and Control Peer Review, 11. April 2016

  13. CSP_MID FunctionZoom Spectral Line Imaging • Simultaneously correlate and integrate input data for all receptors that belong to the same sub-arrayfor: • up to 4 independently tunable spectral ‘windows’ and • across full bandwidth. • Zoom windows are independently tunable : • Bandwidth configurable in range 4, 8, 16, 32, 64, 128, 256 MHz. • Center frequency configurable with the step size of 1MHz (±10%). • Number of spectral channels per zoom window16K. • To reduce the data volume to SDP: adjacent regular spectral channels are channel-integrated to 1 ± 0.1 MHz coarse channels. SKA CSP Monitor and Control Peer Review, 11. April 2016

  14. CSP_Mid FunctionsPhase binning • For a single pulsar per sub-array, segment individual pulse period into equi-spaced ‘phase-bins’ in time (i.e. the accumulation units), integrate and send to the SDP. • Make provision for TM to specify the pulsar period and the number of phase bins acquired. Note: • In this mode, the wideband delay-corrected signal is channelized into channels of width that allow for the number of prescribed phase bins and bin (time) width to be correlated, integrated and sent to the SDP. • 10 µsec bin width is possible for a range of pulsars and number of phase bins. SKA CSP Monitor and Control Peer Review, 11. April 2016

  15. CSP_MID FunctionsPulsar Search Beamforming • Simultaneously form a total of up to 1,500 tied-array beams for pulsar search in up to 16 sub-arrays. • Any number of up to 1,500 pulsar search beams can be formed in any sub-array. • Any set of receptors that belong to a sub-array can be used to form any beam within that sub-array. • Make provision for user, via TM, to specify the set of receptors to be used per beam and/or per sub-array. • The purpose of the beamformer function is to coherently combine the signals from the MeerKAT and SKA1-Mid receptors such that the combined gain of those antennas is directed at a specified point on the sky. SKA CSP Monitor and Control Peer Review, 11. April 2016

  16. CSP_MID FunctionsSearch for pulsars and/or transients, transmit output products • Search for pulsars and produce a list of potential pulsar candidates (and associated parameters) simultaneously for up to 1,500 pulsar search beams in up to 16 sub-arrays. • Search for single pulses and produce a list of candidates and associated parameter data simultaneously for up to 1,500 pulsar search beams in up to 16 sub-arrays. • Send the list of pulsar candidates, fast transient candidates and associated parameter data to the SDP Element. SKA CSP Monitor and Control Peer Review, 11. April 2016

  17. CSP_Mid Functions Pulsar Search (continued) • PSS beams are formed as the sum of selected antennas within ±10 km of the sub-array centre, are used to search for pulsars and fast transient sources. • Acceleration search and single pulse search can be performed simultaneously in up to 1500 Pulsar Search beams (spread over up to 16 sub-arrays), each covering 300 MHz. • Placement of beams within the input bandwidth is configurable per sub-array. • Beamformer provides 4096 channels across each 300MHz beam. • Pulsar search engine captures data samples for the pre-defined time interval and performs search. • Duration of the search time-interval is configurable, range: 180 – 1800 seconds. • Maximum output data rate: • pulsar search data: 32.66 Gbps. • fast transient (single pulse) search data15.79 Gbps. SKA CSP Monitor and Control Peer Review, 11. April 2016

  18. CSP_Mid FunctionsForm Pulsar Timing Beams • Simultaneously form a total of up to 16 tied-array beams for pulsar timing in up to 16 independent sub-arrays. • Any number of up to 16 pulsar timing beams can be formed in any sub-array. • Any set of receptors that belong to a sub-array can be used to form any beam within that sub-array. • Make provision for user, via TM, to specify the set of receptors to be used in beamforming per beam and/or per sub-array. • Form Pulsar Timing beams using full input bandwidth for Bands 1, 2, 3 and 4 and half of the input bandwidth (2.5 GHz) for Band 5. SKA CSP Monitor and Control Peer Review, 11. April 2016

  19. CSP_Mid FunctionsTime Pulsars, Transmit IPPs • Produce Integrated Pulse Profiles (IPPs) and associated parameters for a single pulsar per beam, simultaneously for up to 16 Pulsar Timing beams. • Make provision for TM to specify ephemerides and pulsar phase/longitude predictors for a single pulsar per PST-beam. • Send Integrated Pulse Profiles and associated parameters to the SDP Element. SKA CSP Monitor and Control Peer Review, 11. April 2016

  20. CSP_Mid FunctionsForm VLBI beams, Transmit VLBI beam data • Simultaneously form up to 4 VLBI beams in up to 4 sub-arrays. • Any number of up to 4 VLBI beams can be formed in any sub-array. • Any set of receptors that belong to a sub-array can be used to form any beam within the sub-array. • Transmit the VLBI beam data in a format that meets the requirements of the VDIF Transport Protocol. Concurrency: • When performed concurrently with the VLBI beamforming in the same sub-array, correlation produces reduced number of spectral channels. • Pulsar Search and Pulsar Timing can be performed concurrently with the VLBI beamforming, but not in the same sub-array. SKA CSP Monitor and Control Peer Review, 11. April 2016

  21. CSP_MID FunctionsECP: Transient Buffer • Store raw or delay-corrected samples. TBC: Store up to 10 sec (TBC) of data samples (for the highest input data rate). • When a trigger is received, send the stored data samples for all receptors that belong to the same sub-array to the SDP. • Include timestamps in the output data packets. • The trigger is generated by SDP and received via TM. SKA CSP Monitor and Control Peer Review, 11. April 2016

  22. CSP_MID - Concurrency • Up to 16 independently operated sub-arrays. • Allow TM to configure, start and stop signal processing in each sub-array independently. • In the table below, Observing Modes marked with x in the same column can be executed concurrently within the same sub-array. • For the purpose of calibration, visibilities must be produced in all non-idle modes, but number of channels may be reduced (TBD). SKA CSP Monitor and Control Peer Review, 11. April 2016

  23. CSP_Mid First level of decomposition Sub-elements: • Correlator and Beamformer (Mid.CBF) • Pulsar Search Engine (Mid.PSS) • Pulsar Timing Engine (Mid.PST) • Local Monitor and Control (CSP_Mid.LMC) • Sub-element Integration Infrastructure (Mid.SII) SKA CSP Monitor and Control Peer Review, 11. April 2016

  24. CSP_MID Context Interface to SDP 64 * 100Gb/s links E.1 100Gb/s each E.2 40Gb/s each SKA CSP Monitor and Control Peer Review, 11. April 2016

  25. CSP_Mid Correlator and Beamformer - Mid.CBF Functions • Receive input from receptors • Correlate input data received from receptors • Perform pulsar phase binning • Form tied-array beams for Pulsar Search • Form tied-array beams for Pulsar Timing • Form tied-array beams for VLBI • Transmit visibilities, PSS, PST and VLBI beam data • Support sub-arrays • Store a limited amount of the raw or delay corrected samples received from each antenna and send to the SDP when trigger is received. SKA CSP Monitor and Control Peer Review, 11. April 2016

  26. Mid.CBFContext Diagram Source: Mid.CBF DDD SKA CSP Monitor and Control Peer Review, 11. April 2016

  27. Mid.CBF Functions and Output Products SKA CSP Monitor and Control Peer Review, 11. April 2016

  28. MID.CBF Design: cca 300 LRUs, mostly custom, some COTS (servers, switches) Source: Mid.CBF DDD SKA CSP Monitor and Control Peer Review, 11. April 2016

  29. Sub-arraying • The diagram is a simplified representation of the correlator resources in the presence of sub-arrays. • Correlator ‘cells’ that receive input from the antennas which belong to different sub-arrays are not used. • Correlator resources are allocated indirectly, by assigning antennas to sub-arrays. SKA CSP Monitor and Control Peer Review, 11. April 2016

  30. CSP_Mid Pulsar Search Engine - Mid.PSSFunctions • Receive input data from the CSP_Mid.CBF • Search for pulsars (acceleration search) • Search for transients (single pulse search) • Transmit pulsar candidates and single pulse data products • Support sub-arrays SKA CSP Monitor and Control Peer Review, 11. April 2016

  31. Mid.PSS Design • The PSS Engine consists of a cluster of COTS computer nodes. • Each node is housing compute accelerators - Graphical Processing Units (GPUs’)and/or Field Programmable Gate Arrays (FPGAs). • Each PSS node is designed to carry out all the functionality of pulsar search signal processing for up to two tied array beams. • Number of nodes: Mid=750 (Low=250)+12 for contingency. • cca 100 network switches (two per rack + aggregation of the PSS output). • Monitor and Control Server, COTS server, dual redundancy. • Total: 40 racks (cabinets) of equipment. SKA CSP Monitor and Control Peer Review, 11. April 2016

  32. CSP_Mid Pulsar Timing Engine - Mid.PSTFunctions • Receive input data from the CSP_Mid.CBF • Produce Integrated Pulse Profiles (IPPs) • Transmit IPPs • Support sub-arrays SKA CSP Monitor and Control Peer Review, 11. April 2016

  33. Mid.PST Context Diagram (Mid.PST DDD) SKA CSP Monitor and Control Peer Review, 11. April 2016

  34. Mid.PST Pulsar Timing Engine • Mid.PST consists of GPU-CPU compute clusters composed of common-use COTS equipment. • 2-rack, 18-compute-node cluster to handle the processing of up to 16 tied-array beams independently (one node handling the processing for one beam + 2 nodes for redundancy). • Mid.CBF each node must be capable of handling up to 96 Gb/s of data. • Low.CBF each node must be capable of handling up to 15 Gb/s of data. • PST M&C Server implements interfaces with CSP_Mid.LMC • The compute load is observation dependent and largely determined by the requested dispersion measure and time resolution to process to. • The maximum output data rate from the PST cluster will be ~10 Gb/s. SKA CSP Monitor and Control Peer Review, 11. April 2016

  35. Mid.PST Hardware Components SKA CSP Monitor and Control Peer Review, 11. April 2016

  36. CSP_Mid Local Monitor and Control - CSP_Mid.LMCFunctions • Support sub-arrays • Implement interface with TM. • Monitor, control and co-ordinate CSP_Mid sub-elements and report on behalf of CSP to TM. • CSP.LMC provides a level of abstraction which allows TM to monitor and control CSP as a single system. Design: • COTS server, dual redundancy, • Network switch, dual redundancy, • Custom developed software. • Implementation and general functionality is the same in CSP_Mid.LMC and CSP_Low.LMC. • The difference is in the details of implementation: different parameters supported in each telescope. SKA CSP Monitor and Control Peer Review, 11. April 2016

  37. CSP_Mid Sub-element System Integration Infrastructure Mid.SII • The cost of equipment, tools and human resources required for integration of the CSP_Mid sub-elements are assigned to CSP_Mid.SII. SKA CSP Monitor and Control Peer Review, 11. April 2016

  38. SKA1_Low Telescope SKA CSP Monitor and Control Peer Review, 11. April 2016

  39. CSP_Low Functions – receive input data • Receive input from 512 stations • Input bandwidth 300MHz (50-350 MHz) • Input from each station consists of 384 channels. • Up to 8 beams per station (total number of coarse channels fixed). SKA CSP Monitor and Control Peer Review, 11. April 2016

  40. CSP_Low Functions – overview of the observing modes • Correlate input from 512 stations • Number of channels per baseline 64K • Up to 8 beams per sub-array • Minimum integration time (requirement): 0.9 seconds • Up to 4 zoom windows + bandwidth across full input BW • Pulsar binning • Form up to 500 pulsar search beams • Form up to 16 pulsar timing beams (same as MID) • No support for VLBI beamforming • No transient buffer (LFAA responsibility) SKA CSP Monitor and Control Peer Review, 11. April 2016

  41. CSP_LOW Sub-elements SKA M&C Harmonization Workshop, 11. April 2016

  42. CSP_Low Design • Low.CBF and MID.CBF are being designed by two different groups: • LOW.CBF design by CISRO, ASTRON and NZA, custom, FPGA based solution, Parentie technology. • MID.CBF design led by NCR-Herzberg with participation from MDA, NZA, custom, GPGA based solution, see www.powermx.org. Same design for LOW and MID: • Pulsar Search Engine, • Pulsar Timing Engine and • LMC SKA CSP Monitor and Control Peer Review, 11. April 2016

  43. LOW.CBF Context SKA CSP Monitor and Control Peer Review, 11. April 2016

  44. Low.CBF Overview - Design and Data Flow cca 88 LRUs SKA CSP Monitor and Control Peer Review, 11. April 2016

  45. SKA1 CSP to SDP Visibilities Data Rate(source CSP to SDP ICD) SKA CSP Monitor and Control Peer Review, 11. April 2016

  46. CSP_LOW Output Data Rates • The CSP-Low shall send and the SDP-Low shall receive visibility data at a rate in the range 2.49 to 3.73Tbps TBC. • 64-bit integer UTC timestamp epoch of the last sample that was integrated (i.e. whether flagged/not correlated, or not). 32 bits is 1 second count since UTC epoch; 32 bits is fraction of a 1 second, providing ~0.25 ns timestamp resolution. • The CSP-Low shall send and the SDP-Low shall receive pulsar search data at a maximum rate of 10.89 Gbps. • The CSP-Low shall send and the SDP-Low shall receive fast transient/single pulse search data at a maximum data rate of 5.303 Gbps • The CSP-Low shall send and the SDP-Low shall receive pulsar timing data at a maximum rate of 1.07 Gbps (this data rate is for the payload data only and does not include overhead for the file format, protocol, etc.) SKA M&C Harmonization Workshop, 11. April 2016

  47. SKA CSP Monitor and Control Peer Review, 11. April 2016

  48. Telescope Monitor and Control - Main Functions • Planning, scheduling and execution of the astronomical observations. • Management of telescope hardware and software sub-systems in order to perform the observations. • Management of the data required to support operators, maintainers, engineers and science users in achieving operational, maintenance and engineering goals (excluding management of the science data products). SKA M&C Harmonization Workshop, 11. April 2016

  49. CSP LMC Functions • Configure: Set-up the configuration of the subsystem in question to operate in the commanded mode as well as set up the configuration of any monitoring or special functions. • Control: For a pre-defined time, activate and provide parameters for any control actions required by the sub-system. • Monitor: Activate, interrogate and receive monitor data from the sub-system. • Provide Diagnostics: Activate diagnostics actions needed for maintenance of system ‘health’ and fault identification. • Store Data: Accumulate and store monitor data. Some monitor data will require only short term storage. It may also be necessary to keep data that is not normally kept as part of the system state, but may be interrogated for diagnostic reasons. Such data might be kept in a circular buffer that overwrites itself after a predetermined time. SKA M&C Harmonization Workshop, 11. April 2016

  50. List of topics to address as a part of this review • CSP Architecture (completed) • CSP Control Context • Detail CSP Monitor and Control Architecture • Prototyping - Status • Proposed Design SKA CSP Monitor and Control Peer Review, 11. April 2016

More Related