1 / 17

Digital Packaging Processor

Digital Packaging Processor. G. Hurford Conceptual Design Review - NJIT April 25, 2011. OVSA Expansion Software and Data Handling - Kickoff Meeting. 23-Oct-10. GH/JM+NJIT. OVSA-specific CASPER-based Miriad-based RHESSI-based. Monitor and Control. Operator. DSPU.

vanna
Download Presentation

Digital Packaging Processor

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Digital Packaging Processor G. Hurford Conceptual Design Review - NJIT April 25, 2011

  2. OVSA Expansion Software and Data Handling - Kickoff Meeting 23-Oct-10 GH/JM+NJIT OVSA-specific CASPER-based Miriad-based RHESSI-based Monitor and Control Operator DSPU Antennas, receivers, analog subsystems NJIT Data Packaging Processor Interim Data Base Data selection, averaging, reformatting & calibration Burst identification Tohban Light curve, spectra display Archive Databases ~6 GB/day Light curve, spectra generation Map generation Map display Quick look & metadata products Hi-Level Analysis IDL Shell Browser Users

  3. OVSA Expansion Software and Data Handling 23-Oct-10 GH/JM+NJIT OVSA-specific CASPER-based Miriad-based RHESSI-based Monitor and Control Operator DSPU Antennas, receivers, analog subsystems NJIT Data Packaging Processor Interim Data Base Data selection, averaging, reformatting & calibration Burst identification Tohban Light curve, spectra display Archive Databases ~6 GB/day Light curve, spectra generation Map generation Map display Quick look & metadata products Hi-Level Analysis IDL Shell Browser Users

  4. OVSA Real-time Data Handling 25-Apr-2011 Monitor and Control Operator Antennas, receivers, analog subsystems Engineering Data Base State Frame Backend Correlator Data Packaging Processor Interim Data Base

  5. OVSA Real-time Data Handling 25-Apr-2011 Monitor and Control Operator Antennas, receivers, analog subsystems Engineering Data Base State Frame Backend Correlator 4 external interfaces Data Packaging Processor Interim Data Base

  6. OVSA Real-time Data Handling 25-Apr-2011 Monitor and Control Operator Antennas, receivers, analog subsystems State Frame Engineering Data Base Backend Correlator Data Packaging Processor 4 external interfaces Interim Data Base

  7. Digital Packaging Processor ACC State Frame Correlator Start / End Scan Cmds Scan-independent Calibration Parms Scan Parameters Frame parameters <P>, <P2> Correlations Frame status report DPP Miriad-format Interim Data Base

  8. DPP-Correlator Interface • 4096 x 2 x16^2 visibility components • 4096 x 2 x 16 <P2> values • 4-byte Floating point ? •  9 MB / integration  450 MB/s • Dedicated 10 Gb Ethernet link(s) ? • Interface architecture driven by correlator design

  9. DPP State Frame Interface • Specifics TBD • DPP will assume ‘header’ parameters are stable during scan • Otherwise DPP reads state frame every ½ second • Assume state frame has flags indicating update state • DPP updates state frame with processing status • ISSUE: Should RFI statistics be ignored, written to state frame, or included in IDB?

  10. DPP – ACC Interface • Low data volume • Shared Ethernet line • ACC issues start scan, end scan commands • ACC provides scan-independent calibration parameters (or should these be in state frame?)

  11. DPP – Interim Data Base Interface • DPP writes directly to dual-port IDB disk • DPP writes data in a Miriad-compatible format • Science frame: ~1 MB  1.4 MB/s  60 GB/day (0.7 second, 500 science channels, 2 polarizations, 2 byte data, 16^2 visibility components, x2 overhead) • IDB disk is also accessible to a data analysis platform

  12. DPP – Processing Tasks (1) • Stage 1 – Every integration interval • Evaluate kurtosis data to identify RFI-affected subbands as a function of frequeny only. • Combine with pre-flagged subbands to generate a destination vector for each subband • Apply complex gains at subband level ?? • Average subband data into science frequency bands • Save 1st 3 moments of averages. • Save RFI statistics ??

  13. DPP Processing Tasks (2) • Stage 2 Processing – Every science frame • Convert antenna-based flags (e.g. slewing) from state frame to baseline-based, frequency-independent flags • Apply time-dependent complex gains if available • Apply baseline corrections • Apply non-linearity corrections • Correct for attenuator settings • OPTIONAL – correct for spectral simultaneity

  14. DPP Processing Tasks (3) • Stage 3 processing - every science frame: • Convert visibility, uv and analysis-relevant state-frame data to Miriad format • Write science frame to IDB • Send DPP status to state frame

  15. DPP - Implementation • Original concept was to follow FASR plan for a cluster-based DPP • Estimate processing requirements for EOVSA at ~100 MIPS = 1/60 of FASR requirements • Implementation will be based on a single multi-core machine • Software organization will be compatible with migration to a cluster if necessary

  16. DPP Software Architecture Interim Database ACC State Frame Correlator DPP Coordination Task I/O, data assembly, no processing per se C1 Pointers within shared memory Header Processing Stage 1 Processing Stage 2 Processing Stage 3 Processing C2 C3, C4 C2 C2 Conventional, time-independent processing tasks Cn = core within quad core processor

  17. DPP Next Steps • Agreement on approach • Detailed definition of interfaces • Definition of processing algorithms • Machine selection and purchase • Development platform? • Language selection (Fortran + ?) • Test data sets? • Early timing tests

More Related