1 / 24

Mark 6 VLBI Data System

Mark 6 VLBI Data System. Alan Whitney Roger Cappallo Chet Ruszczyk Jason SooHoo MIT Haystack Observatory. 11 Oct 2013 2 nd International VLBI Technical Jeju , S. Korea. Why Mark6? - drivers for ever-increasing data rates. -Sensitivity!

olaf
Download Presentation

Mark 6 VLBI Data System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Mark 6 VLBI Data System Alan Whitney Roger Cappallo Chet Ruszczyk Jason SooHoo MIT Haystack Observatory 11 Oct 2013 2nd International VLBI Technical Jeju, S. Korea

  2. Why Mark6? - drivers for ever-increasing data rates -Sensitivity! -VLBI2010 – enables smaller antennas & shorter scans for better sampling of the atmosphere -EHT – enables coherent detection of Sgr A* at mm wavelengths (through a fluctuating atmosphere)

  3. Mark 6 goals - 16Gbps sustained record capability >=32Gbps burst-mode capability - Support all common VLBI formats possibly general ethernet packet recorder - COTS hardware relatively inexpensive upgradeable to follow Moore’s Law progress - 100% open-source software Linux O/S • Other considerations • playback as standard Linux files • e-VLBI support • smooth user transition from Mark 5 • preserve Mk5 hardware investments, where possible

  4. Mark 6 Physical Layout 8 monitor LEDs(one per disk) Module-front-panel connectors for two standard SAS2 cables System chassis (supports 8Gbps by itself) Mark 5/6 enclosure Power supply Chassis backplane kit Data-electronics hardware 5U 1U Retractable cable panel (for easy management of eSATA data cables) Expansion chassis (needed for 16Gbps) Mark 5/6 enclosure Power supply Chassis backplane kit 4

  5. Mark 6 hardware High-speed data connections to module front-panel via two standard SAS cables Existing Mark 5 chassis is upgradeable to Mark 6 New chassis backplanes fordisk power management Cable-management panel(unused cables retract into panel) Existing Mark 5 SATA disk modulesare upgradeable to Mark 6(new backplane and front panel) 5

  6. Mark6 block diagram

  7. c-plane -control plane -author: Chet Ruszczyk -written in python -interface to user (e.g. field system) VSI-S and XML control/monitor -responsible for high-level functions -disk module management creating mounting & unmounting -scan-based recording -status, error-checking, etc.

  8. dplane -data plane -author: Roger Cappallo -written in C -implements the high-speed data flow -input from NIC’s -output to disks within mk6 modules -manages: ▪ start and stop of data flow via packet inspection ▪ organization of data into files addition of metadata to files

  9. dplane block diagram

  10. dplane- Technical Highlights ▪pf_ringused for high-speed packet buffering ▪ efficient use of multiple cores – based on # of available cores ▪smp affinityof IRQ’s ▪thread bindingto cores ▪ most of physical RAM (up to 64-128GB) used for large ring buffers and locked in ▪ one large ring buffer per stream ▪ can be changed dynamically from 1 to 4 streams

  11. dplane – file modes scatter mode ▪ ~10 MB blocks scattered to files resident on different disks ▪prepended block# for ease of reassembly ▪ uses faster disks to keep up with flow, but balances disk usage as much as possible ▪resiliet to standalone program gather ▪ efficiently writes data in correct order to single file ▪ not necessary for single-file (RAID) mode ▪ front end merging software planned for difx RAID mode data written to single file ▪ typically on a RAID array ▪ good mode for single module of SSD’s

  12. Mark 6 data hardware ASRock X79 Champion LGA 2011 Intel X79motherboard64GB DDR3 SDRAM 24002 x Myricom 10G-PCIE2-8B2-2C 10GE NICLSI 9201-16E Host Bus Adapter Mark 6 14

  13. Additional Features ▪ capture to ring buffers is kept separate from file writing ▪ helps to facilitate e-VLBI ▪ FIFO design decouples writing from capturing (e.g. keep writing during slew) ▪ option to conver mk5b format packets “on the fly” into vdif packets ▪all Mk6 software is open source for the community ▪ Mk6 electronics hardware is non-proprietary & openly published

  14. Timeline -Feb 2013: bistatic radar observations of asteroid DA14 recorded at Westford continuous 8 Gb/s on 2 modules -Mar 2013: VLBI2010 stand-alone testing -Sep/Oct 2013: delivery of eight Mark 6 systems (5 - ALMA VLBI; 3 – VGOS) -Sep/Oct 2013: Developing Mark 6 interfacing software for DiFXcorrelator; completing standalone self-test; complete documentation

  15. Proof of Concept Experiment -done with prototype software (v.0)-June 2012 -Westford – GGAO -technical details VDIF format 16 Gb/s onto 32 disks 4 GHz bandwidth on the sky dual polarization with 2 GHz IF’s processed as four 512 MHz channels

  16. Recent progress ▪continuous 16Gbps error-free operation onto 32 disks using ‘scatter’ file system ▪ ‘scatter’ file writing upgraded to support writing operations as high as 31 Gbps onto 32 disks ▪ ‘gather’ file-reading software reconstructs data written in ‘scatter’ mode ▪ FUSE front-end for file reading hides ‘gather’ operation ▪ testing of high-level interface software ▪ long duration simulated experiment usage ▪ tests with SSD-equipped Mark 6 modules ▪ Still to be done: FS support

  17. VLBI Support Forum at <vlbi.org>

  18. Mark 6 availability • Several options: • Purchase new Mark 6 system from Conduant(~US$13k with 64GB RAM memory, no disk modules) • Upgrade existing Mark 5 system (either yourself or with kit from Conduant) • Upgrade Mark 5 SATA-modules (with upgrade kits from Conduant) • Purchase Mark 6 modules (with or without disks) Contact Greg Lynott of Conduantgreg@conduant.com 21

  19. Mark 5 SATA Drive Module Upgrade to Mark 6 New Front Panel Front Panel Connectors for two eSATA cables 8x LED (1 per drive) Re-use Handle from old Module New Latch provided (pre-installed on new Panel) Cooling slots Rear Panel New PCB andpower connector Easilyremovable disks 22

  20. Recording-rate capability vs time mostly COTS COTS 23

  21. Recording-rate cost vs. time COTS mostly COTS 24

  22. Questions?

More Related