1 / 28

SA1: Production e-VLBI service

The annual review highlights operational advancements, data rate improvements, new modes, and telescope integrations in the global e-VLBI service, showcasing successful demos and the push towards higher bandwidth and reliability. Discover the journey towards overcoming connectivity challenges and simplifying station connections.

cfoskey
Download Presentation

SA1: Production e-VLBI service

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SA1: Production e-VLBI service Arpad Szomoru JIVE

  2. Outline • e-VLBI science/test runs • Operational improvements • Global e-VLBI: the demonstrations • Hard- and software developments EXPReS Annual Review: SA1

  3. e-VLBI science/test runs • 1 Target of Opportunity project (target: nearby supernova) • 1 response to a triggered proposal (target: accreting X-ray binary in a special X-ray state) • 2 exploratory projects (both targets were candidate radio counterparts to so far unidentified gamma-ray source) • 1 Spectral line project (targets: candidate MASER sources) • October deadline only had one triggered proposal • Lack of triggers meant no science runs until March 2009 • ...but, available time was used very well • Different proposal/selection mechanism clearly needed • 3rd year already much more productive EXPReS Annual Review: SA1

  4. e-VLBI: operational improvements • Overall improvement in second year: • Huge increase in robustness, reliability and user-friendliness • Not limited to e-operations • Increase of operational data rate from 256 to 512 Mbps • First experiments at ~1 Gbps (with European telescopes) • Demonstrations of global e-VLBI with Chinese and Australian telescopes • Solution of long-haul problems • Inclusion of Effelsberg, Metsahovi, Shanghai, Arecibo (again), TIGO EXPReS Annual Review: SA1

  5. e-VLBI: data rate improvements EXPReS Annual Review: SA1

  6. Towards global e-VLBI • Long-haul high-bandwidth data transport: • TCP Reno on old linux kernels clearly inadequate • Modern TCP stacks much better, UDP even better, but impossible without modification of Mark5A control code • And can be hostile to other users on open network • Stability of original Mark5 code a serious issue EXPReS Annual Review: SA1

  7. Global e-VLBI: hacking time • With several high-profile demos coming up: time to take matters in our own hands • First step: re-enable UDP in Mark5A code • abysmal performance... • Incorporate VLBI-UDP in Mark5 code, in collaboration with colleagues from UMan • Simplified version developed at JIVE, used in both APAN and EXPReS-Oz demo • Complete re-write of Mark5 control software, with rigorous thread control, enabling packet dropping, re-ordering, padding • Used successfully to transfer data at near- line-speed from European telescopes, from Arecibo and, recently, from South Africa and South America • Large development effort, but has paid off in a big way EXPReS Annual Review: SA1

  8. The road to China: the demo at Xi’An EXPReS Annual Review: SA1

  9. Needed: Reliable transport from standard Mark5 at Shanghai (ancient Linux kernel) Physical connection between two non-communicating Chinese NRENs Some way to bypass the Great Chinese Firewall Interface between correlator and Australian telescope control systems On-the-fly translation from LBA to Mark5B format What could possibly go wrong...... The APAN demo EXPReS Annual Review: SA1

  10. Opened up Siberia connection Excellent PR Longest (e) baseline ever APAN results EXPReS Annual Review: SA1

  11. The EXPReS-Oz demo • Three Australian telescopes (Mopra, ATCA, Parkes) • 12 hours at 512 Mbps without a hitch • Beautiful detection of SN1987a EXPReS Annual Review: SA1

  12. Correlator upgrades • Network upgrade: • All internal JIVE and e-VLBI traffic now through own routers (HP5412zl and HP3500yl-48 ProCurve) • Found quite a few of (serious) bugs in HP equipment • So far, one bug fix out of 5 active cases • Multiple 1-Gbps UDP streams not foreseen by HP engineers? • Dedicated network monitoring platform alongside routers • Removal of dependencies on ASTRON network • Upgrade of Mark5 hardware (motherboards, memory, power supplies) completed • New data acquisition platform (Solaris AMD server, fully interchangeable with redundant control computers EXPReS Annual Review: SA1

  13. New modes, feedback and streamlining • Completed: • General speedup of correlator control code • Real-time Mark5 debug info feedback to stations • Re-connecting of individual stations without job re-start • Space-craft tracking mode • Use of Westerbork synthesis data to improve e-VLBI calibration and target selection • Network monitoring tools • Currently under development: • Adaptive observing • On-the-fly fringe fitting • Real-time station-log processing, display of Tsys, status information • Mixed-mode e-VLBI • Integration of Mark5B • 1024 Mbps e-VLBI (and more..) EXPReS Annual Review: SA1

  14. New network monitoring tools... EXPReS Annual Review: SA1

  15. And new telescopes... EXPReS Annual Review: SA1

  16. In summary • Guaranteed high bandwidth through use of an aggressive protocol *and* reliable software • Lightpath technology offers clear advantages; routed connections need negotiations • Convincingly demonstrated that it is possible to overcome long-haul connectivity problems • Simplicity paid off: trivial to connect new EVN stations once they come online EXPReS Annual Review: SA1

  17. EXPReS Annual Review: SA1

  18. EXPReS Annual Review: SA1

  19. SA1: state of deliverables • A total of 15 deliverables were due by month 24, 11 at JIVE and 4 at the University of Manchester • 8 out of 11 JIVE deliverables were completed • Most of the work on the remaining 3 was finished • 3 deliverables that were not due until the last year of EXPReS were completed well ahead of time • The 4 UniMan deliverables were delayed because of the delay in the delivery of the e-Merlin correlator. Most of the IBOB design work has been completed, but tests await the arrival of the first I/O boards, in the summer of 2008 EXPReS Annual Review: SA1

  20. Outsourcing • Three workpackages were outsourced to MIT: • SA1.24: Mark5A code modifications • Working version available (though no “official” release yet) • SA1.25: Mark5B code modifications • Dimino (sending) side largely done in-house • Domino (receiving) side developed at MIT, debugging in progress • Extensive engineering support in integration of Mark5B • SA1.20: Improved network applications • mainly done in-house, with MIT support EXPReS Annual Review: SA1

  21. SA1: the next 18 months • The focus of SA1 for the next 18 months will be on • Adaptive scheduling • Further integration of Mark5B in EVN operations • Streamlining operational data rate adjustments • Further improving the robustness and flexibility of the correlation process • Mixed-mode operations • Increasing the data rate beyond 1 Gbps EXPReS Annual Review: SA1

  22. Specific tasks • Fast/adaptive (re-)scheduling • Transform EVN into a truly flexible instrument • Based on automated download and execution of modified observing schedules - possibly on-the-fly • Reactions from stations vary from “excellent idea” to “over our dead bodies” • But will further improve reliability and responsiveness of EVN • All bits tested and in place, full operational test is planned EXPReS Annual Review: SA1

  23. Specific tasks (2) • Data rate adjustments • Packet dropping: • Drop packets at sending side (while preserving headers!) • Pad datastream at receiving side with dummy data • Channel dropping: • Selectively drop frequency channels at receiving side • High demand on cpu, need hyperthreading, right kernels, dual cpu’s • Mixed configurations: • Use different configurations at different stations (limited to 32, 64, 128 etc Mbps) EXPReS Annual Review: SA1

  24. Specific tasks (3) • Mixed-mode operations: • Enable all possible combinations of recording (at stations and/or at correlator), real-time and delayed data transfer • Make best use of complete EVN network, including stations that have less-than optimal connectivity EXPReS Annual Review: SA1

  25. Specific tasks (4) • Beyond 1 Gbps • 1024 Mbps will simply not fit on 1 Gbps... • Westerbork: use of DWDM equipment, dual 1-Gbps connection, channel bonding • Could be used to Jodrell Bank as well (dual 1-Gbps lightpaths to JIVE) • Onsala, Effelsberg: share 10-Gbps connections with eLOFAR, via VLAN (nearly in place) • Next, Torun? • Most sensitive telescopes in Europe soon at a full 1024 Mbps • And tests ongoing at 4 Gbps, from Onsala EXPReS Annual Review: SA1

  26. And soon to come... • Yebes first light (any moment now) • Hartebeesthoek: demo last month at 64 Mbps, S-African universities expect to be connected to Europe at 10 Gbps in 2009 • Arecibo: recent tests at ~512 Mbps • St Croix? • Urumqi: 622 Mbps connection (still in rumor phase) • Japan: talks ongoing about combining VERA telescopes with EVN in e-VLBI pilot project • Australia-China-Japan e-VLBI demo upcoming (at e-VLBI workshop in Shanghai) EXPReS Annual Review: SA1

  27. SA1- Past 12 month overview SA1 Deliverables EXPReS Annual Review: SA1

More Related