170 likes | 344 Views
USNO Software Correlator: Status Report. Dave Boboltz, John Spitzak , Nicole Geiger, Dave Hall, Alan Fey, Chris Dieck. Outline USNO/WACO Background USNO Prototype Software Correlator (UPSC) Use of DiFX Recent Correlator Developments/Additions. USNO’s Role in the Geodetic Community.
E N D
USNO Software Correlator: Status Report Dave Boboltz, John Spitzak, Nicole Geiger, Dave Hall, Alan Fey, Chris Dieck Outline • USNO/WACO Background • USNO Prototype Software Correlator (UPSC) • Use of DiFX • Recent Correlator Developments/Additions
USNO’s Role in the Geodetic Community • USNO is the International Earth Rotation and Reference Systems Service (IERS) Rapid Service/Prediction Center for Earth Orientation. • USNO has several functions within the International VLBI Service for Astrometry and Geodesy (IVS). • Correlator Operations Center. • VLBI Data Analysis Center. • USNO provides several levels of data products. • Daily EOP (rapids). • Weekly EOP updates (Bulletin A). • Long-term (periodic EOP, TRF, CRF solutions).
Geodetic VLBI and the IVS • Geodetic VLBI is coordinated through the IVS. • http://ivscc.gsfc.nasa.gov/
WAshingtonCOrrelator (WACO) • Current MarkIV “hardware” correlator was designed and constructed by MIT Haystack Observatory and is operated in cooperation with NASA. • 1 of 4 IVS correlators • WACO, Haystack, Bonn, Japan. • Only 2 MarkIVcorrelators still in operation. • Experiencing end-of-life issues. • Move to software-based correlator.
WACO Workload • Daily “Intensive” experiments • Used to measure variable rotation rate of the earth (UT1-UTC). • 7 days/week: single-baseline, 1-hour experiment. • Occasional 3-station intensives. • VLBA (MK-PT) intensives. • 1-hr / day, 7 days / week • Weekly R4 experiments • Used to measure EOP (polar motion, nutation, precession). • 1 per week: 5-8 stations, 24-hr experiment. • Other experiments • 1 per month: 2-4 stations, 24-hr CRF experiments. • 4 to 6 per year: 24-hr “R1” experiments backing up Bonn. • Occasionally: other 24-hr experiments (APSG, CONT, etc).
USNO Prototype Software Correlator • Heterogeneous cluster. • 5 nodes (workstations) • 32 cores • 2 Mark5 data playback units. • 1 Gb/sethernet switch. • DiFX package installed. • Proof of concept. • Correlates geodetic VLBI data. • Correlates MK-PT Intensives. • Procurement of full software correlator cluster.
DiFX Processing of Geodetic Intensives • Single baseline KK-WZ • Currently correlating Friday intensive in parallel with h/w correlator • E-transferred data written to Mark5 modules • Correlated on UPSC • difx2mark4 • Fringe-fitting in HOPS/Fourfit. • Geodetic databases created.
USNO-NSF-AUI/NRAO MoA • USNO contracting with NRAO for: • Access to MK - PT baseline. • 1 hr intensive. • 7 days/week. • Continued VLBA participation in the RDV experiments. • Six 24-hr experiments per year. • Vital to the CRF and TRF. • Requires high-speed connections.
Status: MK-PT Fiber Links • Pie Town link (1 Gbps). • Available March 2011. • Numerous transfers of VLBI data. • 100 - 300 Mbps to USNO. • Contract with NRAO. • FY13 Contract in process. • Mauna Kea link (1 Gbps). • Installed and available July 2011. • Numerous transfers of VLBI data. • 100 - 300 Mbps to USNO. • Contract with Univ. of Hawaii. • FY13 contract signed.
VLBA MK-PT Intensive Experiments • IVS Intensive4 (INT4) series • IVS Nyyddd • Dual S/X band • 4 S-band, 12 X-band • 32 MHz/channel • ~40 min. / experiment, ~30 scans • 16 second scan lengths • Limits data to be transferred • Source lists from USNO, ICRF def. • Data fringed and analyzed. • Oct. 2011 – May 2012
UT1-UTC Results Leap Seconds
UT1–UTC Results: Comparison IERS C04 • Oct. 2011 – May 2012. • IVS 24-hr sessions (red). • IVS Intensives (green). • wrms = 16.1 msec. • MkPt Intensives (blue). • wrms = 54.5 msec. • Un-modeled motion likely in Pie Town antenna.
New USNO Software Correlator • USNO contracted with NRAO to build correlator. • Phases I & II • Leverage NRAO DiFX/Cluster expertise • Phase I (Sept. 2012). • Replacement for current 15 station hardware correlator. • 30 compute nodes. • 16 cores/node = 480 cores • Delivered to USNO Friday! • Phase II (2013 - 2014). • Correlator expansion. • 30 station VLBI2010 compatible
Phase I Correlator • 2 infrastructure nodes • 1 head node • 30 compute nodes • Diskless • Mount operating system and DiFX from infrastructure nodes • 10 Gbps Ethernet network • Arista 10GbE switch • 64 SFP+ ports • 40 Gbps bonded connection to SAN
Phase I at NRAO, Socorro 7 node Test Cluster will remain at NRAO 33 node Production Cluster
Phase I Delivered to USNO Mark III correlator Mark IV correlator Seismic rated rack Cluster hardware
Future Plans • Get the new correlator up and running • James Robnett (NRAO) visit to USNO 24-28 Sept. • Unpack, rack, power-up, network connections. • Connect to SAN. • Test correlations off SAN. • Connect Mark5 units via 10GbE • Emulex OCe10102 10 GbpsiSCSI network adapters installed. • 6 Mark5 units available now. • Parallel operations. • Run both correlators in parallel for ~6 months. • Compare results. • Phase out Mark IV correlator • Go operational with MK-PT Intensives