1 / 34

Timing and data-exchange between CERN and LNGS

Timing and data-exchange between CERN and LNGS. D.Autiero/IN2P3 Lyon, 6/4/06. The CERN and LNGS UTC systems The beam data exchange database The early warning signal. The Global Positioning System (GPS).

callum-holman
Download Presentation

Timing and data-exchange between CERN and LNGS

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Timing and data-exchange between CERN and LNGS D.Autiero/IN2P3 Lyon, 6/4/06 • The CERN and LNGS UTC systems • The beam data exchange database • The early warning signal

  2. The Global Positioning System (GPS) The GPS system was built and it is controlled by the US militaries (DOD), it is composed by at least 24 satellites orbiting at 20000 Km Depending on the place and the time of the day on average between 5 and 8 satellites are visible by an observer on the earth

  3. Each satellite is equipped with a Cs atomic clock and transmits its time and position (computed with its ephemeris): e.g.: x1,y1,z1,t1 The observer on the earth, getting the quadrivectors of 4 satellites has to solve a system of 4 equations in order to find his own time and position: x0,y0,z0,t0 (x0-x1)2 + (y0-y1)2 + (z0-z1)2 = c2(t0-t1)2 (x0-x2)2 + (y0-y2)2 + (z0-z2)2 = c2(t0-t2)2 (x0-x3)2 + (y0-y3)2 + (z0-z3)2 = c2(t0-t3)2 (x0-x4)2 + (y0-y4)2 + (z0-z4)2 = c2(t0-t4)2 Then x0,y0,z0,t0 can be converted in the more familiar Latitude,Longitude,Altitude and time

  4. GPS UTC clock system. Built in 1991 by the company ESAT located in Torino: www.esat.it Macro detector paper: NIM A 486 (2002) 663-707 INFN/TC-01/2001

  5. Master clock unit = GPS unit + 10 MHz Rubidium oscillator The GPS works normally in time only mode (its position is already very well known) Rubidium oscillator: Short term stability (Allan variance) better than 3x 10-12 (dn/n) over 100 s Long term stability (aging/temperature) about 1 micros/day Oscillator adjustment: Each second the local clock scale is compared to the GPS (20 ns units). The average of 60 of these time differences is produced every minute (Dt) 44 ns RMS Long term variations are adjusted using a linear fit of 180 Dt points (every 3 hours). After adjustment the residuals of the local time to the GPS are at the level of tipically 15 ns RMS on Dt

  6. LNGS time distribution system to underground labs: 2 independent master clock units are installed in the external laboratory. The UTC time scale is known at better than 100 ns. Every ms (1000 better than the 1 PPS standard) a synchronization pulse and the time/date string are sent through the 8 Km long monomodal optical fiber (1310 nm) to slave clocks in the caverns. The ligth pulse is converted in electric serial pulses. Optical signal sent every ms Edge (start of the ms) 80 bits code Date etc … GPS antenna Master clock GPS receiver 8 Km mono-modal 1310 nm optical fiber Slave clock(s) Rb clock Underground LABs External LAB Propagation time is measured once and subracted systematically at the syncronization level

  7. Slave clocks have to regenerate the local time with 10 MHz TCXO oscillators with a stability of 1 PPM (it has to keep on track over 1 ms). The time is then available in a 80 bits format and it is known at better than 100 ns MACRO was reading it with CAMAC modules BOREXINO with VME OPERA built its own slave clock unit (with much improved performance) completely compatible with the existing optical fiber distribution sistem

  8. The OPERA master clock card PCI cards replacing the old slave clock and providing the clock system in OPERA Totally compatible with the LNGS clock system (ESAT consultancy) Developed at IPN Lyon in 2003 Operative at LNGS since beginning of 2005 PPmS from optical fiber 10 MHz ovenized quartz local oscillator (reset every ms) almost a Rb clock: Allan variance 2*10-12 over 1s T stability 5*10-11 0°-50° Aging 7*10-8 over 5 years Master clock signal edge tracing within 20 ns

  9. Node card i SM2 SM1 Master card 0 MLVDS MLVDS O/E converter 1:2 splitter Optical fiber Master clock PCI card The OPERA clock distribution system Time stamp distributed in units of 100 ns to the detector nodes The FE electronics works in units of 10 ns. The DAQ works in cycles of 1 s (up to 256) with a fine counter of 10 ns. Delays are measured and subtracted at the nodes levels. The DAQ counters are cross-referenced with the UTC date stored by the master card 10 MHz high stability local oscillator 8 Km mono-modal 1310 nm optical fiber Sync. signal sent every ms (PPmS)

  10. CERN system Symmetricom (ex True-Time) XL-DC system Standard receiver specifications 150 ns peak to peak Rubidium oscillator Also used in K2K and MINOS Oscillator 10-9 Temp. Stability 10-6

  11. Which accuracy ? XL-DC standard also used in K2K and MINOS with the goal of « 100 ns » accuracy, what does it mean ? K2K XL-DC: • The LNGS system in the past was quoted with an overall UTC tracing accuracy better than 100 ns • We are aimining for CNGS at having two systems with comparable accuracies (<100 ns) which have then to be combined togheter. We planned some cross-checks: • The CERN system was sent to the Swiss metrology institute METAS. They performed a calibration and released a certification • 2) We organized to cross calibrate the two systems by bringing the CERN clock to LNGS and cross tagging the two systems for a long period. (this will not include all what comes after the master clock) + remeasurement of the optical fiber delay

  12. CERN-LNGS UTC clocks intercalibration Performed in collaboration among: J.Serrano. P.Alvarez, J.Lewis (CERN), D.Autiero (OPERA), S.Parlati, G.Di Carlo(LNGS) : Put the two UTC systems side by side: CERN system brougth to LNGS on 7/3/2006 and reinstalled with its own antenna and a DAQ chain and a time interval counter (300 ps res.) in order to monitor the time difference of the output of the two systems as a function of time. Data taking performed under various conditions till yesterday (5/4) The CERN system was calibrated vs UTC a few month ago by the Swiss metrology institute METAS (including antenna and 30m of cable) The cable delay is correctly accounted in the configuration Both CERN and LNGS will have a system with double units

  13. 10 MHz Rb oscillators Clock 1 PC for monitoring and control Optical fiber transmitters Clock 2 Active, connected to underground LNGS: Clock 2 active and connected to underground lab Clock 1 on, not connected (backup) Clock 3 (off) Clock 1 active (backup)

  14. CERN system Symmetricom (ex True-Time) XL-DC system (also with rubidium oscillator) CERN system installed in the LNGS UTC clock room with the DAQ chain PC+labview Fluke time interval counter PM 6681 GPIB-ethernet interface Digital scope

  15. Long run in auto-> time mode 10/3-22/3 (> 1 million of measurements, 1/s) Converged after 2 days to time mode with: N42° 25’ 15.4’’ E13° 30’ 53.0’’ Alt 1035 m asl 22/3 10/3 Time difference stabilizes on a plateau around 350 ns (clock2 anticipates CERN) CERN-clock2 (ns) Time (s) x 103

  16. 1 day=86400 s Plateau 355 Band of +-23 ns CERN-clock2 (ns) 1 day Time (s) x 103 Good cross-tracking of the two systems, which is the origin of the offset ?

  17. Fine details of the time difference evolution

  18. LNGS UTC Clocks block diagram nanostepper

  19. Discovered that the two LNGS UTC clocks which should be identical (clock2 and clock1) have an offset about of 140 ns, quite stable with time. Different software configuration ? How the antenna cable delays are taken into account in the configuration of the two systems Measurement 8/3 ns Measurement 22/3 ns

  20. Clock1-clock2 monitored over 13 hours

  21. The CERN system converged to the following coordinates: N42° 25’ 15.4’’ E13° 30’ 52.9’’ Alt 1037 m asl WGS84 The coordinates of the antenna used by clock2 are known from precise measurements: N42° 25’ 15.6’’ E13° 30’ 52.8’’ Alt 985 m asl WGS84 The CERN system if off by 50 m in altitude, it comes out that the same bias was noticed also at CERN (50m=167 ns). Both at CERN and LNGS this is confirmed also by other portable GPS systems Apparently both systems refer to the geoid model WGS84. Difference under investigation Perform the same measurement with the CERN system in time only mode with forced LNGS coordinates

  22. WGS84

  23. Long run in time mode with CERN system forced to known LNGS coordinates 22/3-5/4 22/3 Time difference stabilizes on a plateau around 240 ns (-110 ns) 5/4 CERN-clock2 (ns) Time (s) x 103

  24. <Dt>=355.6 ns CERN, Auto coordinates ns CERN, Forced LNGS coordinates <Dt>=248.5 ns ns

  25. Complete calibration of the time distribution chain: Telecom room Telecom room

  26. Conclusions for the UTC intercalibration : 1) The two GS clocks differ by 140 ns, check with ESAT their software configuration and how the antenna cable delay is accounted for the two 2) The CERN system tends not to converge to the correct coordinates by overestimating altitude by 50 m (a cheap GPS does it fine). Checks with METAS in progress, WGS84 ? 3) Once reached the plateau the two systems (XL-DC and clock2) are tracking each other in a band of 40 ns (+-20) and an offset of 356/248 ns. This is a quite nice result provided the offsets that we found are completely understood and under control 4) A viable scheme has been setup in order to complete the calibration of the LNGS time distribution system by measuring the delays (+jitter) of the chain The CERN system today has been dismounted and sent back to CERN (it is needed for the machine startup) First it will be sent to METAS with all the new informations for a cross-check, contacts with ESAT as well. We aim at fully understand the offsets Work in progress

  27. The CNGS-LNGS database • Needs: • We need to correlate the events recorded by the experiments with the beam spills windows (10.5 microseconds). This is done through the UTC times recorded at CERN and LNGS: T(LNGS)=T(kicker)+TOF, accuracy better than 100 ns. We need the list of spills UTC times from the CERN database • We need also to have online some rough, synthetic spills quality informations (intensity, status word) • We need a full database of the beam conditions for physics analysis and comparison with the beam simulations, (e.g. NOMAD) important for numu->nue analysis • We need to record in the beam database the feedback of the far (LNGS) beam monitoring: muons from neutrino interactions in the rock, neutrino interactions in the detectors • We need to be able to consult at LNGS the beam status, like it was done at WANF with a graphical application

  28. The CNGS database (ORACLE based) produces 1 TB/day of information, mostly technical and it is in the hidden accelerator controls network at CERN (LHC logging database) After a few meetings among OPERA/CNGS and the CERN Database people it was decided to implement a new gataway server (ORACLE) on the public network in order to exchange the relevant informations among CERN and LNGS for the data-taking, beam monitoring and beam analysis: The data will be available on the public gateway about 15 minutes after recording in the main database The database is structured per CNGS cycles The server will be bidirectional in order that also LNGS experiments will be able to write the events observed in correlation to the spills (some discussions started in OPERA and LVD) A new server has been bougth A selection of a subset of the variables of the main database to be copied for a complete beam physics analysis has been discussed among Edda,Paola and myself

  29. Status: (Ronny Billen) Scheme under test in a development environment (not yet with the final server machine) Database scheme ready Data transfer procedures under development Definition of CNGS relevant variables (99 up to now) ongoing Data flow will be tested as soon as the DAQ chain allows for it The user documentation will be provided and made available on the CNGS site

  30. Early warning signal It is useful to know in advance the arrival time of neutrinos in order to prepare the DAQ and prevent operations which could interfere with the neutrino events recording (e.g. OPERA from time to time performs calibrations with flashing LEDs) The next neutrino spill time can be predicted in advance of several seconds. The information can be transmitted through the network EW Two fast extractions/cycle 10.5 micro seconds Separated by 50 ms Window for calibrations

  31. SPS Cycle Proposal for 2006 • Commissioning: 12s FT + 6s CNGS • Run 1 12s FT + 6s CNGS + 4.8s MD • Run 2 12s FT + 3x6s CNGS + 4.8s MD

  32. An experiment like OPERA is continously taking data, independently of the beam informations (data-base and early warning access through the network): 1) The beam informations are needed to validate the bricks extraction once per day 2) The calibrations can be conservatively skipped for some time in case the early warning signal does not arrive. In case the EW is there the calibrations will be performed till a conservative window around the neutrino spills The transmission of the early warning from CERN to LNGS has been implemented and tested since the beginning of March by J.Lewis on the OPERA DAQ gateway computer 1) It is based on UDP packets. The transmission time is around 10 ms 2) The packets are sent every 1.2 s (all possible SPS cycles are multiple of 1.2s which is the PS-booster cycle). The packets contain the date of the future neutrino extraction. This date can be the same for many packets of the same cycle. The idea of the 1.2 s repetition is to keep a constant rate link among CERN and LNGS 3) The DAQ looks at the arrival time of the packet, compares to the extraction time written in the packet and decides if there is time or not to perform calibrations

  33. The UDP packets can be transmitted from the CERN machine also to other LNGS nodes (we have to decide if it is more convenient to dispatch from CERN to many LNGS nodes or to use one LNGS node as repeater) The documentation will be put on the CNGS site

More Related