320 likes | 442 Views
Dimuon Forward Spectrometer May 2008 Cosmic Run. Gener al Planning +To Do Lists Day by Day report. General Planning. muon tracking has to run in a dedicated trigger class with muon trigger and V0 : trigger send by muon trigger test stability
E N D
Dimuon Forward Spectrometer May 2008 Cosmic Run • General Planning • +To Do Lists • Day by Day report
General Planning • muon tracking has to run in a dedicated trigger class with muon trigger and V0 : trigger send by muon trigger • test stability • test calibration trigger mixed with physics trigger • run with pulser (600Hz) from muon triger • DAQ/TRIGGER (cluster of detectors triggered by different trigger classes) tests week 15 • Planning • run with St1, St2 and Ch5 (ventilation and temperature probes is mandatory) • include part of St4 or St5 ?! • station 9 is the candidate
To do list • LC2 installation : • do not overwrite the files in the $LC2_CONFIG • go to a new directory /opt/daq-MCH/ECS/ • Trigger dispatching crate : • missing kernel (Anton ?) • user interface vs smi • CROCUS software upgrade • Finish the DA/script for calibration… • Probably a full revision of the DCS FSM is necessary ; it will imply some work on the ECS side as well. “Long term thinking” to have a safe operating procedure for the muon spectrometer. → St1 : ~45 manus and 1 patch bus to be changed → St2 : few manus to change, fix problem with crocus 2565 (backplane vs software ?) → St345 : check with Herve.
To do list • temperature probes in St3 • pressure sensors in St 2 (the one for St1 is not responding properly, having the value from St2 could help to know if the problem come from the ELMB readout) → Need to finish MUON interlock (if ventilation goes off or gas stop flowing…) → HV trip problem : could it come from the gas purity ? This need to be check urgently !! → Send trigger PS back to Wiener. Ask them to check the problem detected by L.Wallet (bit rate transfer is not save when rack’s tunred off). + this PS sometimes switch off (apparently because of a over/under current) ; we still need to investigate the origin of this.
Tuesday 8th April • DCS intervention (network, racks powered down) of Monday 7th is still has still some effects... • Status of the low voltage power suplies : • some problems • CH1R and CROCUS R are missing (still in germany) → A document summarizing the different issues/changes should be written ! • DCS review of the detector (see next slide) 4pm meeting :
DCS review : To Do List for MCH • New DCS framework installed v3.2.0 • but FSM component in this version is “too” old and need to updated • new component with the “dim upgrade” compatible with ECS should be available soon • → get new version from alidcs website • Cleanup : PVSS projects (on all worker nodes) should contains only 2 directory (that will be backup) • mch_env and mch_components (not sure about the correct names…) • Alarm documentation • Interlocks : retrieve the DSS signals (ventilation and gas) and implement some software action. Delay between a stop of ventilation and sending the interlock to LV should stay around 10 min. • Use of majority • Common repository scheme (to minimize the transfers over ethernet) : all PVSS project files should be put in a common directory (path to be defined) on the operator node. This way everything is loaded at the beginning and the global DCS only needs to communicate with the operator node and not/less with the worker nodes • Local dim dns server should be set to the operator node • Access control list : users and privileges must be checked (Peter Chochula) • Archiving : is the smoothing applied ? • [are any PVSS recipes used ? (I don’t think so, not so easy to setup according to Marco). Could be really convenient to define a full set of values depending on the use of DCS, tests or physics] • MCS (magnet monitoring) should be included in the MCH DCS. Framework component are ready (See with lionel wallet) Only for MCH (not part of the DCS review) • Modif : in the sequence to turn on the low voltage, the order should be first LV for the chambers and then LV for CROCUS. Doing all at the same time may be not so good for the CROCUS. • Modif : could be convenient to have a DCS button that power cycle the electronics (CROCUS+ LV) without changing the HV status.
DCS review : update with Ivan (13/05/2008) • Normally it is ok now • done • needs to be done by someone (not Ivan) • Interlocks : not done • Majority will be set to a full chamber; if all HV trips error is send to ECS and will stop the run ! • probably ok; needs to be check • ok • Access control list : to be done • Archiving : smoothing is applied. Up to now it’s a 5% change of the value that triggers the writing. Should be set a 1 volt • PVSS recipes: not used. see later • MCS (magnet monitoring) : Ivan will ask Lionel Wallet to do it for us. Only for MCH (not part of the DCS review) • Modif switch on sequence: Ivan will do it (setting up a delay when switching on a single LV = crocus) • Modif power cycle : not so easy to implement in the actual scheme. Especially when one considers that control is given to central DCS. Ivan may try to do something + should we use the ECS command “configure” to power cycle all the low voltages (at beam tuning state) + what about GMS. Ask Raphael if he thinks that GMS panel should be merged with MCH panel. Ivan has saved some space to do so.
DCS review : Marco (27/05/2008) • ok • not properly done. For instance the directory D:\PVSS_Projects should contain only the folders mch_ui and mch_ui_fw (the present name mch_ui_fwComponents should be changed) • needs to be done by someone (not Ivan) + alarms instructions • Interlocks : not done (one should also consider software interlock for both LV and HV). • ok; being tested • not done : ask Ivan to check according to peter rosinsky’s email from 23/05/2008 • ok but one should switch to global dim dns and keep the possibility to go back to local • Access control list : to be done + implement the different action possible according to the user rights. • ok • PVSS recipes: not used. Not easy to implement • MCS (magnet monitoring) not done yet. Only for MCH (not part of the DCS review) • Done • Modif power cycle : ECS command “configure” to power cycle all the low voltages (at beam tuning state) implemented • GMS : Raphael is ok prepare a GMS status panel that we could display in the MCH DCS to have the basics information.
Wednesday 9th April • Few trigger tests done to answer the trigger questionnaire • some missing items clarified by Ben Yun. • need to get back the completed version the document from Trigger group • DAQ/Trigger with multiple clusters/partitions • to be continued • Some “offline discussion” with trigger people : Bogdan couldn’t see the trigger word in the CTPraw. At the same time an error was reported in the offline meeting by Cvetan that CDH have problems… Some info about the detector mask are written in the OCDB (some rcfg files indexed by run number). • example of run 24841 and 24836 4pm meeting : 5pm meeting :
Thursday 10th April • Tests with trigger when adding delay in the L0 (from 8 ns to 26ns) : no effect seen. • fix the problem with the IP address of CH2L • CROCUS 2565 is back in the weird state observed during the cosmic run : decision is taken to remove the backplane so it can be fixed in Orsay (a tested backplane will be send by mail). • Dispatching (Gerard program’s name) can reset the crocus, but this is still different from a power cycle. Might needs to reprogram some FPGA. → Some work on the calibration script to get the calibration parameter file send correctly in the OCDB via the shuttle. 4pm meeting : 5pm meeting :
Friday 11th April • Continuing tests of DAQ clusters with 3 trigger classes • runs ok (with muon trk, trg and hmpid) • Test of overlapping DAQ clusters with 3 trigger classes • successful (for the first time ever) • Test of overlapping DAQ clusters with 3 trigger classes + another partition in parallel with acorde only • successful as well… • Some problem during global runs : CROCUS get busy because either a hard reset was send or a power happened • PVV advised us to test the readout with a little trick : generate a busy comparable to DAQ high back-pressure (eventDump : -i –M see help) → Latest version of the ECS scripts send to Vincent to build a new RPM installing program in the proper directory (/opt/daq-MCH/ECS) 4pm meeting : 5pm meeting :
The trigger crate PS • 29/04 start to cause again some problems (rack has been switch off) • no spare at the cern pool • only spare is in Orsay and in use for soft dev. • A new power supply unit will be ordered to wiener • Reparation for the water leak still not done !
Wiener PS • A (short) report from Herve : Ch1L and CrocusL LVPS are in place. Concerning LVPS, are missing : - outside : Ch7L (at Wiener for noise modification) - inside: - Ch7/8R (I think also at Wiener , also for modification. It was one chamber that had been sent a long time ago on which Wiener should have done the first modif planned (change from 10 to 100 nF); they found that this LVPS had not the good specification and modified at that time an other one; the Ch7/8R is then still at Wiener, they had repaired it and they will also modified it for noise). - Ch9R1: should be still at CERN electronic pool: was send to EP because they wanted to make tests for noise (Jens must tell me if it has still this one). - MUON_TRK ORSAY: still at Wiener to change the water join - MUON_TRK SPARE: still at Wiener to change the water join I believe we need still a very summary (excel file like) of what has been send and received (with date), modified, characteristics versus serial number vs position (which chamber/crocus, Inside or Outside). So it will be possible to follow the noise results versus the PS characteristics.
DAQ/CTP test of station 2 • Crocus CH3L fixed in orsay • DAQ test on April 29 and 30 • get results summary from Mamu
Monday 12/05 • To Do list for MCH (by PVV) : • Send trigger test protocol to MCH (Marian) done • Update MCH trigger questionaire • Schedule CROCUS software releases (to be tested in week 21) • Assert busy IF TTCrx_READY or QPLL_Locked cleared • Reset FEE buffers on TTC_FEEReset done • Reconfiguration for L0-L1 time done • Trigger error detection L0 missing, L1 missing, L1 message missing, L2 message missing done • Errors in CDH block length done • Test of MCH station 2 • Check that detector is not blocked with BUSY asserted during data taking • Impossible to read correctly all the temperature sensors since may 6th…. It was fine before. Something changed ? [email send to Herve, Andre and Ivan] • Need to get some info on the gas purity measurements • Interlocks status ??? 16h30 meeting : Only a few subsystem present. Access to FASS ok (but one should first get the appropriate token at Bldg55)
Tuesday 13/05 • Discussion on DCS new features with Ivan. He will try to implement some of the required functionalities. • Fixing Station 1 bad FEE (dead channels on manu boards) • Quick readout tests on CH5. Noise is very/too high : 25 on Bending and 50 on NonBending • not possible to measure the effect of dipole on the noise • probably hard to use it in the tracking for next cosmic run • Impossible to read correctly all the temperature sensors since may 6th : Andre will have a look at it. Need to get some info on the gas purity measurements Interlocks status ??? Ivan in vacation the week before the run ! We really need a member of dimuon tracking to learn the basics of DCS 16h30 meeting : DAQ off Thursday 15th in the morning. Tests of CTP trigger classes up to 50 (with T0 and SSD-> problem at 32 classes/bits). MTR second DARC being installed
Wednesday 14/05 • Fixing Station 1 bad FEE (dead channels on manu boards) : done • HV on chambers 5 : need to check the gas flow. JL Charvet will have a look. The idea is to power up HV over night to have an idea of its behavior (stability, trips/no trips, etc…) • gas flow needs to be check before. • Some (very) good news on slats (Ch7O) noise…? • Impossible to read correctly all the temperature sensors since may 6th : Andre will have a look at it ELMB expert will come tomorrow in the FASS to check the ELMB board • reading of temperatures is mandatory for the cosmic run Need to get some info on the gas purity measurements contact with Antonnello : gas analyzer ready in two weeks + possibility to install O2meter on exhaust gas. Interlocks status ? Still to be checked 16h30 meeting : DAQ off Thursday 15th in the morning + DCS. Tests of CTP trigger classes still ongoing. MTR second DARC installed. Most of subsystem are working standalone…
Thursday 15/05 • Gas problem on Rack 61 : status is off • send email to gas.support@cern.ch or call “le piquet gaz” 162516. • this has been solved with the gas group: it was related to a pressure transducer alarm. St1 and St2 pressure transducer have been fixed in the meantime. • ELMB reading are back (S. Drouet and me) disconnected and reconnected the cable then restarted the ELMB management hygrometer and all temperatures reading are there. Station 3 ELMB is chained and can be tested now ! • Pressure sensors are back : St1 PT6104 (stable readout 0.36 mbar under alarm) on gas rack 61 and St2 PT6204 (readout 0.43 mb, stable, not alarmed) on gas rack 62. • HV/LV on chamber 5 to be done : temperatures readout, gas check, ventilation • Chamber 6 is in place ; thus the whole tracking is now installed ! Let’s read it out… Infos on the gas purity measurements : 02 measurement on Rack 61 exhaust (20ppm -very good-, new ongoing measurement overnight); idea is now to check the gas is really 80/20 as it is given by the mixer (wrong reading seen before by the gas group). Coming soon… Interlocks status (ventilation to LV) : I add a discussion with A. Augustinus. Need to pull more cables for interlocks. He can provide connectors for interlocks. 16h30 meeting : Skipped but… Wednesday 8h30 14h perturbations on electrical network, up to 40% over-voltages during 50 ms. Advise to switch off everything.
Friday 16/05 • “Old” OPC server crash occurred again after a HV trip • Ivan will try to upgrade again CAEN firmware • Status of chamber 5 : readout, gas, HV, ventilation, temperatures • J.L.Charvet ramped up high voltages in the evening; all stable except 2 slats for which voltage is set lower Infos on the gas purity measurements : 02 measurement on Rack 61 exhaust : latest measurement give 120 ppm which is bad. Need to be check again on a “reference” input gas. Gas group will be there Monday to check this and Chamber 5 gas distribution/leak. Interlocks status (ventilation to LV) : Additional cables will be pull for ventilation interlocks. Need a connector chain to propagate interlock signal, Andre will give us the connectors. Visualization of signals from DSS in DCS coming... 16h30 meeting : skipped
Saturday-sunday 17-18/05 • Almost nobody working in the ACR/cavern (SSD,TRD,MTR,MCH) • Readout test on station 1 to make it fully ready • try to understand some funny reading of the gas flow : • output flow is larger than the input flow ? • email send to date-support to ask ECS to send SOR/EOR sequence to MCH in order to test HV archiving/shuttle. 16h30 meeting : DAQ/Trigger test successful. 3 detector clusters (SSD,MTR,SSD+MTR) and 31 trigger classes (random trigger) ran ok with 5 GDC. Handling of busy (when 1 det. is busy the other one continue) and GDC busy tested ok. • ramp up high voltages on St1; some problems. Readout is very stable. • some trips occurred, it had the positive effect to bring the confirmation that HV trip do not seems to damage/destroy manas input as it has been suspected • input gas flow re-tuned for all St1 chambers 16h30 meeting : Wednesday power upgrade 8.00 to 14.00 !!! DAQ/Trigger test still ongoing. Probably DAQ/Trigger night shifts will not happen next week since too few subsystems are ready for overnights runs.
Monday 19/05 • gas stopped all morning for software upgrade of the gas DCS : • try to ramp up St1 HV when no Ar/CO2 is flowing : still some problems • gas will be stopped until Wednesday ! Not really since it starts to flow again from 16h… but there will probably be again interruptions tomorrow • DCS new features thanks to Ivan : • Configure : power cycle the FEE/Crocus without modifying the HV • First implementation of majority Trying to plug a crocus from Chamber 9 in the global readout. Still need some work on chamber 5 : readout, gas, HV, ventilation, temperatures 16h30 meeting : CTP readout gives errors at the vent builder level when included in the DAQ. DCS OPC installation/upgrades detector by detector over the next 2 weeks. PHOS cabling ongoing. Some funny CDH errors (timing) in global partition (SSD,T0,TRD,ACORDE) seen at constant intervals…?
Cooling of tracking chambers : ventilation • Settings for the muon tracker air cooling : • T= 18°C • point de rosée 11°C • ΔP = 400 Pa (maximum at 600) • Observed • T = 18.1°C • point de rosée 8.8°C • Etat actuel de la ventilation du bras Dimuon releve sur l'ecran de controle vendredi 16 mai: • Ventilation ST1-2 (UAPX 281): T consigne 18C avec Point de rosee 11C. En sortie de l'unite: T sortie 18,1C Point de rosee 7,2C Pression de sortie 400 Pa. Soufflage ST3 et Trigger (UAPX 282): T consigne 18C avec Point de rosee 10C. En sortie de l'unite: T sortie 17,9C Point de rosee 6,8C Pression de sortie 280 Pa. • L'equipe Tracking n'avait pas termine ses travaux sur la station 3; des plaques de bois couvrent encore partiellement les diffuseurs. L'equipe Trigger avait termine l'installation du soufflage au pied des chambres; les 8 clapets terminaux sont maintenant en position ouverte. Alain Tournaire • http://www.cactus2000.de/fr/unit/masshum.shtml
Tuesday 20/05 • Problem with ELMB (central DCS) in the morning, fixed rapidly. But the readout MCH temperatures didn’t came back : • restarting the project on alidcscom051 didn’t help • blocking process (..Can25+…exe) had to be killed ! Then project restarted normally • Something wrong in the DIP publication of the DSS data points corresponding to interlock of the ventilation of St12,St3 and St45. Values do not appear in the DIP browser…. This need to be check with Andre and/or DSS people • what should be the returned value ? true = interlock on ? thus no ventilation ? • Ventilation of St3 and St45 should be started • Readout of 1 crocus of chamber 9 : • Configure : power cycle the FEE/Crocus without modifying the HV Still need some work on chamber 5 : readout, gas, HV, ventilation, temperatures 16h30 meeting : L3 doors will be closed on june 3rd. Ongoing work to understand the problem of multiple trigger types : software and physics trigger needed for MTR.
Wednesday to Sunday 25/05 • Percentage of Chamber 5 readout • ?? • Percentage of Chamber 9 readout • ?? • DIP publication of the DSS data points : now checked • St1&2 signal were correct but St3 and St4&5 signals were forced to false. Now all values are readout. • what should be the returned value ? true = interlock on ? thus no ventilation ? • Answer from Andre : if ventilation is running ok the signal published in DIP should be true. Still need some work on chamber 5 : readout, gas, HV, ventilation, temperatures
Wednesday to Sunday 25/05 • New Crocus software installation : • V1.0 of trigger frame error management • Handles L0, L1a&L1am missing at low rate (1% of error) • Does not handle L2a/L2ar missing • Does handle TTC init protocol with 1s busy after its reception • CROCUS no longer busy after initCrocus.x execution to be compliant with run starting sequence • This version has been installed over the whole station1 • Tested in global run for 1,3 Million events • Must be more thoroughly tested to improve stability • Tunable time between L0 and L1a signal • Implemented, not yet tested
Wednesday to Sunday 25/05 • Caen OPC server upgraded (problem seen : everything was working fine excepted that one couldn’t raise the HV). • this has been fixed by Ivan • Results from the gas group on Ar/CO2 mixing proportions : appeared that the mix was wrong and subject to large fluctuations (measurement done just after the mixer, could be better in rack 61/62) • Les massiques CO2 et Ar ont été contrôlés. Co2 3% négatif Ar 20% positif (déviations a pleine échelle) L'analyse CO2 actuel en ligne dans le mixer oscille de 18.7 à 20.5% Co2. La proximite du point d'analyse peut expliquer ces variations. Albin Wassem
Monday 26/05 • Introduction of a GetCrocusRes.x after each command in ECS scripts (required to change time alloted to CONFIGURATION.sh to 40s) • still some problems with validator (ldPed) • Scripts ECS will be move in a new directory : sent the exact path to S.Chapeland (/opt/daq-MCH/ECS). • Beam_Tuning state does not accept the SOR (in READY SOR/EOR is Ok). Email send to Ivan. • this implies that the pedestal run can only be done in ready state for the moment… • Station 1 crocus support new software version compatible with latest trigger requirements (excepted some error detection). • tested with Run 32672 (physics) in global partition, SOR/EOR phases ok. Need to check if HV have been transferred to OCDB. Chamber 5 status: readout, gas, HV, HV+source, ventilation, temperatures 16h30 meeting : . TPC took 18 Mevents of Kr data over the week end. DCS sessions started; MCH tomorrow at 14h00. Lots of subsystems have problems are are not ready to be included in the global partition yet.
Tuesday 27/05 • A few standalone runs at 40MHz to test the new crocus software • 32856 : 6.7 Mevents… • MCH (St1) included in global run : seems ok. • New crocus version will be propagated to others stations • Amanda shuttle fixed run 32881 should sent HV to OCDB • HV values retrieved by L.Aphecetche : looks coherent with what is expected. • Station 1 crocus support new software version • tested in Alice_multi with MTR/MCH (run 32970) in two trigger classes + generating busy (by monitoring all event) at the ldc/gdc level ; both clusters ran fine independently. Around 10. and 7 Mevents for MCH/MTR @1800Hz/1200Hz without errors ! Chamber 5 status: readout, gas, HV, HV+source, ventilation, temperatures 16h30 meeting : Lot of debugging for all detectors. Not so much global runs. MTR gas not before Friday. CTP interruption in the morning.
Wednesday 28/05 • More standalone runs at 40MHz to test the new crocus software • Trigger review : 3 issues spotted • re-configuration of Lv0-Lv1 time not ready (need the interface) • some runs do not start because of busy (no systematic, not understood yet). It usually happen when we are doing a DAQ or Trigger test ;-) • L2messages missing error not reported yet All three should be fixed in the final (end of june) version • ECS scripts now located in /opt/daq-MCH/ECS on aldaqpc089; change to all other ldc will be done tomorrow. Chamber 5 status: readout, gas, HV, HV+source, ventilation, temperatures 16h30 meeting :
Thursday 29/05 • Station 1&2 in standalone with new crocus release : seems fine • Need to change Crocus PS to be send back to Wiener • test with a modified on the left side : not done tomorrow morning • St1&2 in global partition : • worked fine (with acorde trigger at 100 Hz) • try with higher tomorrow Chamber 5 status: readout, gas, HV, HV+source, ventilation, temperatures Chamber 9 status: to be checked 16h30 meeting : skipped
Friday 30/05 • Quick DCS test (beam_tunning->ready->beam_tuning) in the morning: • MCH (and MTR ?) ok • Need to change Crocus PS to be send back to Wiener • can not power up the crocus properly (3.3 requested doesn’t work, sense voltage appear to be too low) • Agree with Herve not to send them for modification : test will be performed on surface during the cosmic run to understand • St1&2 in global partition : • worked fine (with acorde trigger at 100 Hz) • try with higher tomorrow • Problem with wiener PS IP address: when PS are moved from one position to another, IP address need to check/set BEFORE plugging the ethernet cable! This is very important. Chamber 5 status: readout, gas, HV, HV+source, ventilation, temperatures Chamber 9 status: to be checked 16h30 meeting : new DAQ/ECS release in the morning. Problem with trigger mix in physics trigger CTP do not set the correct mask, and DAQ has to remove these events; not acceptable for the muon trigger. Gas for MTR starting on Monday. TPC data with ITS planned this week-end (do no work… I am writing this on Saturday ;-)
Saturday 31/05 and Sunday 1/06 • Take some standalone runs with St1&2 • problems with left CROCUS PS IP address solved in the morning when PS are moved from one position to another, IP address need to check/set BEFORE plugging the ethernet cable! This is very important. • HV set at 1200 and 1550 for all operating channels : no trips. • 8 modified power supplies arrived on Friday • installed on station 3 • not configured can not test readout yet. Chamber 5 status: readout, gas, HV, HV+source, ventilation, temperatures Chamber 9 status: to be checked 16h30 meeting : no meeting