FED Status & Production Plans

1. FED Status & Production Plans CCLRC, Rutherford Appleton Laboratory, Oxon, UK Imperial College, London, UK Brunel University, London, UK presented by John Coughlan RAL http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

2. 9U FEDv1 Production Summary Productions Jan 2003 : 2 boards. Working. June 2003 : 3 boards. Working. Oct 2003 : 6 boards. Major problems. Not for use outside UK. March 2004 : 6 boards. In production. New manufacturers. Locations of FEDs outside UK 3 at CERN in Tracker Lab. 1 at PISA since last Tracker week. • FEDv1 recent manufacture • FEDv1 test activities • FEDv2 design changes • Schedule http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

3. FEDv1 Recent Production 6 FEDv1s assembled by DDi Technologies, UK. Manufacture was carried out in a very professional fashion. Thorough Quality Controls. Good communications between technical staff at DDi and RAL. Testing: Automated Optical Inspection; Ersascope and X-ray on BGAs ; Flying probe tests on assembled boards. Delivered on schedule in March. All 6 boards are now fully working. 2 are at CERN since couple of weeks. Comprehensive production report also provided with suggestions for improvements for next pcb design... Conclusion: Very positive experience. Promising candidate for production manufacture. http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

4. FEDv1 Recent Production http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

5. FEDv1 Design Testing Tests completed since Feb Tracker Meeting: • Front-End FPGA Algorithms : Zero-suppression working. • S-LINK : High rate random triggers @ 80MHz link readout working. • TCS : Throttle signals tested. • FPGA reprogramming: Compact Flash can now be reloaded via VME. • Baseline Firmware is complete. • Improvements are now being made to diagnostics, error handling, resets as described in URD v0.54 Hardware Testing • Power : Power & Currents well within limits. Original requirements of 300A on both 3.3V and 5V . • Temperature close to limits @ Front-End. • Protection : Over-temperature shutdown working. http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

6. Control via VXI-MXI-2 APVE VME Backplane Sim. TCS Slink Transmitter FED Controller Transition Card (ECAL) Sim. FMM Clock & Trigger FED Generic PCI Card Slink Receiver Slink Controller LVDS Cable S-LINK Tests (James Leaver) Hardware Throttle http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

7. S-LINK Tests (James Leaver) • Trigger FED using APVE and read out test events via Slink – check received data for errors • Current maximum Slink verification rate: • No errors observed so far • but at current data rate, would take ~146 days to guarantee no more than one error per week of CMS operation • With random (Poisson distributed) 100 kHz triggers • Sent ~1108 triggers, events successfully received • Verifying payload data transmitted at 100 kHz by sending triggers in controlled bursts (reducing average data rate to < 4106 words/sec) • No errors so far • Will shortly begin testing FED performance in ‘Frame Finding’ mode, using fake data from FED Testers • ‘Real World’ test of FED at high trigger/data rates Conclusion: FED can operate at 100 kHz http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

8. Sim. FMM FED S-LINK Tests (James Leaver) Optical (Simulated) APV Frame Data FED Tester Sim. TCS Clock & Trigger from FT Hardware Throttle from APVE (combines FED and APV buffer throttles) APVE Hardware Throttle from FED http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

9. Will components on FED exceed operating temperature in CMS? Analogue OptoRx < 70°C Differential Buffer < 85°C ADC < 85°C Need to emulate conditions we will have in CMS LHC crate. All measurements adjusted for intake air at 18°C. 3 FEDs placed next to each other in crate. The 2 external FEDs restrict airflow and mimic to some extent heating from crate of FEDs 1 FT Ensemble used to drive all FED channels with 100kHz frames. Temperature Testing (Greg Iles) http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

10. Position of temperature sensors ADC Analogue OptoRx 0 Differential buffer Analogue OptoRx 1 Analogue OptoRx 2 Not visible.... Built in FED OptoRx temperature sensor is on opposite side of the PCB http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

11. Temperature of ADCs & Buffers • The following have temperature range of -40 to +85 °C • ADC (AD9218) • Differential buffer (EL2140) • Predicted temperature in CMS by: • Measuring temperature between 2 thermocouples • One in air flow below crate • One attached to top of component with araldite (near top of FED – i.e. where it gets hot) • Added expected air temperature from heat exchanger (18 °C). • Measured devices twice. All values with 2 °C. Worst case shown. • Nominal fan speed of LHC crate = 3000rpm http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

12. Temperature of Opto Receivers http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

13. Assume heat exchanger can provide air at 18 °C. We will place a large amount of power in localised region of heat exchanger. 100kHz, “typical” frames that I have used match those in CMS. AOH driving minimum laser signal necessary for link (i.e. keeping power dissipation in experiment to minimum) FED in scope mode, 6 samples and delivering data to s-link oblivion. Only have 1 FED either side if FED under test and left hand side FED is at only ~70% power. Can we achieve a fan speed of 3480 rpm in a CMS rack. FEDs will provide a greater impedance to the air flow, but there will be more fans. How well is fan speed correlated to air flow? Caveats FED Analogue Warm Cold Heat Exchanger Hot Warm FED Analogue http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

14. FEDv2 Pre-production Board Baseline functions of FEDv1 are now verified. Minimal Hardware changes for FEDv2: • Power Block : FET controller. • QDR Memory : Replacement part identified. • FE FPGA : 2M gate pin compatible with existing part. • ADC : AD9218 Device Bug. Reduce gain by half. Simple mod. • FPGA Configuration : VME Boot device reprogram via JTAG cable. • S-LINK & TCS Signals : New VME Transition Card. • TTCrx : Intermittent problems now understood. Simple fix. http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

15. FED Medium Term Plans Production Plans Q1/2004 : Finalise design changes for FEDv2. Sign off against FED User Requirements Document. Done. Q2/2004 : Implement changes FEDv2 and review. Q3/2004 : Manufacture couple of FEDv2s. Q4/2004 : Test FEDv2. Parts for 30+ FEDs now in hand or on order. EU Tender Q1/2004 : Place OJEC advert, invite EoI. Q3/2004 : Dispatch calls for Tender. Q1/2005 : Award contract. http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

16. FED Schedule http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

17. FED Software Enhancements 1. Interface to Fed9UDescription and Fed9UVmeDevice have been brought in line, so now you can replace your description object with a FedDevice object and it will work seamlessly. 2. First Stable release version 1 was delayed for the reason of testing the interface change thoroughly. Release will now go ahead at the end of this week 3. Fed9USupervisor now integrates the LOG4C logging and XML description configuration which has been tested in the LAB at CERN. 4. I would like everyone to move to DAQ version 2 and xerces version 2 ASAP. Then we can discontinue support of Xerces Version 1 in the release of FED software version 2.0 5. The lib install has been tested for version 0.9 at cern in AFS, and works. So I am confident that we will not have to release source code to the main users. 6. Fed 9U Website now up and running. Contains release documentation of the high level interface to the FED software. Has links to the latest installers and to all previous install versions. Currently you have a separate installer for Xerces 1 and 2. one page will be the change log, which will be automatically updated every time we make a release. This is to be done this week to be available along with the first release. 7. Release script now uploads the latest release installer to the website, and will update the webpage contents automatically, so future management of the website content should be minimal. • See Online Software Meeting http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

18. Summary • Prototype FEDv1 • 6 more FEDv1’s made successfully . • Commissioning tests at (CERN, Imperial & RAL) proceeding well. • Baseline Hardware and Firmware design verified. • Investigating Temperatures with number of FEDs in crate. • Pre-production FEDv2 • Implement design changes in Q2/2004 • Manufacture minimum couple of boards in summer for test in Q4/2004. • EU Tender • Waiting for expressions of interest for 9U board production. http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

19. End Intentionally left blank http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

20. EU Tender Process EU Procurement Directives: Goods & Services > £150 K • Open Procedure : Any party can tender, suitable for off the shelf items (e.g. components) • Restricted Procedure : Only N selected parties are invited to tender. N to be specified in advance. Two part process. Market survey + Tender. • Publish OJEC Notice for Expressions of Interest. Can pre-notify selected companies. Associated Questionnaire. Rather general document. Elicit size, history of company, financial standing, standards adherence, facilities etc. • Evaluation and Selection of N companies to tender. Detailed enquiries. Visits. • Issue Tender. Detailed specification. • Still learning about these procedures. And how they fit with CERN (and CMS) procedures. • RAL is just about to issue a general pcb manufacture notice. Plan to use FED as example for tender exercise. • Some issues to decide, e.g. is separate tender for FPGAs necessary? http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

21. Restricted Procedure http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

22. Testing • At Assembly plant. • Do as much as possible. Discuss with companies. Positive responses. Q. Cost? • Boundary Scan. • Flying probe or ATE. • VME Crate single card tests. (without Opto inputs, using internal test features). • Major software effort to devise suitable tests with diagnostics. • Draft document for “Reduced Test Setup at Assembly” just released. • Full System Tests at RAL/Imperial • Full crate tests. • Interfaces to TTC. • Full optical soak system tests before shipping. • Installation and commissioning. • Ship FEDs to Prevessin staging point. • Repeat standalone soak tests? • Install boards individually at CMS. http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

23. Spares • Manufacture all in one go. • Need 440 for CMS Tracker. • 60 working spares (for lifetime of CMS). Guesstimate. • Assume final 500 delivered passing all commissioning tests. • Issues for later productions. • Component availability. Memories. • Expert availability. Design and Test. • Changes in pcb and assembly processes. Standards. Markets. • Pb Free Directive • Failure rates • Hard to determine yet. • Based on previous experiments assume normal failure rate is low after commissioning, bath tub distribution (catastrophic crate loss?). • Consider accelerated ageing tests offered by assembly companies? • Assume most failures can be repaired. • OptoRx can be removed. http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

24. Pb Free Directive • We know it’s coming… • WEEE July 2006. In principle a good thing. • Japan already claims Pb Free. • Military, Automotive exempt (but small markets). We may be exempt. • Contract manufacturers constrained to switch for mainstream production. • No drop in replacement to eutectic Sn/Pb solder. • Alternatives require higher soldering temperatures. • Manufacturing processes not fully understood. • BGAs solder type? • No long term experience of alternative solder e.g. failure in service. • May require changing pcb base material from FR4. • Would advise to make FEDs before industry switches… http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

25. First FED Prototype (Jan 2003) JTAG Boundary Scan OptoRx VME64x 9U board CFlash 34 x FPGAs ~40K-2M gates 96 channels Event Buffers Analogue Power TTC Deliver FED “Package”: Hardware Firmware Software Primary Side (Secondary side has 1/2 analogue) http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

26. First Fully Assembled Board Secondary Side Primary Side FE Unit 9U VME64x BGAs 676 pins @ 1 mm pitch PCB (2mm) 14 layers (incl 6 power & ground) Test with JTAG Boundary Scan ~ 6 K components (smallest 0402) ; ~ 25 K tracks 96 ADC channels : AD9218 Dual package 10 bit @ 40 MHz 1/2 Analogue circuitry on Secondary Side Highest density at Front-End Units http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

27. CMS Tracker FED Firmware Status 14th July 2003 System ACE EPROM VME FPGA Ed DAC Opto Rx EPROM Temp ADC System ACE DAC Opto Rx VME Bus Temp ADC Temp VME I2C Clocks Serial Controls Regs Clocks Serial Comms Regs Input Serial Controls VME LINK Data Data Header Mode Scope Mode Frame-Findng Mode Input Ed FIFOs DELAY FPGA x 3 x 8 Output BE FPGA Saeed Cluster Finding Mode Scope Mode Serial Comms Regs VME Link External Devices FE FPGA x 8 Ivan To be Implemented Control S-LINK S-LINK Clocks Headers Under Simulation Throttle TCS Input Under Test on FED QDR Write QDR Read TTC chanA TTCrx Data Readout Saeed, Ivan “Working” on FED Chan B QDR Controls Ed, John QDR Only for FEDv2 http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/

28. FEDv1 Firmware 15th March 2004 System ACE EPROM VME FPGA Ed->Saeed DAC Opto Rx EPROM Temp ADC System ACE DAC Opto Rx VME Bus Temp ADC VME Clocks I2C Clocks Serial Comms Regs Clocks Serial Comms Regs Data Input Serial Comms VME LINK Spy Data Header Mode Header Mode Cluster Mode Input Ed->Saeed FIFOs DELAY FPGA x 3 x 8 Scope Mode Output BE FPGA Saeed Scope Mode Serial Comms Regs VME Link External Devices FE FPGA x 8 Saeed To be Implemented Control S-LINK S-LINK Headers Under Simulation Throttle TCS Input Under Test on FED QDR Write QDR Read TTC chanA TTCrx Data Readout Saeed, Ivan “Working” on FED Chan B QDRs Controls Ed, John FEDv2 http://www.te.rl.ac.uk/esdg/cms-fed/qa_web/