1 / 31

Project Overview

Project Overview. Ted Liu Fermilab Sept. 27 th , 2004 L2 Pulsar upgrade IRR Review. Materials for the review can be found at:. http://hep.uchicago.edu/~thliu/projects/Pulsar/L2_upgrade_meeting.html. Q&A. Are we ready for operation? No Why are we having this review then?

beryl
Download Presentation

Project Overview

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Project Overview Ted Liu Fermilab Sept. 27th, 2004 L2 Pulsar upgrade IRR Review Materials for the review can be found at: http://hep.uchicago.edu/~thliu/projects/Pulsar/L2_upgrade_meeting.html

  2. Q&A • Are we ready for operation? • No • Why are we having this review then? • To help us getting ready for final operation. To review • what we have achieved so far • our plans for final commissioning • is there anything missing on our to-do-list? • is manpower enough? • Is the schedule realistic? • … • Will have another review early next year

  3. Qutline • Baseline design for operation and overall status • Project Organization: who is working on what • General testing&commissioning methodology • Firmware Overview and necessary improvements • Firmware Version Control • Comments on possible future system optimization • Manpower issues • Summary

  4. Baseline for operation • Pulsar-based system is very flexible • Should only focus on the baseline for operation • single algorithm node • deliver non-SVT and SVT data on separate PCI path • simple algorithm in pre-processor Pulsars • simple data merging • use S32PCI64 cards instead of FILAR … • Will comment on some system flexibility later • further possible improvements/optimization depend on initial operation experience, not required for operation

  5. L1 muon L1 XTRP L1 trigger Muon Merger TS PC L2toTS Cluster SLINK L2 CAL (CLIST/Iso) PreFred SVT SVT Electron ShowMax (RECES) Merger Baseline for operation Pulsar pre-processors • Commissioning strategy: • Use Pulsar Tx  Rx in teststand • Split all input signals to Run IIa system, run parasitically with detector and beam

  6. Main tasks for the project • Hardware: custom: Pulsar,mezzanine&AUX, LVDS & fiber active splitter commercial: fiber passive splitter, SLINK mezzanine, SLINK-PCI cards, Linux processors • Firmware: transmitters and receivers for all data paths SLINK merger and L2 to trigger supervisor interfaces  about 14 different types of Pulsar firmware • Software: board testing, VME DAQ readout, online/offline monitoring, software for algorithm&control node, interface with rest of world …

  7. Overall Status • Hardware: All custom hardware in hand, and all fully tested except fiber active splitter (see Chris’s talk) need to order two Algorithm PCs soon, one in hand • Firmware: All firmware in place and tested in beam…some needs fine tuning for either performance optimization or robustness  more on firmware later • Software: main work left is related to algorithm and control node see later talks for details

  8. Who is working on what (I) • Muon/XTRP/L1 + SVT + Merger + RECES + L2TS • Tx Firmware: Tomi (work done, left) • Rx Firmware: Sakari • Testing: Burkard, Frans (left), Cheng-Ju (L2TS) + Sakari • Commissioning: Burkard + Cheng-Ju • Monitoring for Muon/XTRP/L1/SVT/Merger: Shawn • Monitoring for L2TS: Cheng-Ju • Monitoring for RECES: Chris with Daniel’s help earlier • Active Splitter for RECES: Chris • LVDS Splitter Board: Wojtek (only the main soldiers are shown here)

  9. Who is working on what (II) • Cluster/PreFred • Tx Firmware: Tomi (work done, left) • Rx Firmware: DataIO FPGA: Vadim Control FPGA: Sakari (similar to other cases) • Testing/commissioning/monitoring: Vadim and Wojtek • Algorithm node software: Kristian • Control node software: Daniel (only the main soldiers are shown here)

  10. Who is working on what (III) • General • Pulsar Hardware: Burkard • VME readout software: Jane from DAQ + Burkard’s help • PulsarMon design: Vadim • RunCTRL/TriggerDB etc interfaces: Daniel • Firmware coordination: TL • Commissioning coordination: in the past: TL in the future: Trigger SPLs (See Cheng-Ju’s talk on plan and schedule)

  11. Testing and Commissioning Methodology • Pulsar is self-testable, at both board and system level • This design feature is the key to success • Pulsar Hardware testing all done with self-testing capability • Pulsar firmware testing mostly done with self-testing capability • Software development all done with Pulsar self-testing capability • Even at system commissioning, a few Tx boards are used • This has made the commissioning work much easier • System commissioning relies heavily on input signal splitting • Saved us LOTs of dedicated beam time • So far, only used a few hours dedicated beam time • Input signal splitting has been and will be extremely helpful • Will keep input signals split even after final commissioning • to have a “copy” of the new system running parasitically  for hot spares, for firmware/software improvements

  12. Pulsar Firmware: BIG effort • Pulsar has three main FPGAs • L2 decision upgrade:14 different type Pulsars (Tx/Rx) • Key to success: • Design: common re-usable modular design • Simulation: no simulation  no testing • Dedicated testing support • Version Control • Careful firmware update procedures …

  13. people 14 Pulsar types/firmware for L2 decision upgrade 081          L2 Pulsar Muon/XTRP Rx IIa083          L2 Pulsar SVT Road Warrior085          L2 Pulsar Muon/XTRP/L1 Tx or SVT XTRP-emu086          L2 Pulsar Muon/XTRP/L1 Rx IIb087          L2 Pulsar RECES Tx088          L2 Pulsar RECES Rx089          L2 Pulsar Cluster/PreFred Tx090          L2 Pulsar Cluster/PreFred Rx091          L2 Pulsar SVT Tx092          L2 Pulsar SVT Rx093          L2 Pulsar Merger Tx094          L2 Pulsar Merger Rx095          L2 Pulsar L2TS “Tx” (reserved spare ID) 096          L2 Pulsar L2TS097          L2 Pulsar L1 Scaler098          L2 Pulsar SVT Track Fitter 099          L2 Pulsar test Tx100          L2 Pulsar test Rx101          L2 Pulsar XFT Stereo Tx102          L2 Pulsar XFT Stereo Rx 103 L2 Pulsar SVT Hit Buffer 104 L2 Pulsar SVT AMSRW Board ID Sakari Tomi Sakari Tomi Sakari Tomi Vadim/Sakari Tomi Sakari Tomi Sakari Sakari Sakari Sakari One Hardware -- many applications other types for SVT/XFT or future improvement

  14. FPGAs and data interfaces Interfaces (Tx/Rx): Muon Reces Cluster Iso Slink Interfaces (Tx/Rx): SVT/XTRP/L1/PreFred TS & SLINK (P3)

  15. General simplified data flow CDF CTRL Mezz interface DataIO algo slink formatter DAQ buffers VME response DAQ buffers CDF CTRL slink formatter merge Same as above DataIO DAQ buffers VME response DAQ buffers XTRP/SVT

  16. Well organized firmware effort is the key:clean/easy-to-understand/robust with minimal maintenance effort DataIO or Control FPGA CDF CTRL Input interface Slink formatter algo Output FIFO DAQ buffers DAQ buffers VME response • all common parts are shared by all firmware • also share many low level routines (latency timer, bunch • counter etc)

  17. Common core firmware (library) muon VME responses SLINK formatter DAQ buffers CDFCtrl responses Mezzanine interfaces Latency timer Bunch counter … SVT track LVDS + ext. FIFO cluster hotlink L1 LVDS Taxi Iso TS showermax Knowing one data path Pulsar  knows all Pulsars Data specific algorithm difference is just minor details core firmware developed by people dedicated to the task

  18. SLINK data format Keep it simple 8 4 2 8 8 2 was defined as data size before, which means one has to wait until all data arrives before sending out SLINK package. will remove it, and use it as latency stamp instead. Format version Data source Region ID Reserved Bunch count Buffer # Latency (previous) Latency (current) 16 16 Data elements 16 16 Data size Error flags

  19. Four Slink mezz cards AUX card Slink output Slink inputs SLINK Merger  fragments into event package Slink merger output 0xE0F00000 Merger trailer End of fragment Input 4 trailer Slink input 4 data Input 4 header 2 Input 4 header 1 End of fragment Input 3 trailer Slink input 3 data Input 3 header 2 Input 3 header 1 End of fragment Input 2 trailer Slink input 2 data Input 2 header 2 Input 2 header 1 End of fragment Input 1 trailer Slink input 1 data Input 1 header 2 Input 1 header 1 Merger header 2 Merger header 1

  20. 32-bit SLINK to 64 bit PCI interface card: S32PCI64 • highly autonomous data reception • 32-bit SLINK, 64-bit PCI bus • 33MHz and 66 MHz PCI clock speed data Slink to PCI mem data Slink to PCI L2 decision CPU PCI to Slink

  21. General comments on Firmware • Firmware is not hard if it just has to be “sort of working” • The hard part is to get it “rock solid” • Firmware is not a place to show how smart one is • rather to show how many mistakes one could make • Firmware testing is as important as writing/simulating • Firmware testing is often much more time consuming than the actual writing • How good the firmware depends on how well it is tested • Our general strategy: firmware developer supported by people dedicated to testing

  22. Files kept • VHDL source code • Files necessary for recreating this version • Readme with revision history • FPGA configuration files the FPGAs • Source code (working version) under CVS control • Firmware archive updated on different PCs

  23. How to keep Firmware in order • We have so many different types of firmware • Keeping them in order is non-trivial: • under CVS control: source VHDL code, pin files etc • Firmware version ID VME register per FPGA (has to update by hand) • README file update • Follow careful firmware update procedures • There has been no confusion about firmware versions when the procedure has been followed

  24. te

  25. slot 05: IDPROM: 0016 086 PULSAR MUON RX CTRL: 04408060,DataIO1: 08407020, DataIO2: 08407020 slot 13: IDPROM: 0004 092 PULSAR SVT RX CTRL: 05408060, DataIO1: 09407280, DataIO2: 09407280 slot 15: IDPROM: 0028 093 PULSAR SLINK TX CTRL: a0406300, DataIO1: 08407230, DataIO2: 08407230 slot 17: IDPROM: 0061 096 PULSAR TS INTERFACE CTRL: 0a407280, DataIO1: 09408030, DataIO2: 09406040 slot 19: IDPROM: 0012 094 PULSAR SLINK MERGER CTRL: 02408080, DataIO1: 01407220, DataIO2: 01407220 slot 21: IDPROM: 0014 094 PULSAR SLINK MERGER CTRL: 02408080, DataIO1: 01407220, DataIO2: 01407220 Example of Pulsar crate ID&version scan by Burkard on Aug 19th after Beam test: Not all boards are shown due to limited space Allow us to go back to the same configuration after shutdown How can we check these against database when in operation mode?

  26. Necessary firmware optimizationbefore operation • Reces: zero-suppression + latency optimization • Reces data comes into Pulsar 8 us after L1A • current firmware is not optimized • needed to reduce Reces path latency by a few us • Status: in progress, estimated time: ~ one month • Slink formatting: sending data words out upon arrival (not waiting for all data to arrive) • improving Slink merging latency (~ a few us) • estimated time: ~ one month (including testing) • More timing measurements • add latency timer in a few other places (relatively easy) • Cluster path improvement (see Vadim’s talk)

  27. Future possible system optimization after operation • Use FILAR cards instead of S32PCI64 • Remove the final merger, data directly goes into FILAR • Possibly also remove Reces merger with another FILAR • Also allow us to run SLINK faster: 40 MHz  ~60 MHz • Should reduce the latency by ~ few us • Need PCI software modification (no firmware change) • Move L1 scaler into firmware if needed • Move to 4 PCs instead of single PC (how much do we gain?) • Additional firmware (L2TS) work and some software work • Muon and track matching in firmware • Bring upgrade XFT stereo info into L2 (this will happen) • …much more, depends on what we learn&need by then

  28. FILAR can improve the system latency  Higher bandwidth, less latency overhead 64-bit/66MHz PCI bus, four 2 Gbit/s S-LINK channels Reduced PCI protocol overhead (compare to S32PCI64)

  29. System with FILARs Pulsar pre-processors L1 muon L1 XTRP L1 trigger Muon Merger TS PC L2toTS Cluster SLINK L2 CAL (CLIST/Iso) PreFred SVT Electron Send all data paths directly into 2 FILARs… ShowMax (RECES) Merger SVT

  30. Manpower Issues • Why should we worry? • Tomi and Frans left (two key people on firmware/testing) • Only10% Burkard’s time available from now on (key person on hardware/VME software/firmware testing/system commissioning  need replacement SOON) • Kristian needs to graduate … (key person on algorithm/PCI software  need replacement SOON …) • Need new L2 software coordinator (Daniel?) … • anyone else will be full time on the project? • This is very serious …need this committee to help!

  31. Summary • Review should focus on baseline for operation • Keep in mind that system is flexible • Hardware is in good shape (still need to fully test the active splitters) • Firmware is in reasonable shape, needs fine tuning/testing in some cases  key is to follow our testing methodology • Software needs more work, no show stoppers  clear spec is the key • Manpower issues serious (in a few areas): need new people! • Details are in the rest of talks …

More Related