1 / 39

Pulsar Status and Plan

Pulsar Status and Plan. Ted Liu, Oct. 13, 2003, RunIIb DAQ review. People&Institutions involved Project status: what’s been done & what’s left to be done Roadmap to the final new system Man power issues. More details can be found at:

nitara
Download Presentation

Pulsar Status and Plan

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pulsar Status and Plan Ted Liu, Oct. 13, 2003, RunIIb DAQ review • People&Institutions involved • Project status: what’s been done & what’s left to be done • Roadmap to the final new system • Man power issues More details can be found at: http://www-cdf.fnal.gov/internal/WebTalks/0310/031010_2bl2.html Project web page: http://hep.uchicago.edu/~thliu/projects/Pulsar/ CDF L2 upgrade Pulsar e-log: http://www-cdfonline.fnal.gov/cgi-bin/det-system-elog.pl?nb=l2upgrade

  2. People&Institutions involved in Pulsar project  from few people last year to now: • Related to L2 decision crate upgrade: • ANL • R. Blair, J. Dawson, B. Haberichter, J. Schlereth, J. Proudfoot • FNAL • R. Demaat, M. Hakala, R. Kivilahti, J. Lewis, C. Lin, T. Liu, T. Masikkala, • F. Marjamaa, J. Patrick, S. Pitkanen, B. Reisert, P.Wilson • UC • M. Bogdan, Y. Kim, W. Fedorko, H. Frisch, S. Kwang, V. Rusu, H. Sanders, • M. Shochet • Upenn • K. Hahn, P. Keener, J. Kroll, C. Neu, F. Stabenau, R. Van Berg, P. Wittich • Related to SVT upgrade (using Pulsar): • INFN • F. Spinella, L. Ristori • LBL • A. Cerri

  3. New people • Two new posdocs joined the project recently • Chris Neu (Upenn), and Vadim Rusu (UC) • Both are damn good, and they will make a big difference. • Y2K also has a new student (first year), Wojciech Fedorko • YoungKee’s group is joining the project…

  4. Project status &what’s left to be done? • Many aspects: • Hardware; • firmware; • VME standalone testing software; • DAQ code; • online monitoring code; • offline validation code; • PCI/CPU interface, L2 algorithm etc Much easier to discuss this with one “baseline” system configuration …

  5. Pulsar pre-processors New L2 Baseline Configuration L1 muon L1 XTRP L1 trigger SVT SLINK muon SLINK PC 0 SLINK/GBE PC 3 merger L2 CAL cluster PC 1 PC 2 merger TS RECES ShowMax L2toTS merger

  6. Pulsers (Tx) Pulsar pre-processors Hardware involved (with Txs) SLINK muon SLINK PC 0 SLINK/GBE PC 3 merger cluster PC 1 PC 2 merger TS RECES Pulsar mergers or GBE switch L2toTS merger

  7. Roadmap (as shown in Jan 30th, 2003) • Three phases/stages: • Phase A: Teststand, core firmware, HW prod/testing • Algorithm firmware specification • PCI/CPU/OS/infrastructure software • We are now entering phase B … • Phase B: Rx algorithm firmware implementation/testing, • system integration in standalone mode • Phase C: System integration in parasitic mode, test runs • with cosmic and beam

  8. Project Overview (as shown in Jan.30th, 2003) • So far has been divided into four “parallel” efforts: • E1: Pulsar hardware/firmware/VME software • E2: Design spec. for data format/algorithm/system interface • E3: RoIB option effort (a backup option) • E4: PCI/CPU architecture/OS/Infrastructure SF/L2D SF … E1 E2 E3 E4 new L2 Phase A Phase B Phase C

  9. What has been done so far this year The project has gone extremely well so far TONs of work has been done, due to the hard work by many … We have achieved most of the goals (much more in some areas) we set in Jan. 30th, 2003. Lots of work done by young students, This is definitely a “students friendly” project. Will summarize each effort first

  10. E1:What has been done so far • ALL mezzanine cards production&testing finished; • AUX card production/testing started; • Pulsar pre-production on-going; • All data paths have been initially tested in standalone mode; • VME standalone testing code automated for production testing; • fiber splitter has been tested in standalone mode; • initial RECES Rx firmware tested in standalone mode; • initial CLIST/Isolation Rx firmware testing on-going; • firmware for Pulsar  S32PCI64 (CPU) interface tested; • working closely with SVT folks for SVT II upgrade using Pulsar; • Muon/XTRP path fully established & tested with beam; • Young people who made significant contribution to E1 effort: • FNAL: Burkard/Sakari/Mikko/Tomi/Cheng-Ju/Fran, UC: Shawn • Muon: special thanks to Eric James, DAQ folks (esp. Jane Nachtman), • operation managers and friendly shift crews …

  11. E2:What has been done so far • Pulsar Data format spec for all data paths in place; • worked closely with DAQ folks for muon path; • offline software for muon/XTRP path working (five banks); • emulation software for muon/XTRP internal algorithm working; • online monitoring software for XTRP/muon path working; • Alpha code to separate XTRP/muon data working with beam; • Alpha code for L2 muon algorithm on-going; • extensive offline data analysis for Pulsar muon/XTRP beam data; • worked very closely&effectively with E1, esp. for commissioning • the L2 muon/XTRP path with the beam; Young people who made significant contribution for E2 effort: FNAL:Cheng-Ju/Frans/Burkard, UC: Shawn

  12. E4:What has been done so far • Algorithm timing performance done (alpha vs modern CPUs); • S32PCI64 Tx as back channel established; • Initial testing software in place; • Pulsar  S32PCI64CPU  S32PCI64Pulsar test done; • Timing (w/o algorithm running) measured with Pulsar  CPU • Gigabit Ethernet option testing on-going… people who made significant contribution for E4 effort: Upenn:Peter/Paul/Kristian/Fritz, ANL: Bob

  13. Pulsar is designed to be fully self-testable: at board level as well as system level For every input/output, there can be an output/input, All interfaces are bi-directional (except VME) TSI SVT/XTRP one for all and all for one SRAMs Gigabit Ethernet RF clock It is this feature which allows us to develop&tune an upgrade system in standalone mode Overall planning is based on this feature

  14. One slide from my talk on Jan 30th, 2003 Guideline for the project planning: Project planning follows Pulsar design philosophy/methodlogy (1) In God we trust, everything else we test (2) One for all and all for one Anyone who has a problem with this is in the wrong project Success in muon path: Pulsar methodology at work 

  15. June 2003: Jimmy ordered us to “jump”, we asked “how high?” Pulsar RunIIA Muon path: From concept to error free path takes ~ 3 months Jimmy’s office: discussion in June about L2 Pulsar muon Path for RunIIA

  16. Muon path commissioning: to kludge up something is easy, to have something robust & error free is not • The 2rd day we came up with a detailed plan for the • first 6 weeks, since then: • each week the goals were met for all 6 weeks; • after that, another 6 weeks to FULLY debug the full path; • at the same time, we took care of taxi mezzanine cards • production/testing; initial Pulsar pre-production testing, • training three new Finnish students… • during that period (three months time span), • Sakari/Burkard/Cheng-Ju each took ~ 2 weeks • vacation (at different time) • the team worked together in a highly coherent way! • details see L2 upgrade Pulsar e-log • This gives you some idea how much it takes to fully • commission one data path. The experience gained • is extremely useful for the rest of data paths…

  17. Muon path commissioning experience as a guide Output Options Pulsar Muon Board Input Options L2 TrkList Board in b0l2de00 L1 Muon (16 hotlink fibers) and/or L1 XTRP Cable OR Pulsar Receiver OR Pulsar muon+xtrp data transmitter • hardware & firmware • VME standalone test software • DAQ code • online monitoring (XFLD/XTRD/TL2D/TCMD/TP2D) • offline validation (XFLD/XTRD/TL2D/TCMD/TP2D) • commissioning: teststand,cosmic/L2 torture/beam We didn’t waste one second of beam time(except the reformatter errors), The full chain test with beam works on the first try (error free).

  18. Test Runs Summary STORE 2985        2003/09/03 21:49  2003/09/04 14:51 Run#         L3A  Lumi_live  Max. rate     Avg. rate     Inst.Lumi (E30)(nb-1)  L1 / L2 / L3   L1 / L2 / L3   start/end168766    335,930    196.220  15.0k/268/58   13.3k/255/53   35.0/30.2168767    155,974    85.949  13.7k/240/53   13.0k/230/50  29.5/28.0168775    596,981   258.197  12.5k/190/45   10.6k/169/39  19.7/15.8STORE 2988        2003/09/04 21:26  2003/09/05 10:21Run#         L3A  Lumi_live  Max. rate       Avg. rate       Inst.Lumi (E30)(nb-1)  L1 / L2 / L3   L1 / L2 / L3   start/end168819     90,076    76.570  16.5k/148/30    13.8k/132/27   24.7/23.1168820  1,101,959   471.469  14.7k/235/50   12.0k/193/47  22.7/14.8168821    166,146   114.954   9.6k/ 94/22    8.6k/ 86/20  14.5/13.2168822    121,685    50.591   8.4k/134/34    7.3k/121/30  13.2/12.6STORE 2988        2003/09/04 21:26  2003/09/05 10:21Run#         L3A  Lumi_live  Max. rate       Avg. rate       Inst.Lumi (E30)(nb-1)  L1 / L2 / L3   L1 / L2 / L3   start/end168889    2,816,155    1,359.049  15.5k/300/60    12.2k/205/45   40.0/13.7 ALL data collected for Pulsar beam test are good for physics analysis. No single Pulsar hardware/firmware problem was seen (~ 5M evts.)

  19. Muon path commissioning experience as a guide • of course, “doing it the right way” doesn’t mean “success” @ CDF • I was in the mountains when I heard the rumor: “Pulsar is it”… • you get much more publicity if you do it the wrong way… Jade Dragon Mountain, Yunnan, China, Sept. 24, 03

  20. Hardware status (no revision to any board) Hotlink Tx/Rx Prod./testing done Taxi Tx/Rx Prod./testing done MOAB (Motherof All Boards) Pulsar Expect two pre-production boards arrive this week … eight more in the same batch (next month)… Expect production to start next month ANL SLINK->GBE (production) SLINK LSC/LDC (have some, need to order more) AUX Card Six S32PCI64 in hand production started two weeks ago

  21. Pulsers (Tx) Pulsar pre-processors All Tx firmware in place&tested Fine tuning on-going… SLINK muon SLINK PC 0 SLINK/GBE PC 3 merger cluster PC 1 PC 2 merger TS RECES L2toTS merger Firmware status

  22. Pulsers (Tx) Pulsar pre-processors Muon/XTRP path firmware in place&tested with beam… Need SLINK output formating SLINK muon SLINK PC 0 SLINK/GBE PC 3 merger cluster PC 1 PC 2 merger TS RECES L2toTS merger Firmware status

  23. Pulsers (Tx) Pulsar pre-processors Clist input DAQ firmware written, testing on-going. Iso input firmware written, simulation on-going… Soon ready to test with real system. SLINK muon SLINK PC 0 SLINK/GBE PC 3 merger cluster PC 1 PC 2 merger TS RECES L2toTS merger Firmware status

  24. Pulsers (Tx) Pulsar pre-processors Reces input DAQ firmware written, tested ok in standalone mode, ready to test with real RECES inputs… SLINK muon SLINK PC 0 SLINK/GBE PC 3 merger cluster PC 1 PC 2 merger TS RECES L2toTS merger Firmware status

  25. Pulsers (Tx) Pulsar pre-processors Pulsar  S32PCI64/CPU interface firmware in place&tested, timing measured with both LA and Pulsar internal counters (via VME)… SLINK muon SLINK PC 0 SLINK/GBE PC 3 merger cluster PC 1 PC 2 merger TS RECES L2toTS merger Firmware status

  26. Pulsers (Tx) Pulsar pre-processors Initial SLINK merger firmware existed long ago. New version is straightforward. (to be done) Simple firmware for L2toTS, to ensure L2A in L1A FIFO order (to be done) SLINK muon SLINK PC 0 SLINK/GBE PC 3 merger cluster PC 1 PC 2 merger TS RECES L2toTS merger Firmware status

  27. Pulsers (Tx) Pulsar pre-processors Online code for the Muon/XTRP path (with five banks) running in TrigMon. Other paths need to be written… SLINK muon SLINK PC 0 SLINK/GBE PC 3 merger cluster PC 1 PC 2 merger TS RECES L2toTS merger Online monitor code status

  28. Pulsers (Tx) Pulsar pre-processors Offline code written and used extensively for the Muon/XTRP path (with five banks) & to convert real data into test patterns. Other paths need to be written… SLINK muon SLINK PC 0 SLINK/GBE PC 3 merger cluster PC 1 PC 2 merger TS RECES L2toTS merger Offline validation code status

  29. Pulsers (Tx) Pulsar pre-processors Have tested CPU performance (alpha vs modern CPUs). Initial software for S32PCI64 (Tx/Rx) in place & tested with Pulsar, timing measured. SLINK muon SLINK PC 0 SLINK/GBE PC 3 merger cluster PC 1 PC 2 merger TS RECES L2toTS merger PCI/CPU status

  30. Round-Trip timing:

  31. Round-Trip timing measured & the performance looks good w/ algo : Blue w/o algos : Red • PULSAR ↔ PC  Round-trip timing with realistic hardware setup • Zero suppressed, • S-Link formatted data • Pulsar Firmware support for • timing & L2 decision verification • Pulsar On-board timing counter

  32. Still lots of work ahead of us: devil is in the details path firmware Spec/test DAQ monitor comment RunIIa muon mostly done (Shawn: alpha code) Sakari Tomi Burkard Cheng-Ju Cheng-Ju Frans Muon/ XTRP Jane student & help from BR/CJL Sakari Tomi Jane/ Vadim Vadim Cluster Vadim Jane/ Chris Sakari Tomi student & help from BR/CJL Reces Chris Chris Merger/ SVT Sakari Tomi Burkard Cheng-Ju Cheng-Ju Frans + Mikko Jane Sakari Tomi Burkard Cheng-Ju Cheng-Ju Frans + Mikko L2toTS Jane PCI/ CPU + Risto & others Peter/Bob/Paul/Kristian/Fritz

  33. Comments on firmware effort • This is a big project from firmware point of view, • well organized effort is the key to success; • so far, we haven’t got engineer support from lab on firmware, • we have had to train our own engineers! • every young person involved in this project has been trained • with firmware tools (Peter/Sakari/Burkard/Cheng-Ju/Tomi/ • Mikko/Frans/Shawn/Kristian/Fritz …). • Sakari & now Tomi are the dedicated (100% of time) people • writing firmware, they have done a wonderful job. • Firmware implementation isn’t for everyone, very time consuming. • Typically takes ~ half year FULL time to learn before one really • knows how to write good firmware (more difficult than C++). • physicists are strongly encouraged to get involved to • understand the firmware and learn how to do simple modifications; • Pat Lukens will find a lab engineer for long term support

  34. Well organized firmware effort is one of the keys to have a system: clean/easy-to-understand/robust & with minimal maintenance effort Common core firmware muon VME responses SLINK formatter DAQ buffers CDFCtrl responses SRAM ctrl Mezzanine interfaces Diagnostic RAMs Spy buffers … SVT XTRP LVDS + ext. FIFO cluster hotlink L1 LVDS Taxi Isolation TSI Reces Knowing one data path Pulsar  knows all Pulsars Data specific algorithm difference is just minor details have dedicated people responsible for developing the core firmware

  35. Coordinate the efforts • Hardware/VME test software (coordinator: Burkard) • main tasks: train new people/test all new Pulsars/system test software • PCI/CPU/L2 algorithm (coordinator: Peter) • main tasks: production version software/simulation/train new people • Data format/spec/monitor (coordinator: Cheng-Ju) • main tasks: algorithm spec/monitoring for all paths/train new people • DAQ liason (coordinator: Chris or Vadim?) • main tasks: DAQ spec/code for all paths/train new people • Firmware (coordinator: TL, will find a replacement later) • main tasks: firmware spec for all cases/train new people

  36. Roadmap to final system • plan to have each path fully tested next spring. • this should be possible in teststand mode, and with • cosmic/L2 torture with fiber splitters. We will need to • request for beam time (in parasitic mode only). • to make this easier, the RunIIa Pulsar muon board needs • to be moved into L2 decision crate at some point. • system performance simulation to guide system configuration. • all SLINK hardware need to be purchased by next spring. • system integration next summer, in teststand mode • first, then with the system and with fiber splitters. • will need beam time testing before next summer shutdown. • will discuss the details of integration early next year.

  37. System performance simulation study need help from Yale • We need to perform the simulation study soon, to guide system • configuration,to find out how much performance gain we can • expect with the new L2 system. For example: • gain from modern technology (speed in FPGA/CPU/PCI device); • gain from architecture flexibilities: • (1) having four CPUs each dedicated for a given buffer event, • with L2A issued in L1A FIFO order; events do not • have to “wait” in the pipeline while previous event(s) • is being loaded or processed; • (2) CPU doesn’t need to wait for SVT data before processing • a given event; • (3) gain from SVT II/Silicon DAQ performance improvements; • (4) more …

  38. To make sure we can deliver a solidnew systemnext summer,we will need: (1) Fermilab hire Sakari as visiting engineer for one year. (2) Extend the stay of the three Finnish students for 6 months. We have invested a lot in them &they have done wonderfully well Tomi (firmware), Frans(Pulsar emulation, online/offline code, data integrity checks), Mikko(hardware testing, VME software, Pulsar GUI). They haven’t learned how to BS, & will not on this project. They are young & work only for food&beer ($1500 per month per person). (3) New posdocs have to spend significant amount (if not full) time on the project in the coming year (Chris & Vadim)  Without any of the three above, next summer is not possible. To make sure the new system is maintainable, Fermilab has to provide an engineer for long term firmware/hardware support. Each group needs to provide new student(s) and posdoc(s) sometime next year.

  39. From the last slide of my talk on Jan. 30th, 2003 RunIIA and RunIIB 2B or not 2B? This is no longer the question. The only people who could stop, slow down, screw up or speed up this project is us. Whatever we do, we have to do our very best The only way we can deliver a solid new L2 system next summer is to work hard && work as a coherent team… There is no need or motivation to bring up an unreliable system

More Related