1 / 18

Ongoing Developments for the ATLAS LAr Subdetector Readout

Ongoing Developments for the ATLAS LAr Subdetector Readout. Possible replacement of VME for the upgrades of ATLAS 12 July 2012 Guy Perrot on behalf of the LAr upgrade collaboration. Outline. Context US Developments (BNL/Stony Brook & U. of Arizona) ATCA Development Blade

evan
Download Presentation

Ongoing Developments for the ATLAS LAr Subdetector Readout

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Ongoing Developments for the ATLAS LArSubdetector Readout Possible replacement of VME for the upgrades of ATLAS 12 July 2012 Guy Perrot on behalf of the LAr upgrade collaboration

  2. Outline • Context • US Developments (BNL/Stony Brook & U. of Arizona) • ATCA Development Blade • AMC ROD Injector • European Developments (Dresden & LAPP) • 10 GbE on Fabric interface • IPM Controller & ATCA Test Blade • ROD Evaluator • LAr Today’s Path for 2018 • Answers to your questions • Summary Possible VME Replacement for the ATLAS upgrades G.Perrot

  3. Current & Future Architectures Phase I 2018 Digital Trigger 320 scells/LTDB ~120 LTDBs ~ 30 ATCA DPS Blades ~24 Tbps IN ~24 Tbps OUT Up to 1.5 Tbps Monitoring Phase II 2021 FEB & ROD Change 128 cells/FEB ~1600 FEBs ~100 ATCA ROD blades ? 150 Tbps IN 0.4 Tbps OUT Current system 1600 FEBs 192 RODs ~120 TBBs Possible VME Replacement for the ATLAS upgrades G.Perrot

  4. US developments (1) Zone 3 (to transition module) LVL1 communication • BNL ATCA Development Blade • No IPM Controller (was supposed to be DESY Firmware but lack of time) • Shortcuts to get power on. • Room for 2 Mezzanines (not AMC) • Used by Dresden to test 10 GbE on Fabric Update ChannelGlobal Clocks 12 fiber inputs SNAP12 format DDR2SODIMM Memory Virtex 5 FXT FPGA Base Interface (Gigabit Ethernet)Fabric Interface (10Gig Ethernet) Serial Debug Port -48v Power Input and IPMI interface Power Conversion Possible VME Replacement for the ATLAS upgrades G.Perrot

  5. US developments (2) • BNL DPS/ROD sPU AMC • In design phase • DESY/CPPM/CERM MMC (ATMEL Processor) • Virtex7 FPGA • 48 10Gbps links • U. of Arizona ROD Injector AMC • 2 x 12 channels @ 6.25 Gbps (Front) • Stratix EP4SGX230KF40 • 12 channels @ 5 Gbps (GbE or PCIe) on Fabric • MMC implemented using an ARM Processor • Future AMCs to use CPPM/CERN MMC • AMC only used in uTCA Shelf Possible VME Replacement for the ATLAS upgrades G.Perrot

  6. European developments (TUD) • TechnischeUniversitat Dresden: 10 GbE through ATCA Fabric Interface • ATCA shelf: 20port 10GbE Switch, BNL SubROD (Virtex 5 FPGA) • Server: Dual 4-Core Xeon, 4x Dual Port Myricom 10GbE NIC • Link: optical between shelf and server • Results: • 80 byte UDP packets: ~400Mbps only! with packet losses • 1500 byte packets: ~4.4Gbps with packet losses • 9000 byte jumbo packets: ~10Gbps without packet loss Possible VME Replacement for the ATLAS upgrades G.Perrot

  7. European Developments (LAPP) ATCA Controller Mezzanine FMC Mezzanine • IPM Controller • Communication with Shelf manager through IPMBus A & B • Hot Swap, Power management etc.. • ATCA blade manager • Communication via Ethernet (front panel or ATCA Base Interface) • User functions • Firmware Upgrade • ATCA blade monitoring & configuration • User stuff…. • Software libraries available • FMC (FPGA Mezzanine Card) compliant • up to 160 customizable links( including 74 differential links) • Low cost • Features: • ARM cortex M3 processor • Xilinx Spartan 6 for highly configurable user IO • Ethernet / USB / JTAG interfaces connector Power Supplies Ethernet IO µCLM3S9B92 FPGASpartan 6 bus USB JTAG JTAG JTAG I2C IPMBus A&B EEPROM µC FPGA IO 69mm 76.5mm Possible VME Replacement for the ATLAS upgrades G.Perrot

  8. European Developments (LAPP)ATCA Test Blade • ATCA Controller Mezzanine tests • IO connections (JTAG boundary scan tests) • IPMI management with the Shelf manager • Ethernet communication through ATCA Base Interface • ROD Evaluator tests • Check Blade configuration with the ATCA Controller Mezzanine (Firmware upgrade, Configuration upload etc..) • Test ATCA compliant power supplies • Check FPGA Design (communications with DDR3, Flash, configuration with Flash) Possible VME Replacement for the ATLAS upgrades G.Perrot

  9. European Developments (LAPP)ATCA Controller Mezzanine Version 2 • Smaller size: DDR3 VLP Mini-DIMM • Based on 2 µC ARM cortex M4 from ST Microelectronics • IPMC features • IPMB_0 with on board buffers, Hardware address detection • Hot Swap management with ATCA Leds and front panel switch • Management of up to 8 AMC + RTM • On board Event LOG • FRU & SDR via I2C • Access to ATCA board sensors via I2C • Configurable User Signals for Payload management • JTAG Master • JTAG master via Ethernet (ATCA board debugging, firmware upgrade) • Custom interface • Possibility to have a custom interface between the Mezzanine and the ATCA board for custom functionalities • Other • USB and UART interfaces (debugging etc..). 18.30 mm 82.127 mm Possible VME Replacement for the ATLAS upgrades G.Perrot

  10. European Developments (LAPP)ATCA Test Board Version 2 • ATCA Blade with 4 AMC slots. • MMC mezzanine • Designed to test all the functionalities of the ATCA controller mezzanine. • Could be used as “Reference Design” for users. AMC 1 PWR Mgt MMC AMC 4 PWR Mgt Base Interface switch ETh RJ45 Mezzanine I2C HA Etc.. Led, switch User IO RSVD FRU T° FPGA DC/DC Power Mgt Possible VME Replacement for the ATLAS upgrades G.Perrot

  11. European Developments (LAPP)ROD Evaluator SNAP12 Rx (Carrier) • Fully compliant ATCA blade. • To evaluate high speed and high density optical & electrical transmissions. • To evaluate high power DSP cells from FPGA • Board management trough Ethernet (Firmware upgrade, DSP configuration…) - 48 optical Rx links @8.5Gbit/s • 48 optical Tx links @8.5Gbit/s • 8 Electrical Rx/Tx links @8.5Gbit/s 16 Electrical Rx/Tx LVDS links @ 1.6 Gbit/s between FPGAs • Connection to Fabric & Base interface DDR3 FPGA FLASH CPLD DC/DC : 0.9V SNAP12 Tx (Carrier) DC/DC : 3.3V,2.5V,1.8V,1.5V ATCA IPMC (FMC carrier) ATCA DC/DC Possible VME Replacement for the ATLAS upgrades G.Perrot

  12. Phase I Architecture Phase I 2018 Digital Trigger 320 scells/LTDB ~120 LTDBs ~ 30 ATCA DPS Blades (LDPB) ~24 Tbps IN ~24 Tbps OUT Up to 1.5 Tbps Monitoring Possible VME Replacement for the ATLAS upgrades G.Perrot

  13. LDPB Data Path Block Diagram 820 Gbps 24.2 Tbps Data Processing DATA 40 MHz RESULTS 40 MHz L1 Calo FEX FELTDB 205 Gbps 49 Gbps (8 GbpsRes. only) 820 Gbps 1.45 Tbps (240 Gbps) 24.2 Tbps Memory DATA + RESULTS 400 KHz Monitoring Data Collector RESULTS 100 KHz DATA Data Processing Memory Monitoring Data Collector L0 400 KHz Latency 2.5-10 us 2 Gbps 60 Gbps LTDB (LAr Trigger Digitizer Board) : 118 Modules LDPB (LAr Digital Processing Blade) : 29 ½ Blades (4 LTDB / LDPB) FE (Front End) & FEX (Feature Extractor) Monitoring Data Collector L1 100 KHz Latency 100s of us Possible VME Replacement for the ATLAS upgrades G.Perrot

  14. Back-End Block Diagram (In Work) Switch GbE Shelf Shelf Manager Blade: Switch Blade:LDPB Scells Raw data LTDB Switch GbE PM PC AMC Data Processing & Memory Raw data?, result on L0 Switch 10 GbE TTC, Control, Monitoring Raw data?, result on L1 Control, Monitoring Switch GbE Results to FEX L1 Calo TTC FEX Switch 10 GbE Monitoring Data Collector AMC Data Processing & Memory Blade Ctrl. & Monitor. IPMC TTC TTC LDPS Partition (NEW!) Busy TTC Distribution Possible VME Replacement for the ATLAS upgrades G.Perrot

  15. Answers (1) • What is the expected usage of the backplane? • ATCA IPM control of blades. • Blade & FE LTDB configuration & monitoring via base interface. • Data collection via fabric interface. • Do you use the fat pipes. If yes which protocol? • For the time being we are thinking 10 GbE. • Would you agree for a control through Ethernet? • Yes, we plan it that way! • Do you use synchronisation signals and common clocks? • No common clock for operation of blades. • TTC for clock transmission to FE and Evt signature at FE & BE. • Do you plan to use redundancy and/or hot swapping • Hot swapping YES, redundancy NO. • Do you plan full mesh or dual star backplane? • Full mesh, though we are using a dual star architecture, as it can do both. Possible VME Replacement for the ATLAS upgrades G.Perrot

  16. Answers (2) • Do you need a local processor? • For the time being we don’t think so. • Which shelf manager do you plan to use? • The one that will come with the shelf (we have Pigeon Point). • Which IPMC do you plan to use? • We are developing an IPMC and are planning to use it. • Typical expected power consumption? • No idea yet, but probably quite a lot: 200 W/blade => 2.8 KW ??? • Which crate size to you plan to use (i.e. how many slots available)? • ATCA format with 14 slots + shelf manager. • Do you plan implementation with AMC mezzanine and a mother board? • We are planning to have 4 AMCs on an ATCA carrier blade. Possible VME Replacement for the ATLAS upgrades G.Perrot

  17. Answers (3) • How sure are you that you need ATCA. Could you envisage µTCA? • We went straight for ATCA as we have a very big number of channels. • A study would be needed to understand data throughput on the uTCA backplane compared to ATCA, nb of slots needed, xTCA switches and external switches. We haven’t done that! • Do you need zero, one or more ATCA switch blades in your ATCA shelf? Which kind of switch? • To get the bandwidth for data collection we need 2 switch blades (10GbE) 12 in (fabric) / 12 out (front). • GbE for the control of blades via base interface. • TTC distribution ?????????   • Which MMC do you want to use on your AMC or in uTCA? • We are planning to use the DESY/CPPM/CERN design. Possible VME Replacement for the ATLAS upgrades G.Perrot

  18. Summary • A basic BE system will be installed in 2013-14 using our ROD Evaluator (2 blades). • The Digital Trigger system in 2018 (Phase I) will allow us to develop, test and master all the technologies needed for the Phase II developments (replacement of FEBs and RODs) • It will also allow us to understand what is the best compromise in term of density, power consumption, data collection. Possible VME Replacement for the ATLAS upgrades G.Perrot

More Related