Atlas Wisconsin/LBNL Group John Joseph March 21 st 2007

1. Upgrade to the Read-Out Driver for ATLAS Silicon Detectors Atlas Wisconsin/LBNL Group John Joseph March 21st 2007 ATLAS Pixel B-Layer Upgrade Workshop

2. Silicon Read Out Driver (SiROD)

3. ROD Control/VME Path Back Of Crate (BOC) BOOT SLAVE DSP FROM FLASH 128Mb Flash CNFG & BOOT Memory MOVE HPI INTERFACE TO VME CONTROLLER VME Slave FPGA Configuration Controller ROD BUSY Histogram ROD CONTROLLER FPGA XC4VFX20-10 Power PC Core FORMATTER FPGA XC3S4000-8 2x DSP FARM TI 320C6713 220MHz 384MB SDRAM 4x 1.5 GB SDRAM 256 MB SDRAM DIAGNOSTIC FIFOs EFB /ROUTER FPGA XC4VLX60-10 ADDR CNFG DATA to All FPGAs VME Bus A32/D32 D32 ROD BUSY ADDR RESET ARB D8 D16 D48

4. SiROD DATA PATH S-Link BOC TIM BOC FORMATTER FPGA FORMATTER FPGA INPUT FIFO 32K Deep DSP FARM TI 320C6713 220MHz 384MB SDRAM 4x EFB/ROUTER FPGA XC4VLX60 ROD CONTROLLER FPGA XC4VFX20 Event Fragments XC3S4000 Xoff DATA Xoff DATA Xoff EVENT ID STATUS TTC READOUT ROD BUSY MDSP SP FE CMD (48)

5. COMPONENT COMPARISONS: FORMATTER • RevF SiROD • Spartan2E Series: XC2S600-6FG456 • Cost: $100 x 8 = $800 • Available resources: 6912 Slices, 288K RAM Bits per FPGA • Used resourses (1) : 2850 Slices (41%), 268K RAM Bits (93%) • Used resourses (8) : 22.8K Slices (41%), 2140K RAM Bits (93%) • Module FIFOs: 32 – 2048 x 32 Dual Port RAM Blocks • Next Generation SiROD • Spartan3 Series: XC3S4000-8FG900 • Cost: $220 x 2 = $440 • Available resources: 54K Slices, 3456K RAM Bits per FPGA • Required resources (2) : 23K Slices, 1072K RAM Bits • Module FIFOs: 32 - 4096 x 32 Dual Port RAM Blocks

6. COMPONENT COMPARISONS: EFB/ROUTER • RevF SiROD • Spartan2E Series: XC2S400-6FG676/XC2S400-6FG456 • Cypress 6 - 16K x18 15ns FIFOs • Cost: $115 + $85 + $210 = $410 • Available resources: 9600 Slices, 1856K RAM + FIFO Bits • Used resourses : 6080 Slices (63%), 1824K RAM + FIFO Bits • Trap FIFOs: 4 1024 x 32 Dual Port RAM Blocks • Next Generation SiROD • Virtex4 Series: XC4LX60-10FG1152 • Cost: $558 • Available resources: 26K Slices, 2880K RAM Bits • Required resources: 6080 Slices, 1824K RAM Bits • Trap FIFOs: 4 - 4096 x 32 Dual Port RAM Blocks

7. COMPONENT COMPARISONS: CONTROLLER • RevF SiROD • Spartan2E Series: XC2S600-6FG676 / TI320C6201-200 DSP • Cost: $130 + $100 = $230 • Available resources: 6912 Slices, 288K RAM + FIFO Bits • Used resourses : 5130 Slices (74%), 188K RAM Bits (65%) • DSP Processor • Next Generation SiROD • Virtex4 Series: XC4FX20-10FG • Cost: $228 • Available resources: 8544 Slices, 1224K RAM Bits, Power PC Core • Required resources: 5130 Slices, 188K RAM Bits, uProcessor • Conversion from TI DSP to Xilinx Power PC core: • Risk Evaluation Required

8. Silicon RevE(F) SiROD Clock Distribution Slave DSPs Router FPGA VME Interface 9U VME Controller FPGA Event Fragment Builder FPGA Master DSP Formatter FPGAs

9. SiROD New Layout BOC J3 VME J1 J2 INPUT FIFO 32K Deep FORMATTER FPGA 128Mb FLASH FORMATTER FPGA XC3S4000 XC3S4000 EFB/ROUTER FPGA XC4VLX60 DSP FARM TI 320C6713 220MHz 384MB SDRAM DSP FARM TI 320C6713 220MHz 384MB SDRAM DSP FARM TI 320C6713 220MHz 384MB SDRAM DSP FARM TI 320C6713 220MHz 384MB SDRAM VME CONTROLLER FPGA XC3S400 ROD CONTROLLER FPGA XC4VFX20 128MB SDRAM JTAG CONNECTOR ON FRONT PANEL TO PROGRAM VME CONTROLLER FPGA ROM

10. WHERE DO WE GO FROM HERE? • Preliminary Design Study • Recommendations: • Purchase 2 Xilinx Virtex4 Evaluation Boards • One for the LX series • One for the FX series • Combine the EFB/EventMemory/Router Designs to simulate and implement the proposed changes for verification • Simulate and implement the Formatter as proposed for verification • Simulate and implement the Controller to learn how to use the Power PC core and to plan a transition from the TI 6201 to the Xilinx PC core. • Make the new design transparent to the DAQ system. • Simulate the Control path to understand the limitations of VME BW

11. Additional Slides

12. PIXEL FORMATTER FPGA (8x) Read Out Controller Link Input MUX & Half Clock Counters Control & Status Registers (32) Link FIFO 2048 x 32 Link Decoder: 40/80/160MHz Link FIFO 2048 x 32 Link Decoder: 40/80MHz Link FIFO 2048 x 32 Link Decoder: 40MHz Link Decoder: 40MHz Link FIFO 2048 x 32 RODBus Interface Trailer Detect ModeBits FIFO 512x24 DATA OUT (40) IN0 IN1 IN2 IN3 IN4 IN5 IN6 IN7 IN8 IN9 IN10 IN11 ROD BUSY HT LIMIT TOKEN FMT ID TRAILER (12) FMT Type RODBus Data (16) RODBus Address FMT MB (12) RODBus Control

13. EVENT FRAGMENT BUILDER FPGA EventMemB FIFO 16Kx48 EventMemA FIFO 16Kx48 FIFO Controller Purpose: Collect Formatter output, check L1 and BC IDs, count errors, generate Event Header & Trailer Engine 0 Halt Output Formatter Num. (2 bits) Link Num. (6 bits) Link Num. (4 bits) Time-out Error To/From Formatter Output FIFO Data (43) Data (32 bits) L1ID Error INC/DEC L1ID Format & Count Error Check L1/BC ID Check Time Out BCID Error Data (32 bits) Data Valid Xoff Error Summary Word Event Data & Trigger Type To/From Router Header and Trailer Data (32) Header/Trailer Generator To/From Controller L1 & BC IDs Dynamic Mask EventMemC Error Summary Word Data Valid Data (32 bits) BCID Error Time Out INC/DEC L1ID Format & Count L1/BC ID Check Error Check L1ID Error Data (32 bits) To/From Formatter Time-out Error Link Num. (4 bits) Link Num. (6 bits) Formatter Num. (2 bits) Halt Output Engine 1

14. ROD Router (FPGA) & Slave DSPs Purpose: Route formatted data to Level-2 and/or Slave DSPs (histogram) Halt Output XOn / XOff S-Link Dsp Halt Output S-Link Data Event Header and Trailer Data (32b) S-LINK To/From ROS Read, format and direct the event data To/From Event Fragment Builder Output FIFO (43b) Data Valid Error Format Data 2 Traps 2 Traps 2 Traps 2 Traps Event Type: ATLAS, ROD or TIM Event Type: ATLAS, ROD or TIM Event Type: ATLAS, ROD or TIM Event Type: ATLAS, ROD or TIM DMA Transfer Engine DMA Transfer Engine DMA Transfer Engine DMA Transfer Engine 1024 32-bit FIFO 1024 32-bit FIFO 1024 32-bit FIFO 1024 32-bit FIFO SDSP 2 SDSP 0 SDSP 3 SDSP 1 Texas Instruments 6713 floating point DSPs running at 220 MHz for monitoring and calibration histogramming