OpenSPARC-Xilinx Collaboration

1. Durgam Vahia Paul Hartke Durgam.Vahia@Sun.Compaul.hartke@xilinx.com OpenSPARC Engineering Xilinx University Program (XUP) RAMP Retreat, UC Berkeley, January 2007 OpenSPARC-Xilinx Collaboration

2. Agenda Goals OpenSPARC T1 – Quick Recap What we have been up to – T1 on FPGAs Current Status and Results Road-map

3. Big Goals • Proliferation of Sun OpenSPARC technology • Proliferation of Xilinx FPGA technology • Make OpenSPARC FPGA-friendly • Create reference design with complete system functionality and proven path to hardware • Boot Solaris/Linux on the reference design • Open it up .. • Seed ideas in the community Significant enabler for future research in multi-core

4. What is OpenSPARC T1 • SPARC V9 implementation • Eight cores, four thread each - 32 simultaneous threads • All cores connect through a 134.4GB/s crossbar switch • High BW 12 way associative 3MB on-chip L2 cache • 4 DDR2 channels (23 GB/s) • 70W power • ~300M transistors

5. OpenSPARC T1: Design Choices Double-click to add graphics • Simpler core architecture to maximize cores on die • Caches, DRAM channels shared across cores • Shared L2 decreases cost of coherence misses significantly • Crossbar good for b/w, latency and functional verification

6. OpenSPARC Core MUL EXU IFU MMU LSU TRAP • Four threads per core • Single issue 6 stage pipeline • 16KB I-cache, 8KB D-cache • Unique resources per thread • Registers • Portions of I-fetch datapath • Store and Miss buffers • Resources shared by 4 threads • Caches, TLBs, Execution units • Pipeline registers and DP

7. OpenSPARC Pipeline All processor IO (including interrupts) via Crossbar interface http://opensparc-t1.sunsource.net/specs/OpenSPARCT1_Micro_Arch.pdf

8. OpenSPARC T1 on FPGAs Create single core, single thread implementation of T1 for FPGAs Map it on Xilinx FPGA board and use board peripherals to build the working hardware system Boot commercial OS on it

9. OpenSPARC FPGA Implementation Single core, single thread implementation of T1 Small, clean and modular FPGA implementation About 39K 4-input LUTs, 123 BRAMs (synplicity on Virtex{2/2Pro/4}) Synchronous, no latches or gated clocks Better utilization of FPGA resources (BRAMs, Multiplier) Functionally equivalent to custom implementation, except 8 entry Fully Associative TLB as opposed to 64 entry Removed Crypto unit (modular arithmetic operations)

10. Single Thread T1 on FPGAs Functionally stable Passing mini and full regressions Completely routed No timing violations Easily meets 20ns (50MHz) cycle time Expandable to more threads Reasonable overhead for most blocks (~30% for 4 threads) Some bottlenecks exist (Multi-port register files)

11. System Block Diagram Xilinx Embedded Developer’s (EDK) Design MultiPort Memory Controller FPGA Boundary Block must be developed External DDR2 Dimm MCH-OPB MemCon PCX-FSL Interposer SPARC T1 Core Microblaze Proc Microblaze Debug UART SPARC T1 UART processor-to-crossbar interface (PCX) Fast Simplex Links interface (FSL) 10/100 Ethernet IBM Coreconnect OPB Bus

12. System Theory of Operation OpenSPARC T1 core communicates exclusively via the processor-to-crossbar interface (PCX) PCX is a packet based interface Microblaze softcore will sit in a polling loop and accept these packets, perform any protocol conversion, and forward them to the appropriate peripheral Could even implement floating point operations via the Microblaze FPU unit Microblaze will also poll (or accept interrupts from) the peripherals, convert the info to a PCX packet, and forward it to the PCX interface Microblaze has its own UART for its own diagnostic input/output

13. Implementation Results XC4VFX100-11FF1152 FPGA 42,649/84,352 LUT4s (50%) 131/376 BRAM-16kbits (34%) 50MHz operation Have not attempted any faster Synplicity Synthesis: 25 minutes Place and Route: 42 minutes (Microblaze & Related Logic)

14. Preliminary Virtex5 Results Virtex5 xc5vlx110tff1136 Same as Bee3 FPGA 30,508/69,120 LUT6s (44%) 119/148 BRAM-36kbits (80%) Working through mapping issues…  50MHz placed and routed design Have not attempted any faster

15. OpenSPARC FPGA HW Roadmap Current reference design occupies about 45% of XC4V100FX FPGA. This design includes Single core, single thread of OpenSPARC T1 Microblaze to communicate with peripherals (DRAM, Ethernet) Glue logic to connect T1 core with Microblaze More design paths exist, e.g. 1) Two single thread cores in single FPGA 2) Up to 4 threads per FPGA

16. OpenSPARC FPGA SW Roadmap Boot Solaris and Linux on a single thread FPGA version of the design Include support for all packet types with Microblaze Hypervisor changes to support this variant of T1 Reduction in TLB size Device driver support for the system Emulation routines in OS for floating point operations Mainly for ISA compliance

17. Reference Design ml410 board with Virtex4-100 FPGA (aka ml411) Bit file and elf is stored on CompactFlash card Each design is a hardware implementation of one regression suite test Microblaze soft-core sends the test packets to the OpenSPARC core and verifies the return packets

18. www.opensparc.net • All of this will be available under GPL(2) license • Complete verilog code of FPGA T1 and glue logic to Microblaze • Synplicity scripts for synthesis • The whole reference design

19. www.opensparc.net (2) Verification Environment Very very important – Change and VERIFY Scripts for running regression in three modes Chip8 – Full-chip test-suit Core1 – Single core (four threads) test-suit Thread1 – Single core, Single thread test-suit for FPGA version Supports Synopsys VCS and Cadence NC-Verilog Considering supporting Mentor ModelSim as well Bring down as many barriers as possible

20. Development Team Sun OpenSPARC Team Durgam Vahia Ismet Bayraktaroglu Thomas Thatcher Xilinx University Program Paul Hartke

21. OpenSPARC & Xilinx FPGAs!!