A Case for FAME: FPGA Architecture Model Execution

1. A Case for FAME:FPGA Architecture Model Execution Zhangxi Tan, Andrew Waterman, Henry Cook, Sarah Bird, KrsteAsanovic, David Patterson The Parallel Computing Lab, UC Berkeley ISCA ’10

2. A Brief History of Time • Hardware prototyping initially popular for architects • Prototyping each point in a design space is expensive • Simulators became popular cost-effective alternative • Software Architecture Model Execution (SAME) simulators most popular • SAME performance scaled with uniprocessorperformance scaling

3. The Multicore Revolution • Abrupt change to multicorearchitectures • HW, SW systems larger, more complex • Timing-dependent nondeterminism • Dynamic code generation • Automatic tuning of app kernels • We need more simulation cycles than ever

4. TheMulticore Simulation Gap • As number of cores increases exponentially, time to model a target cycle increases accordingly • SAME is difficult to parallelize because of cycle-by-cycle interactions • Relaxed simulation synchronization may not work • Must bridge simulation gap

5. One Decade of SAME • Effect is dramatically shorter (~10 ms) simulation runs

6. FAME: FPGA Architecture Model Execution • The SAME approach provides inadequate simulation throughput and latency • Need a fundamentally new strategy to maximize useful experiments per day • Want flexibility of SAME and performance of hardware • Ours: FPGA Architecture Model Execution (FAME) • (cf. SAME, Software Architecture Model Execution) • Why FPGAs? • FPGA capacity scaling with Moore’s Law • Now can fit a few cores on die • Highly concurrent programming model with cheap synchronization

7. Non-FAME:FPGA Computers • FPGA Computers: using FPGAs to build a production computer • RAMP Blue (UCB 2006) • 1008 MicroBlaze cores • No MMU, message passing only • Requires lots of hardware • 21 BEE2 boards (full rack) / 84 FPGAs • RTL directly mapped to FPGA • Time-consuming to modify • Cool, useful, but not a flexible simulator

8. FAME:System Simulators in FPGAs Target System B Target System A I$ I$ I$ I$ I$ I$ I$ CORE CORE CORE CORE CORE … CORE CORE D$ D$ D$ L2$ L2$ L2$ D$ D$ D$ D$ DRAM Shared L2$ / Interconnect Host System (FAME simulator) DRAM

9. A Vast FAME Design Space • FAME design space even larger than SAME’s • Three dimensions of FAME simulators • Direct or Decoupled: does one host cycle model one target cycle? • Full RTL or Abstract RTL? • Host Single-threaded or Host Multi-threaded? • See paper for a FAME taxonomy!

10. FAME Dimension 1:Direct vs. Decoupled • Direct FAME: compile target RTL to FPGA • Problem: common ASIC structures map poorly to FPGAs • Solution: resource-efficient multi-cycle FPGA mapping • Decoupled FAME: decouple host cycles from target cycles • Full RTL still modeled, so timing accuracy still guaranteed R1 R2 W1 Rd1 Rd2 R1 R2 R3 R4 W1 W2 Rd1 Rd2 Rd3 Rd4 RegFile RegFile FSM Target System Regfile Decoupled Host Implementation

11. FAME Dimension 2:Full RTL vs. Abstract RTL • Decoupled FAME models full RTL of target machine • Don’t have full RTL in initial design phase • Full RTL is too much work for design space exploration • Abstract FAME: model the target RTL at a high level • For example, split timing and functional models (à la SAME) • Also enables runtime parameterization: run different simulations without re-synthesizing the design • Advantages of Abstract FAME come at cost: model verification • Timing of abstract model not guaranteed to match target machine Functional Model Target RTL Abstraction Timing Model

12. FAME Dimension 3:Single- or Multi-threaded Host CPU1 CPU2 CPU3 CPU4 Target Model IR PC 1 X PC 1 PC 1 PC 1 Y Multithreaded Emulation Engine (on FPGA) GPR GPR I$ Single hardware pipeline with multiple copies of CPU state GPR GPR1 D$ 2 2 +1 • Problem: can’t fit big manycore on FPGA, even abstracted • Problem: long host latencies reduce utilization • Solution: host-multithreading

13. Metrics besides Cycles:Power, Area, Cycle Time • FAME simulators determine how many cycles a program takes to run • Computing Power/Area/Cycle Time: SAME old story • Push target RTL through VLSI flow • Analytical or empirical models • Collecting event stats for model inputs is much faster than with SAME

14. RAMP Gold: A Multithreaded FAME Simulator Rapid accurate simulation of manycore architectural ideas using FPGAs Initial version models 64 cores of SPARC v8 with shared memory system on $750 board Hardware FPU, MMU, boots OS. 14

15. RAMP Gold Target Machine 64 cores SPARC V8 CORE SPARC V8 CORE … SPARC V8 CORE SPARC V8 CORE I$ D$ I$ D$ I$ D$ I$ D$ Shared L2$ / Interconnect DRAM 15

16. RAMP Gold Model Timing Model Pipeline Functional Model Pipeline Arch State Timing State 64 cores • SPARC V8 ISA • One-socket manycore target • Split functional/timing model, both in hardware • Functional model: Executes ISA • Timing model: Capture pipeline timing detail • Host multithreading of both functional and timing models • Functional-first, timing-directed • Built for Xilinx Virtex-5 systems [ RAMP Gold, DAC ‘10 ] I$ I$ I$ I$ CORE CORE … CORE CORE D$ D$ D$ D$ Shared L2$ / Interconnect DRAM 16

17. Case Study: Manycore OS Resource Allocation • Spatial resource allocation in a manycore system is hard • Combinatorial explosion in number of apps and number of resources • Idea: use predictive models of app performance to make it easier on OS • HW partitioning for performance isolation (so models still work when apps run together) • Problem: evaluating effectiveness of resulting scheduling decisions requires running hundreds of schedules for billions of cycles each • Simulation-bound: 8.3 CPU-years for Simics! • See paper for app modeling strategy details

18. Case Study: Manycore OS Resource Allocation

19. Case Study: Manycore OS Resource Allocation • The technique appears to perform very well for synthetic or reduced-input workloads, but is lackluster in reality!

20. RAMP Gold Performance • FAME (RAMP Gold) vs. SAME (Simics) Performance • PARSEC parallel benchmarks, large input sets • >250x faster than full system simulator for a 64-core target system

21. Researcher Productivity is Inversely Proportional to Latency • Simulation latency is even more important than throughput • How long before experimenter gets feedback? • How many experimenter-days are wasted if there was an error in the experimental setup?

22. Fallacy: FAME is too hard • FAME simulators more complex, but not greatly so • Efficient, complete SAME simulators also quite complex • Most experiments only need to change timing model • RAMP Gold’s timing model is only 1000 lines of SystemVerilog • Modeled Globally Synchronized Frames [Lee08] in 3 hours & 100 LOC • Corollary fallacy: architects don’t need to write RTL • We design hardware; we shouldn’t be scared of HDL

23. Fallacy: FAME Costs Too Much • Running SAME on cloud (EC2) much more expensive! • FAME: 5 XUP boards at $750 ea.; $0.10 per kWh • SAME: EC2 Medium-High instances at $0.17 per hour • Are architects good stewards of the environment? • SAME uses energy of 45 seconds of Gulf oil spill!

24. Fallacy: statistical samplingwill save us • Sampling may not make sense for multiprocessors • Timing is now architecturally visible • May be OK for transactional workloads • Even if sampling is appropriate, runtime dominated by functional warming => still need FAME • FAME simulator ProtoFlex (CMU) originally designed for this purpose • Parallel programs of the future will likely be dynamically adaptive and auto-tuned, which may render sampling useless

25. Challenge: Simulator Debug Loop can be Longer • Takes 2 hours to push RAMP Gold through the CAD tools • Software RTL simulation to debug simulator is also very slow • SAME debug loop only minutes long • But sheer speed of FAME eases some tasks • Try debugging and porting a complex parallel program in SAME

26. Challenge: FPGA CAD Tools • Compared to ASIC tools, FPGA tools are immature • Encountered 84 formally-tracked bugs developing RAMP Gold • Including several in the formal verification tools!! • By far FAME’s biggest barrier • (Help us, industry!) • On the bright side, the more people using FAME, the better

27. When should Architectsstill use SAME? • SAME still appropriate in some situations • Pure functional simulation • ISA design • Uniprocessor pipeline design • FAME necessary for manycore research with modern applications

28. Conclusions • FAME uses FPGAs to build simulators, not computers • FAME works, it’s fast, and we’re using it • SAME doesn’t cut it, so use FAME! • Thanks to the entire RAMP community for contributions to FAME methodology • Thanks to NSF, DARPA, Xilinx, SPARC International, IBM, Microsoft, Intel, and UC Discovery for funding support RAMP Gold source code is available: http://ramp.eecs.berkeley.edu/gold