PRET DRAM Controller: Bank Privatization for Predictability and Temporal Isolation

1. PRET DRAM Controller: Bank Privatization for Predictability and Temporal Isolation SungjunKim Columbia University Edward A. Lee UC Berkeley Isaac Liu UC Berkeley Hiren D. Patel University of Waterloo Jan Reineke UC Berkeley <speaker> CODES+ISSS as part of ESWEEK 2011 Taipei, Taiwan, October 10th, 2011

2. Predictability and Temporal Isolation • Many embedded systems are real-time systems • Memory hierarchy has a strong influence on their performance: • Need for Predictability • Trend towards integrated architectures: • Need for Temporal Isolation Audio + video playback with latency and bandwidth constraints

3. Outline • Introduction • DRAM Basics • Related Work: Predator and AMC • PRET DRAM Controller: Main Ideas • Evaluation • Integration into Precision-Timed ARM

4. Outline • Introduction • DRAM Basics • Related Work: Predator and AMC • PRET DRAM Controller: Main Ideas • Evaluation • Integration into Precision-Timed ARM

5. Memory Hierarchy:Dynamic RAM vs Static RAM • SRAM • Fast  Low Latency • Low Capacity • DRAM • Slow  High Latency • High Capacity from Hennessy and Patterson, Computer Architecture: A Quantitative Approach, 2007.

6. Dynamic RAM Organization Overview • DRAM Device • Set of DRAM banks + • Control logic • I/O gating • Accesses to banks can be pipelined, however I/O + control logic are shared DRAM Cell Leaks charge  Needs to be refreshed (every 64ms for DDR2/DDR3) therefore “dynamic” • DRAM Module • Collection of DRAM Devices • Rank = groups of devices that operate in unison • Ranks share data/address/command bus DRAM Bank = Array of DRAM Cells + Sense Amplifiers and Row Buffer Sharing of sense amplifiers and row buffer

7. DRAM Memory Controller Translates sequences of memory accesses by Clients (CPUs and I/O) into legal sequences of DRAM commands • Needs to obey all timing constraints • Needs to insert refresh commands sufficiently often • Needs to translate “physical” memory addresses into row/column/bank tuples

8. Dynamic RAM Timing Constraints DRAM Memory Controllers have to conform to different timing constraints that define minimal distances between consecutive DRAM commands. Almost all of these constraints are due to the sharing of resources at different levels of the hierarchy: • Needs to insert refresh commands sufficiently often • Banks within a DRAM device share I/O gating and control logic • Rows within a bank share sense amplifiers • Different ranks share data/address/command busses

9. General-Purpose DRAM Controllers • Schedule DRAM commands dynamically • Timing hard to predict even for single client: • Timing of request depends on past requests: • Request to same/different bank? • Request to open/closed row within bank? • Controller might reorder requests to minimize latency • Controllers dynamically schedule refreshes • Non-composable timing. Timing depends on behavior of other clients: • They influence sequence of “past requests” • Arbitration may or may not provide guarantees

10. General-Purpose DRAM Controllers Load B1.R3.C2 Load B1.R4.C3 Load B1.R3.C5 … Memory Controller ? RAS B1.R3 CAS B1.C2 RAS B1.R4 CAS B1.C3 RAS B1.R3 CAS B1.C5 … … … RAS B1.R3 CAS B1.C2 CAS B1.C5 RAS B1.R4 CAS B1.C3 … …

11. General-Purpose DRAM Controllers Thread 1 Thread 2 Load B1.R3.C2 Load B2.R4.C3 Store B4.R3.C5 Load B3.R3.C2 Load B3.R5.C3 Store B2.R3.C5 Arbitration ? Load B1.R3.C2 Load B3.R3.C2 Load B2.R4.C3 Store B4.R3.C5 Load B3.R5.C3 Store B2.R3.C5 Load B3.R3.C2 Load B1.R3.C2 Load B2.R4.C3 Load B3.R5.C3 Store B4.R3.C5 Store B2.R3.C5 Memory Controller

12. Outline • Introduction • DRAM Basics • Related Work: Predator and AMC • PRET DRAM Controller: Main Ideas • Evaluation • Integration into Precision-Timed ARM

13. Predictable DRAM Controllers:Predator (Eindhoven) and AMC (Barcelona) Spread each request over all banks, pipeline accesses to banks. Closed-page policy: timing independent of previously accessed row Statically precomputedsequences for writes, reads, write->read, read->write, refresh. • Predictableand/or composablearbitration: • Predator: CCSP • AMC: TDMA

14. Predictable DRAM Controllers:Predator (Eindhoven) Load B1.R3.C2 Load B1.R4.C3 Store B1.R3.C5 … Predictable Memory Controller: Predator Read Pattern Read Pattern R/W Pattern Write Pattern Closed-page policy: timing independent of previously accessed row Statically precomputedsequences for writes, reads, write->read, read->write, refresh. Spread each request over all banks, pipeline accesses to banks. • increases access granularity

15. Predictable DRAM Controllers:Predator (Eindhoven) and AMC (Barcelona) Thread 1 Thread 2 Load B1.R3.C2 Load B3.R3.C2 Load B3.R5.C3 Store B2.R3.C5 … Predictable and/or Composable Arbitration (e.g. time-division multiple access) ? Load B1.R3.C2 Load B3.R3.C2 Load B3.R5.C3 Store B2.R3.C5 Memory Controller

16. Outline • Introduction • DRAM Basics • Related Work: Predator and AMC • PRET DRAM Controller: Main Ideas • Evaluation • Integration into Precision-Timed ARM

17. PRET DRAM Controller:Three Innovations • Expose internal structure of DRAM devices: • Expose individual banks within DRAM device as multiple independent resources • Defer refreshes to the end of transactions • Allows to hide refresh latency • Perform refreshes “manually”: • Replace standard refresh command with multiple reads

18. PRET DRAM Controller: Exploiting Internal Structure of DRAM Module • Consists of 4-8 banks in 1-2 ranks • Share only command and data bus, otherwise independent • Partition into four groups of banks in alternating ranks • Cycle through groups in a time-triggered fashion Bank 0 Bank 0 Bank 1 Bank 1 • Successive accesses to same group obey timing constraints • Reads/writes to different groups do not interfere Rank 0: Group 0 Group 2 Bank 2 Bank 3 Rank 1: Group 1 Group 3 Provides four independent and predictable resources Bank 2 Bank 3

19. PRET DRAM Controller: Exploiting Internal Structure of DRAM Module Load B1.R3.C2 Load B1.R4.C3 Store B1.R3.C5 … PRET DRAM Controller Read Pattern Read Pattern Write Pattern …

20. Pipelined Bank Access Scheme READ WRITE READ

21. PRET DRAM Controller:“Manual” Refreshes • Every row needs to be refreshed every 64ms • Dedicated refresh commands refresh one row in each bank at once • We replace these with “manual” refreshes through reads • Improves worst-case latency of short requests Dedicated refresh commands vs refreshes through reads. (refresh latencies not to scale)

22. PRET DRAM Controller:Defer Refreshes • Refreshes do not have to happen periodically • Refresh every row at least every 64 ms • Schedule refreshes slightly more often than necessary  Enables to defer refreshes

23. General-Purpose DRAM Controllervs PRET DRAM Controller General-Purpose Controller PRET DRAM Controller Abstracts DRAM as multiple independent resources Refreshes as reads: shorter interruptions Defer refreshes: improves perceived latency Follows periodic, time-triggered schedule “Closed page” policy: access-history independence • Abstracts DRAM as a single shared resource • Schedules refreshes dynamically • Schedules commands dynamically • “Open page” policy speculates on locality

24. Outline • Introduction • DRAM Basics • Related Work: Predator and AMC • PRET DRAM Controller: Main Ideas • Evaluation • Integration into Precision-Timed ARM

25. Conventional DRAM Controller (DRAMSim2)vs PRET DRAM Controller: Latency Evaluation Varying Interference: Varying Transfer Size:

26. PRET DRAM Controller vs Predator:Analytical Evaluation • Predator: • abstracts DRAM as single resource • uses standard refresh mechanism • PRET controller improves worst-case access latency of small transfers

27. PRET DRAM Controller vs Predator:Analytical Evaluation • Less of a difference for larger transfers • Predator provides slightly higher bandwidth due to more efficient refresh mechanism

28. Outline • Introduction • DRAM Basics • Related Work: Predator and AMC • PRET DRAM Controller: Main Ideas • Evaluation • Integration into Precision-Timed ARM

29. Precision-Timed ARM (PTARM) Architecture Overview http://chess.eecs.berkeley.edu/pret/ • Thread-Interleaved Pipeline for predictable timingwithout sacrificing high throughput • One private DRAM Resource + DMA Unit per Hardware Thread • Shared Scratchpad Instruction and Data Memories for low latency access

30. Conclusions and Future Work • Temporal isolation and improved worst-case latency by bank privatization • How to program the inverted memory hierarchy? Raffaello Sanzio da Urbino – The Athens School

31. References Related Work on Memory Controllers: • M. Paolieri, E. Quiñones, F. Cazorla, and M. Valero, “An analyzable memory controller for hard real-time CMPs,” IEEE Embedded Systems Letters, vol. 1, no. 4, pp. 86–90, 2010. • B. Akesson, K. Goossens, and M. Ringhofer, “Predator: a predictable SDRAM memory controller,” in CODES+ISSS. ACM, 2007, pp. 251–256. Work within the PRET project: • [CODES ’11] Jan Reineke, Isaac Liu, Hiren D. Patel, Sungjun Kim, Edward A. Lee, PRET DRAM Controller: Bank Privatization for Predictability and Temporal Isolation, International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS), October, 2011. • [DAC ‘11] Dai Nguyen Bui, Edward A. Lee, Isaac Liu, Hiren D. Patel, Jan Reineke, Temporal Isolation on Multiprocessing Architectures, Design Automation Conference (DAC), June, 2011. • [Asilomar ‘10] Isaac Liu, Jan Reineke, and Edward A. Lee, PRET Architecture Supporting Concurrent Programs with Composable Timing Properties, in Signals, Systems, and Computers (ASILOMAR), Conference Record of the Forty Fourth Asilomar Conference, November 2010, Pacific Grove, California. • [CASES ’08] Ben Lickly, Isaac Liu, Sungjun Kim, Hiren D. Patel, Stephen A. Edwards and Edward A. Lee, "Predictable Programming on a Precision Timed Architecture," in Proceedings of International Conference on Compilers, Architecture, and Synthesis for Embedded Systems (CASES), Piscataway, NJ, pp. 137-146, IEEE Press, October, 2008.