1 / 54

Dezső Sima September 2008

3. Overall design space of main memories. Dezső Sima September 2008. (Ver. 1.0).  Sima Dezső, 2008. Contents. 1. Design space of MMs. 2. Underlying principles of the implementation of MMs. 3. Performance considerations. 4. References. 1. Design space of MMs (). Instruction Set

gamba
Download Presentation

Dezső Sima September 2008

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 3. Overall design space of main memories Dezső Sima September 2008 (Ver. 1.0)  SimaDezső, 2008

  2. Contents 1. Design space of MMs 2. Underlying principles of the implementation of MMs 3. Performance considerations 4. References

  3. 1. Design space of MMs () Instruction Set Architecture (ISA) Micro- architecture Underlying principle of operation Underlying principles of implementation Principles of attaching memory and I/O Von Neumann computational model Figure: Design space of processors

  4. 1. Design space of MMs () Control Set Architecture (CSA) Micro- architecture of the MM Underlying principle of operation Underlying principles of implementation Figure: Design space of main memories (MM)

  5. 1. Design space of MMs () Underlying principle of operation Refreshing (not discussed) Basic operation Figure: Underlying principle of operation of DRAM devices

  6. 1. Design space of MMs () Reads Activate Read Precharge Activate C: Command AD: Device address AB: Bank address AR: Raw address AC: Column address C AD AB AR C AD AB AC C AD AR AB C AD AB AR Read data (RD) tRCD tCL tRP t Writes Activate Write Precharge Activate C AD AB AR C AD AB AC C AD AR AB C AD AB AR Write data (WD) tRP tRCD tCL tWR t Basic operation of DRAM devices (Assuming device/bank/row/column addressing) Figure: Basic operation of DRAM devices

  7. 2. Underlying principles of the implementation of MMs () Underlying principles of the implementation of MMs Principle of communication Type of synchronisation One/two level implementation Bus topology Managing the DRAM status Type of signaling Signal grouping for communication Figure: Main dimensions of the design space of the underlying principles of implementation of MMs

  8. 2. Underlying principles of the implementation of MMs () One/two-level implementation One-level implementation Two-level implementation MM is built up of DRAM devices Figure: One/two level implementation of main memories

  9. 2. Underlying principles of the implementation of MMs () Figure: One level implementaqtion of the main memory (XDR memory of the Playstation 3 [1]

  10. 2. Underlying principles of the implementation of MMs () One/two-level implementation One-level implementation Two-level implementation MM is built up of modules, modules consist of DRAM devices MM is built up of DRAM devices Figure: One/two level implementation of main memories

  11. 2. Underlying principles of the implementation of MMs () Figure: Two level memory implementation (DDR2 modules on an MSI motherboard [2]

  12. 2. Underlying principles of the implementation of MMs () One/two-level implementation One-level implementation Two-level implementation MM is built up of modules, modules consist of DRAM devices MM is built up of DRAM devices Type of mounting Typically soldered Typically socketed Expandability Not expandable Easily expandable Board space needed Large boardspace Small boardspace Signal integrity Good signal integrity Unfavorable signal integrity (Earliest PC main memories) XDR memories All other types of main memories E.g. Figure: One/two level implementation of main memories

  13. 2. Underlying principles of the implementation of MMs () Managing DRAM status Detached from the basic operation Along with the basic operation (via a second dedicated interface) All other types of main memories RDRAM XDR Figure: Options to manage DRAM status This dimension of the design space is not discussed.

  14. 2. Underlying principles of the implementation of MMs () 1 0 1 1 0 0 t MC MC MC Principle of communication Packet-based Parallel bus based Signals are transferred over a parallel bus in one cycle Signals are transferred over a serial bus in a number of cycles 01 11 t E.g: 16 cycles/packet on a 1-bit wide bus 01 E.g: 64 bits in each cycle t E.g: 4 cycles/packet on a 4-bit wide bus Figure: Principles of communication used in main memories

  15. 2. Underlying principles of the implementation of MMs () Signal grouping for communication Signal grouping for communication In case of parallel bus based transmission In case of packet based transmission Line multiplexing Packet concept

  16. 2. Underlying principles of the implementation of MMs () Line multiplexing Due to the basic operation • Commands and addresses • are unidiredctional • (they flow in one direction, from the MC to the MM) • row and column addresses are not used at the same time, that is • they may be multiplexed without any performance penalty. • Data • is bidirectional • (read data flow from the MC to the MM, write data from the MM to the MC). • Read and write data may be multiplexed to reduce cost with a low performance penalty. Assumptions for line multiplexing • In order to avoid performance impediments data are transferred on a private bus, • instead of being multiplexed with commands and addresses. • Additional addresses needed (such as device or bank addresses) are transferred. • along with row and column addresses.

  17. 2. Underlying principles of the implementation of MMs () Line multiplexing Multiplexing row addresses/column addresses vs read and write data Row address/Column address multiplexing Not multiplexed Multiplexed Multiplexed (unidirectional) Read/write data multiplexing Asyncr. DRAMs (from the MK4096 on) First asynchr. DRAMs (before Mostel’s MK4086) Not multiplexed (bi-directional) Synchr. SDRAMs Figure: Multiplexing row and column addresses vs read and write data

  18. 2. Underlying principles of the implementation of MMs () Packet concept Different packet concepts for RDRAM, XDR and FB-DIMM memories.

  19. 2. Underlying principles of the implementation of MMs () The packet concept of RDRAM memories (1) Activate Read Figure: RDRAM: Different packet types for Activate/Precharge and Read/Write commands [9]

  20. 2. Underlying principles of the implementation of MMs () Row packets Column packets Data packets CR: Control Register R/W: Read/Write The packet concept of RDRAM memories (2) • Data packetsover the bidirectional data bus (DQA/DQB) • Row packets over the ROW bus • Column packets over the COL bus • CR R/W packets over the Serial bus Figure: RDRAM: Memory access packets [10]

  21. 2. Underlying principles of the implementation of MMs () Packet concept Different packet concepts for RDRAM, XDR and FB-DIMM memories. RDRAM • Different packet types (transferred via different buses) for Activate/Precharge and • Read/Write commands • Bidirectional data packets • Control register read/write packets.

  22. 2. Underlying principles of the implementation of MMs () Data packets CR R/W packets Request packets The packet concept of XDR memories [11] Figure: The packet concept of XDR memories

  23. 2. Underlying principles of the implementation of MMs () Packet concept Different packet concepts for RDRAM, XDR and FB-DIMM memories. RDRAM • Different packet types (transferred via different buses) for Activate/Precharge and • Read/Write commands • Bidirectional data packets • Control register read/write packets. XDR • Unified packet type for memory accesses (called the Request packets) • (Activate/Precharge and Read/Write commands) • Bidirectional data packets • Control register read/write packets.

  24. 2. Underlying principles of the implementation of MMs () The packet concept of FB-DIMM memories Memory controller Southbound packets Northbound packets • Commands • Write data • Read data • Status M. module M. module M. module Figure: The packet concept of FB-DIMM memories

  25. 2. Underlying principles of the implementation of MMs () Packet concept Different packet concepts for RDRAM, XDR and FB-DIMM memories. RDRAM • Different packet types (transferred via different buses) for Activate/Precharge and • Read/Write commands • Bidirectional data packets • Control register read/write packets. XDR • Unified packet type for memory accesses (called the Request packets) • (Activate/Precharge and Read/Write commands) • Bidirectional data packets • Control register read/write packets. FB_DIMM • Unified packet type for all commands (called the Southbound packets) • including memory accesses and control register reads/writes, • containing up to 3 commands, or write data and a single command. • Unidirectional read data (and status) packets (called the Northbound packets)

  26. 2. Underlying principles of the implementation of MMs () D I M M D I M M MC Bus topology Multi-drop bus Point-to-point connection Allows to connect more than one devices/modules to the bus Allows to interconnect two units (e.g. a mem. controller and a module) Stub-bus Fly-by Daisy-chained Connection via slots (sockets) E.g. Figure: Bus topologies used to connect DRAM devices or modules to the memory controller

  27. 2. Underlying principles of the implementation of MMs () Figure: Stub bus topology [3]

  28. 2. Underlying principles of the implementation of MMs () DRAM DRAM MC D I M M D I M M MC Bus topology Multi-drop bus Point-to-point connection Allows to connect more than one devices/modules to the bus Allows to interconnect two units (e.g. a mem. controller and a module) Stub-bus Fly-by Daisy-chained Connection via slots (sockets) Connection via soldering E.g. Figure: Bus topologies used to connect DRAM devices or modules to the memory controller

  29. 2. Underlying principles of the implementation of MMs () Fly-by topology Figure: Fly-by topology of the RQ bus in a two-channel XDR memory with two XDR devices/channel [5]

  30. 2. Underlying principles of the implementation of MMs () DRAM DRAM MC D I M M D I M M D I M M D I M M MC MC Bus topology Multi-drop bus Point-to-point connection Allows to connect more than one devices/modules to the bus Allows to interconnect two units (e.g. a mem. controller and a module) Stub-bus Fly-by Daisy-chained Connection via slots (sockets) Connection via soldering Connecting units to each other (outputs to inputs) E.g. Figure: Bus topologies used to connect DRAM devices or modules to the memory controller

  31. 2. Underlying principles of the implementation of MMs () Figure: Daisy chained topology of connecting AMBs in FB-DIMM memories [4] (There are two Command/Address buses (C/A) to reduce loading coming from 9 to 36 DRAMs mounted on the module)

  32. 2. Underlying principles of the implementation of MMs () DRAM DRAM DRAM MC MC DRAM D I M M D I M M D I M M D I M M MC MC Bus topology Multi-drop bus Point-to-point connection Allows to connect more than one devices/modules to the bus Allows to interconnect two units (e.g. a mem. controller and a module) Stub-bus Fly-by Daisy-chained Connection via slots (sockets) Connection via soldering Connecting units to each other (outputs to inputs) E.g. Figure: Bus topologies used to connect DRAM devices or modules to the memory controller

  33. 2. Underlying principles of the implementation of MMs () Point-to-point Figure: Point-to point topology of the data bus (DQ) in a two-channel XDR memory with two XDR devices/channel [5]

  34. 2. Underlying principles of the implementation of MMs () DRAM DRAM DRAM DRAM DRAM DRAM DRAM MC MC MC MC DRAM Bus topology Multi-drop bus Point-to-point bus Stub-bus Fly-by bus Daisy-chained bus ConnectingDRAM devices to the MC Used in Very early PCs RDRAMs (except the Serial bus) XDR/XDR2 (memory requests, control register reads/writes) XDR/XDR2 (read/write data) Figure: Overview of bus topologies connecting DRAM devices to the memory controller

  35. 2. Underlying principles of the implementation of MMs () D I M M D I M M MC Bus topology Multi-drop bus Point-to-point bus Stub-bus Fly-by bus Daisy-chained bus ConnectingDIMMs to the MC Parallel connected main memories Used in (FPM/EDO/SDRAM, DDR/DDR2/DDR3) RIMMs (with fly-by device connection on the module) MC Figure: Overview of bus topologies connecting DRAM modules to the memory controller

  36. 2. Underlying principles of the implementation of MMs () Figure: Contrasting the interconnection of RIMM modules with that of DIMMs [12]

  37. 2. Underlying principles of the implementation of MMs () D I M M D I M M D I M M D I M M D I M M D I M M DIMM MC MC MC MC Bus topology Multi-drop bus Point-to-point bus Stub-bus Fly-by bus Daisy-chained bus ConnectingDIMMs to the MC Parallel connected main memories FB-DIMMs (to connect AMBs) Used in (FPM/EDO/SDRAM, DDR/DDR2/DDR3) RIMMs (with fly-by device connection on the module) MC Not feasible Figure: Overview of bus topologies connecting DRAM modules to the memory controller

  38. 2. Underlying principles of the implementation of MMs () DRAM DRAM DRAM DRAM DRAM DRAM DRAM MC MC MC MC DRAM D I M M D I M M D I M M D I M M MC MC Bus topology Multi-drop bus Point-to-point bus Stub-bus Fly-by bus Daisy-chained bus Attaching DRAM devices to the MC Attaching DIMMs to the MC Unfavorable (due to TL discontinuities) Signal integrity Better Good Excellent Up to 16 Gb/s (with increasingly sophisticated termination) Peak transfer rate (recently) Up to 4.8 Gb/s Up to 4.8 Gb/s Up to 4.8 Gb/s Figure: Assessing bus topologies connecting DRAM devices/modules to the memory controller

  39. 2. Underlying principles of the implementation of MMs () Bus topologies of parallel connected synchronous MMs(Summary 1) Synchronous DRAMs (except DDR3) Bus topology Bus designation Buses I/O I I • read/write data • commands • addresses DQ [3:0/7:0/15:0] CS, RAS. CAS, WE BA [7:0], A [N:0] Stub bus Stub bus Stub bus DDR3 Stub bus Fly-by Fly-by I/O I I • read/write data • commands • addresses DQ [3:0/7:0/15:0] CS, RAS. CAS, WE BA [7:0], A [N:0]

  40. 2. Underlying principles of the implementation of MMs () Bus topologies of serial connected MMs (Summary 2) RDRAM Bus topology Bus designation Buses DQA [8:0], DQB [8:0] ROW [2:0] COL [4:0] CMD SIOI, SIOO • Read/write data • Row bus • Column bus • Serial if. • CMD • , SIO1, SIO0 Fly-by Fly-by Fly-by Fly-by Fly-by Daisy-chained I/O I I I/OI/O XDR I/O I O I • read/write data • memory requests) • control register (CR) reads • control register (CR) writes DQ [15:0] RQ [11:0] SDI SDO Point-to-point Fly-by Fly-by Fly-by FB-DIMM (AMBs - memory controller) O I • read data/device status • memory requests/ • write data/CR reads or writes PN [13:0] PS [9:0] Daisy-chained Daisy-chained

  41. 2. Underlying principles of the implementation of MMs () Address/control bus Multi-drop bus Point-to-point Stub-bus Fly-by bus Daisy-chained SDRAM DDR DDR2 (modules) Stub-bus Devices onthe modules DDR3 RDRAM (devices, modules) Multi-drop bus Fly-by FB-DIMM (AMBs on modules) Daisy-chnd Data bus XDR XDR2 (devices) TBI (devices) P2P Figure: Bus topologies of current MMs to connect DRAM devices or modules to the memory controller

  42. 2. Underlying principles of the implementation of MMs () Signals Voltage referenced Differential Single ended S+ VCM VREF S- t t t Typ.voltage swings 3.3-5 V 200-300 mV 600-800 mV TTL (5 V) FPM/EDO LVDS FB-DIMMs SSTL SSTL2 (DDR) SSTL1.8 (DDR2) SSTL1.5 (DDR3) RSL (RDRAM) Used in LVTTL (3.3 V) FPM/EDO SDRAM DRSL XDR (data) Smaller voltage swings LVDS: Low Voltage Differential Signaling LVTTL: Low Voltage TTL (D)RSL: (Differential) Rambus Signaling Level SSTL: Stub Series Terminated Logic VCM: Common Mode Voltage VREF: Reference Voltage Figure: Signal types used in MMs for control, address and data signals

  43. 2. Underlying principles of the implementation of MMs () Voltage swing vs signal rise/fall time Q = Cin x V = I x t tR~ Cin x V/I Q: Charge on the input capacitance of the line (Cin) Cin: Input capacitance of the line V: Voltage I: Current strength of the driver tR: Rise time shorter signal rise/fall times higher speed grades Smaller voltage swings but lower voltage budget higher requirements for signal integrity

  44. 2. Underlying principles of the implementation of MMs () Bus topologies and signaling of parallel connected MMs (Summary 1) Synchronous DRAMs (except DDR3) Signaling Bus topology Bus designation Buses I/O I I • read/write data • commands • addresses DQ [3:0/7:0/15:0] CS, RAS. CAS, WE BA [7:0], A [N:0] Stub bus Stub bus Stub bus Volt. ref. Volt. ref. Volt. ref. DDR3 Stub bus Fly-by Fly-by I/O I I • read/write data • commands • addresses DQ [3:0/7:0/15:0] CS, RAS. CAS, WE BA [7:0], A [N:0] Volt. ref. Volt. ref. Volt. ref.

  45. 2. Underlying principles of the implementation of MMs () Bus topologies and signaling of serial connected MMs (Summary 2) RDRAM Bus topology Signaling Bus designation Buses DQA [8:0], DQB [8:0] ROW [2:0] COL [4:0] CMD SIOI, SIOO • Read/write data • Row bus • Column bus • Serial if. • CMD • , SIO1, SIO0 Fly-by Fly-by Fly-by Fly-by Fly-by Daisy-chained I/O I I I/OI/O Volt ref. Volt. ref. Volt. ref. CMOS Volt. ref. Volt. ref. XDR I/O I O I • read/write data • memory requests) • control register (CR) reads • control register (CR) writes DQ [15:0] RQ [11:0] SDI SDO Point-to-point Fly-by Fly-by Fly-by Differential Volt. ref. Volt. ref. Volt. ref. FB-DIMM (AMBs - memory controller) O I • read data/device status • memory requests/ • write data/CR reads or writes PN [13:0] PS [9:0] Daisy-chained Daisy-chained Differential Differential

  46. 2. Underlying principles of the implementation of MMs () Synchronisation Capturing control/address information Central synchronization Source synchronization Mesochronous synchronization The sourcing device (MC or DRAM) sends a strobe signal along with the signals sent A central clock signal is used to latch the signals Keeps the clock frequency between the sender reference clock and the receiver reference clock but not the phase relationship.

  47. 2. Underlying principles of the implementation of MMs () Central clocking (SDRAM) Address, command and data lines are latched by the rising edge of the central clock (CLK) Figure: Central clocking of address, command and data lines in an SDRAM device while writing random data [6]

  48. 2. Underlying principles of the implementation of MMs () Source synchronous clocking of the data lines (DDR) Command and address lines are latched by the differential clock (CK, CK#) but write data are latched by the rising edge of the source synchronous data strobe (DQS) Figure: Source synchronous clocking (DDR SDRAMs) of the data lines in a DDR device while writing random data [7] (TDOSS: Write command to first DQS latching transition)

  49. 2. Underlying principles of the implementation of MMs () Mesochronous clocking (FB-DIMM) Figure: Mesochronous clocking used to synchronise AMBs in FB-DIMM memories [8]

  50. 2. Underlying principles of the implementation of MMs () Capturing control/address information Central synchronization Source synchronization Mesochronous synchronization Central synch. SDRAM RDRAM XDR3 XDR23 DDR1 DDR22 DDR32 Source synch. Capturing data DDRX? Mesochron. synch. FBDIMM 1: Phase alignement for data reads/writes 2: Phase alignement for data reads/writes by read/write leveling 3: Phase alignement for all signals by FlexPhase Figure: Synchronisation alternatives

More Related