The Fresh Breeze Memory Model Status: Linear Algebra and Plans

1. The Fresh Breeze Memory ModelStatus: Linear Algebra and Plans Jack Dennis Guang R. Gao Xiaoxuan Meng Joshua Slocum Brian Lucas MIT CSAIL University of Delaware Funded in part by NSF HECURA Grant CCF-0937832

2. Problem: I/O Performance P $ File Main Memory L3 $ P $ Disk Memory P $ Other Managed by hardware Managed by software (OS) Culprits: • Operating System Overhead/Noise • Large Units of Data Transfer • Few Concurrent Transfers (OS Limits) 2

3. Solution: Integrated Memory Hierarchy P $ File Main Memory L3 $ P $ Disk Memory P $ Other Managed by hardware Features: • Global Virtual Store • Many Concurrent Transactions – High Bandwidth • Superior Basis for Security / Privacy 3

4. The Fresh Breeze Memory Model Fan-out as large as 16 • Cycle-Free Heap Arrays as Trees of Chunks Master Chunk Data Chunks e.g. 128 Bytes • Arrays: Three levels yields 4096 elements (longs) • Write-Once then Read Only 4

5. Fresh Breeze System Vision P P P P P P P P P P P P L1 L1 L1 L1 L1 L1 L1 L1 L1 L1 L1 L1 L2 Cache L2 Cache L2 Cache AD AC AD AC AD AC AC AC DRAM Flash DRAM Flash Flash DRAM Flash Flash Many Core Processing Chips Switch Switch Main Memory: Associative Directories and DRAM Chips Switch Backing Store: Access Controllers and Flash Devices

6. Unique Features of the Processor Cache Lines are 128-byte Chunks. Registers are tagged to indicate those holding handles of chunks. Hardware task scheduler: Active Task List and Pending Task Queue. 6

7. Spawn and Join Master Task Spawned Worker Tasks spawn (n) Worker 0 Worker n-1 join_update join_update Join_fetch Join Continuation Task 7

8. * * * * + The Dot Product A Sum B A B 5 levels: Vector length = 165 = 1,048,576 Each Leaf Task: Dot Product of two 16-element vectors: 16 multiplies; 15 adds scalar result

9. Linear Algebra: Three Algorithms Dot Product Matrix Multiply Fast Fourier Transform Let’s consider the special characteristics of each. 9

10. Dot Product 16 Leaf Task: Dot Product of 16-element segments A and B Segment A Multiplies 15 Adds + * 31 Operations Segment B • No data reuse • No intermediate data • No chunks written • Large volume of input data 10

11. Matrix Multiply 64 Multiplies Adds Operations Leaf Task: Product of two 4-by-4 matrices 16 dot products of four-element vectors 48 + * 112 • Each input chunk used many times • Result chunks written to memory • No chunks written • Relatively small input data 11

12. Fast Fourier Transform 4 BFLY BFLY BFLY BFLY Leaf Task: Group of Four Butterfly Computations • Log2 (n) stages • Intermediate data • Chunks written and read Eight Data Samples Eight Results Four Twiddle Factors One Butterfly Four Butterflies Multiplies 16 Adds 6 24 Operations 10 40

13. Simulated System Model Memory Processors – 16, 24, 32, 40 P L1 Switch Up to 64 Independent Storage Units P L1

14. Simulation Modes Non-Blocking: A task executing a read simply waits for operation to complete. Models an L2 cache with short access time. Blocking: A task executing a read suspends to permit other tasks to run. Models a main memory with long access time.

15. Estimated PerformanceL2 Cache Model - NonBlocking Dot ProductA: 163B: 164C: 165 Matrix MultiplyA: 16 x 16B: 32 x 32C: 48 x 48 Fourier TransformA: 1024B: 2048C: 4096

16. Estimated PerformanceL2 Cache Model - NonBlocking Dot ProductA: 163B: 164C: 165 Matrix MultiplyA: 16 x 16B: 32 x 32C: 48 x 48 Fourier TransforA: 1024B: 2048C: 4096

17. Findings Data locality of chunks works well. L1 Cache is not very important; only a buffer for input and result chunks of tasks. Task switching for chunk reads is costly; percolation would make a big difference. Percolation: Ensuring that input chunks of a task have been retrieved before the task is scheduled. 17

18. Further Work Model a Three-Level or Four-Level Storage System Develop Hierarchical Work Stealing to improve Load Distribution for Massively Parallel Systems. Compiler Development: Automatic Mapping of Objects to Trees of Chunks. Expand Test Programs to include Transaction Processing and Database Operations . 18

19. Distributed Discrete-Event Simulation Accurate timing for interconnected communicating components. Packet Communication Architecture is a model for distributed systems especially amenable to efficient distributed simulation. UDel and MIT have begun a project to develop a PCA-based Simulation Sandbox for evaluating alternate PXMs for massively parallel computing. 19

20. Relevance to FSIO The Fresh Breeze Memory Model extends to 264 chunks, about 1021 chunks or 100,000 exabytes. The tree-structure is an excellent basis for advanced security and data object sharing. Can simplify check-point / restart. Seamless transition from “in-core” to “out-of-core” operation of arbitrary parallel programs. Provides ability to use any program as a component of new programs – including any parallel program. 20

21. Conclusion Making the best of many-core technology requires study of new program execution models (PXMs). The FSIO challenge can be met by integrating the file system into the system memory hierarchy as a global virtual store. The Fresh BreezePXM demonstrates some of the benefits from departing from conventional system organization. More exploration of new PXMs is needed! 21