1 / 32

Computer Architecture

Computer Architecture. Multi-cores, Multiprocessors and Clusters. Introduction. Goal: connecting multiple computers to get higher performance Multiprocessors Scalability, availability, power efficiency Job-level (process-level) parallelism High throughput for independent jobs

jarvis
Download Presentation

Computer Architecture

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computer Architecture Multi-cores, Multiprocessors and Clusters

  2. Introduction • Goal: connecting multiple computersto get higher performance • Multiprocessors • Scalability, availability, power efficiency • Job-level (process-level) parallelism • High throughput for independent jobs • Parallel processing program • Single program run on multiple processors • Multicore microprocessors • Chips with multiple processors (cores)

  3. Hardware and Software • Hardware • Serial: e.g., Pentium 4 • Parallel: e.g., quad-core Xeon e5345 • Software • Sequential: e.g., matrix multiplication • Concurrent: e.g., operating system • Sequential/concurrent software can run on serial/parallel hardware • Challenge: making effective use of parallel hardware

  4. Parallel Programming • Parallel software is the problem • Need to get significant performance improvement • Otherwise, just use a faster uniprocessor, since it’s easier! • Difficulties • Scheduling • Load Balancing • Synchronization • Communications overhead

  5. Amdahl’s Law • Sequential part can limit speedup • Example: 100 processors, 90× speedup? • Tnew = Tparallelizable/100 + Tsequential • Solving: Fparallelizable = 0.999 • Need sequential part to be 0.1% of original time

  6. General context: Multiprocessors • Multiprocessor is any computer with several processors • SIMD • Single instruction, multiple data • Modern graphics cards • MIMD • Multiple instructions, multiple data

  7. Instruction and Data Streams • SPMD: Single Program Multiple Data • A parallel program on a MIMD computer • Conditional code for different processors

  8. Shared Memory • SMP: shared memory multiprocessor • Hardware provides single physicaladdress space for all processors • Synchronize shared variables using locks • Memory access time • UMA (uniform) vs. NUMA (nonuniform)

  9. Message Passing • Each processor has private physical address space • Hardware sends/receives messages between processors

  10. Loosely Coupled Clusters • Network of independent computers • Each has private memory and OS • Connected using I/O system • E.g., Ethernet/switch, Internet • Suitable for applications with independent tasks • Web servers, databases, simulations, … • High availability, scalable, affordable • Problems • Administration cost (prefer virtual machines) • Low interconnect bandwidth • c.f. processor/memory bandwidth on an SMP

  11. Single-core CPU chip the single core

  12. Multi-core architectures • Replicate multiple processor cores on a single die. Core 1 Core 2 Core 3 Core 4 Multi-core CPU chip

  13. core 1 core 2 core 3 core 4 Multi-core CPU chip • The cores fit on a single processor socket • Also called CMP (Chip Multi-Processor)

  14. The cores run in parallel thread 1 thread 2 thread 3 thread 4 core 1 core 2 core 3 core 4

  15. Within each core, threads are time-sliced (just like on a uniprocessor) several threads several threads several threads several threads core 1 core 2 core 3 core 4

  16. Interaction with the Operating System • OS perceives each core as a separate processor • OS scheduler maps threads/processes to different cores • Most major OS support multi-core today:Windows, Linux, Mac OS X, …

  17. Why multi-core ? • Difficult to make single-coreclock frequencies even higher • Deeply pipelined circuits: • heat problems • difficult design and verification • large design teams necessary • server farms need expensiveair-conditioning • Many new applications are multithreaded • General trend in computer architecture (shift towards more parallelism)

  18. Instruction-level parallelism • Parallelism at the machine-instruction level • The processor can re-order, pipeline instructions, split them into microinstructions, do aggressive branch prediction, etc. • Instruction-level parallelism enabled rapid increases in processor speeds over the last 15 years

  19. Thread-level parallelism (TLP) • This is parallelism on a more coarser scale • Server can serve each client in a separate thread (Web server, database server) • A computer game can do AI, graphics, and physics in three separate threads • Single-core superscalar processors cannot fully exploit TLP • Multi-core architectures are the next step in processor evolution: explicitly exploiting TLP

  20. Multi-core processor is a special kind of a multiprocessor:All processors are on the same chip • Multi-core processors are MIMD:Different cores execute different threads (Multiple Instructions), operating on different parts of memory (Multiple Data). • Multi-core is a shared memory multiprocessor:All cores share the same memory

  21. What applications benefit from multi-core? • Database servers • Web servers (Web commerce) • Compilers • Multimedia applications • Scientific applications • In general, applications with Thread-level parallelism(as opposed to instruction-level parallelism) Each can run on itsown core

  22. L1 D-Cache D-TLB Integer Floating Point Schedulers uop queues L2 Cache and Control Rename/Alloc BTB Trace Cache uCode ROM Decoder Bus BTB and I-TLB A technique complementary to multi-core:Simultaneous multithreading • Problem addressed:The processor pipeline can get stalled: • Waiting for the result of a long floating point (or integer) operation • Waiting for data to arrive from memory Other execution unitswait unused

  23. Simultaneous multithreading (SMT) • Permits multiple independent threads to execute SIMULTANEOUSLY on the SAME core • Weaving together multiple “threads” on the same core • Example: if one thread is waiting for a floating point operation to complete, another thread can use the integer units

  24. Without SMT, only a single thread can run at any given time L1 D-Cache D-TLB Integer Floating Point Schedulers Uop queues L2 Cache and Control Rename/Alloc BTB Trace Cache uCode ROM Decoder Bus BTB and I-TLB Thread 1: floating point

  25. Without SMT, only a single thread can run at any given time L1 D-Cache D-TLB Integer Floating Point Schedulers Uop queues L2 Cache and Control Rename/Alloc BTB Trace Cache uCode ROM Decoder Bus BTB and I-TLB Thread 2:integer operation

  26. SMT processor: both threads can run concurrently L1 D-Cache D-TLB Integer Floating Point Schedulers Uop queues L2 Cache and Control Rename/Alloc BTB Trace Cache uCode ROM Decoder Bus BTB and I-TLB Thread 2:integer operation Thread 1: floating point

  27. But: Can’t simultaneously use the same functional unit L1 D-Cache D-TLB Integer Floating Point Schedulers Uop queues L2 Cache and Control Rename/Alloc BTB Trace Cache uCode ROM Decoder This scenario isimpossible with SMTon a single core(assuming a single integer unit) Bus BTB and I-TLB Thread 1 Thread 2 IMPOSSIBLE

  28. SMT not a “true” parallel processor • Enables better threading (e.g. up to 30%) • OS and applications perceive each simultaneous thread as a separate “virtual processor” • The chip has only a single copy of each resource • Compare to multi-core:each core has its own copy of resources

  29. Multi-core: threads can run on separate cores L1 D-Cache D-TLB L1 D-Cache D-TLB Integer Floating Point Integer Floating Point Schedulers Schedulers L2 Cache and Control Uop queues Uop queues L2 Cache and Control Rename/Alloc Rename/Alloc BTB Trace Cache uCode ROM BTB Trace Cache uCode ROM Decoder Decoder Bus Bus BTB and I-TLB BTB and I-TLB Thread 2 Thread 1

  30. Multi-core: threads can run on separate cores L1 D-Cache D-TLB L1 D-Cache D-TLB Integer Floating Point Integer Floating Point Schedulers Schedulers L2 Cache and Control Uop queues Uop queues L2 Cache and Control Rename/Alloc Rename/Alloc BTB Trace Cache uCode ROM BTB Trace Cache uCode ROM Decoder Decoder Bus Bus BTB and I-TLB BTB and I-TLB Thread 4 Thread 3

  31. Combining Multi-core and SMT • Cores can be SMT-enabled (or not) • The different combinations: • Single-core, non-SMT: standard uniprocessor • Single-core, with SMT • Multi-core, non-SMT • Multi-core, with SMT: our fish machines • The number of SMT threads:2, 4, or sometimes 8 simultaneous threads • Intel calls them “hyper-threads”

  32. SMT Dual-core: all four threads can run concurrently L1 D-Cache D-TLB L1 D-Cache D-TLB Integer Floating Point Integer Floating Point Schedulers Schedulers L2 Cache and Control Uop queues Uop queues L2 Cache and Control Rename/Alloc Rename/Alloc BTB Trace Cache uCode ROM BTB Trace Cache uCode ROM Decoder Decoder Bus Bus BTB and I-TLB BTB and I-TLB Thread 1 Thread 3 Thread 2 Thread 4

More Related