1 / 47

18-447: Computer Architecture Lecture 27: Multi-Core Potpourri

This lecture covers the MESI cache coherence protocol and provides information on the final exam preparation, including review materials and a review session. It also discusses the challenges and tradeoffs in designing multi-core systems.

mgibson
Download Presentation

18-447: Computer Architecture Lecture 27: Multi-Core Potpourri

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 18-447: Computer ArchitectureLecture 27: Multi-Core Potpourri Prof. Onur Mutlu Carnegie Mellon University Spring 2012, 5/2/2012

  2. Labs 6 and 7 • Lab 7 • MESI cache coherence protocol (extra credit: better protocol) • Due May 4 • You can use 2 additional days without any penalty • No additional days at all after May 6 • Lab 6 • Binary for golden solution released • You can debug your lab • Extended deadline: • Same due date as Lab 7, but 20% penalty • We’ll multiply your grade by 0.8 if you turn in by the new due date • No late Lab 6’s accepted after May 6

  3. Lab 6 Grades

  4. Lab 6 Honors • Extra credit • Jason Lin • Stride prefetcher for D-cache misses, next-line prefetcher for I-cache misses • Full credit • Eric Brunstad • Jason Lin • Justin Wagner • Rui Cai • Tyler Huberty

  5. Final Exam • May 10 • Comprehensive (over alltopics in course) • Three cheat sheets allowed • We will have a review session • Remember this is 30% of your grade • I will take into account your improvement over the course • Know the previous midterm concepts by heart

  6. Final Exam Preparation • Homework 7 • For your benefit • Past Exams • This semester • And, relevant questions from the exams on the course website • Review Session

  7. A Note on 742, Research, Jobs • I am teaching Parallel Computer Architecture next semester (Fall 2012) • Deep dive into many topics we covered • And, many topics we did not cover • Systolic arrays, speculative parallelization, nonvolatile memories, deep dataflow, more multithreading, … • Research oriented with an open-ended research project • Cutting edge research and topics in HW/SW interface • If you enjoy 447 and do well in class, you can take it  talk with me • If you are excited about Computer Architecture research or looking for a job in this area  talk with me

  8. Course Evaluations • Please do not forget to fill out the course evaluations • Your feedback is very important • I read these very carefully, and take into account every piece of feedback • And, improve the course for the future • Please take the time to write out feedback • State the things you liked, topics you enjoyed, and what we can improve on  both the good and the not-so-good • Due May 15

  9. Last Lecture • Wrap up cache coherence • VI  MSI  MESI  MOESI  ? • Directory vs. snooping tradeoffs • Interconnects • Why important? • Topologies • Handling contention

  10. Today • Interconnection networks wrap-up • Handling serial and parallel bottlenecks better • Caching in multi-core systems

  11. Interconnect Basics

  12. Handling Contention in A Switch • Two packets trying to use the same link at the same time • What do you do? • Buffer one • Drop one • Misroute one (deflection) • Tradeoffs?

  13. Multi-Core Design

  14. Many Cores on Chip • Simpler and lower power than a single large core • Large scale parallelism on chip Tilera TILE Gx 100 cores, networked Intel Core i78 cores IBM Cell BE8+1 cores AMD Barcelona 4 cores IBM POWER7 8 cores Intel SCC 48 cores, networked Sun Niagara II 8 cores Nvidia Fermi 448 “cores”

  15. With Many Cores on Chip • What we want: • N times the performance with N times the cores when we parallelize an application on N cores • What we get: • Amdahl’s Law (serial bottleneck)

  16. Caveats of Parallelism • Amdahl’s Law • f: Parallelizable fraction of a program • N: Number of processors • Amdahl, “Validity of the single processor approach to achieving large scale computing capabilities,” AFIPS 1967. • Maximum speedup limited by serial portion: Serial bottleneck • Parallel portion is usually not perfectly parallel • Synchronization overhead (e.g., updates to shared data) • Load imbalance overhead (imperfect parallelization) • Resource sharing overhead (contention among N processors) 1 Speedup = f + 1 - f N

  17. Demands in Different Code Sections • What we want: • In a serial code section  one powerful “large” core • In a parallel code section  many wimpy “small” cores • These two conflict with each other: • If you have a single powerful core, you cannot have many cores • A small core is much more energy and area efficient than a large core

  18. “Large” vs. “Small” Cores LargeCore SmallCore • In-order • Narrow Fetch e.g. 2-wide • Shallow pipeline • Simple branch predictor (e.g. Gshare) • Few functional units • Out-of-order • Wide fetch e.g. 4-wide • Deeper pipeline • Aggressive branch predictor (e.g. hybrid) • Multiple functional units • Trace cache • Memory dependence speculation Large Cores are power inefficient:e.g., 2x performance for 4x area (power)

  19. Large vs. Small Cores • Grochowski et al., “Best of both Latency and Throughput,” ICCD 2004.

  20. Meet Large: IBM POWER4 • Tendler et al., “POWER4 system microarchitecture,” IBM J R&D, 2002. • Another symmetric multi-core chip… • But, fewer and more powerful cores

  21. IBM POWER4 • 2 cores, out-of-order execution • 100-entry instruction window in each core • 8-wide instruction fetch, issue, execute • Large, local+global hybrid branch predictor • 1.5MB, 8-way L2 cache • Aggressive stream based prefetching

  22. IBM POWER5 • Kalla et al., “IBM Power5 Chip: A Dual-Core Multithreaded Processor,” IEEE Micro 2004.

  23. Meet Small: Sun Niagara (UltraSPARC T1) • Kongetira et al., “Niagara: A 32-Way Multithreaded SPARC Processor,” IEEE Micro 2005.

  24. Niagara Core • 4-way fine-grain multithreaded, 6-stage, dual-issue in-order • Round robin thread selection (unless cache miss) • Shared FP unit among cores

  25. Remember the Demands • What we want: • In a serial code section  one powerful “large” core • In a parallel code section  many wimpy “small” cores • These two conflict with each other: • If you have a single powerful core, you cannot have many cores • A small core is much more energy and area efficient than a large core • Can we get the best of both worlds?

  26. Performance vs. Parallelism • Assumptions: • 1. Small cores takes an area budget of 1 and has performance of 1 • 2. Large core takes an area budget of 4 and has performance of 2

  27. Large core Largecore Largecore Largecore “Tile-Large” Tile-Large Approach • Tile a few large cores • IBM Power 5, AMD Barcelona, Intel Core2Quad, Intel Nehalem + High performance on single thread, serial code sections (2 units) - Low throughput on parallel program portions (8 units)

  28. “Tile-Small” Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Tile-Small Approach • Tile many small cores • Sun Niagara, Intel Larrabee, Tilera TILE (tile ultra-small) + High throughput on the parallel part (16 units) - Low performance on the serial part, single thread (1 unit)

  29. Can we get the best of both worlds? • Tile Large + High performance on single thread, serial code sections (2 units) - Low throughput on parallel program portions (8 units) • Tile Small + High throughput on the parallel part (16 units) - Low performance on the serial part, single thread (1 unit), reduced single-thread performance compared to existing single thread processors • Idea: Have both large and small on the same chip  Performance asymmetry

  30. Large core Largecore Large core Largecore Largecore ACMP “Tile-Large” “Tile-Small” Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Asymmetric Chip Multiprocessor (ACMP) • Provide one large core and many small cores + Accelerate serial part using the large core (2 units) + Execute parallel part on small cores and large core for high throughput (12+2 units)

  31. Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Accelerating Serial Bottlenecks Single thread  Large core Large core ACMP Approach

  32. Performance vs. Parallelism • Assumptions: • 1. Small cores takes an area budget of 1 and has performance of 1 • 2. Large core takes an area budget of 4 and has performance of 2

  33. Large core Largecore Large core Largecore Largecore ACMP “Tile-Large” “Tile-Small” Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore Smallcore ACMP Performance vs. Parallelism Area-budget = 16 small cores 33

  34. Caveats of Parallelism, Revisited • Amdahl’s Law • f: Parallelizable fraction of a program • N: Number of processors • Amdahl, “Validity of the single processor approach to achieving large scale computing capabilities,” AFIPS 1967. • Maximum speedup limited by serial portion: Serial bottleneck • Parallel portion is usually not perfectly parallel • Synchronization overhead (e.g., updates to shared data) • Load imbalance overhead (imperfect parallelization) • Resource sharing overhead (contention among N processors) 1 Speedup = f + 1 - f N

  35. Accelerating Parallel Bottlenecks • Serialized or imbalanced execution in the parallel portion can also benefit from a large core • Examples: • Critical sections that are contended • Parallel stages that take longer than others to execute • Idea: Identify these code portions that cause serialization and execute them on a large core

  36. An Example: Accelerated Critical Sections • Problem: Synchronization and parallelization is difficult for programmers  Critical sections are a performance bottleneck • Idea: HW/SW ships critical sections to a large, powerful core in an asymmetric multi-core architecture • Benefit: • Reduces serialization due to contended locks • Reduces the performance impact of hard-to-parallelize sections • Programmer does not need to (heavily) optimize parallel code  fewer bugs, improved productivity • Suleman et al., “Accelerating Critical Section Execution with Asymmetric Multi-Core Architectures,” ASPLOS 2009, IEEE Micro Top Picks 2010. • Suleman et al., “Data Marshaling for Multi-Core Architectures,” ISCA 2010, IEEE Micro Top Picks 2011.

  37. t1 t2 t3 t4 t5 t6 t7 Contention for Critical Sections Critical Section Parallel Thread 1 Thread 2 Thread 3 Thread 4 Idle Accelerating critical sections not only helps the thread executing the critical sections, but also the waiting threads t1 t2 t3 t4 t5 t6 t7 Thread 1 Thread 2 Thread 3 Thread 4 Critical Sections execute 2x faster

  38. Impact of Critical Sections on Scalability • Contention for critical sections increases with the number of threads and limits scalability LOCK_openAcquire() foreach (table locked by thread) table.lockrelease() table.filerelease() if (table.temporary) table.close() LOCK_openRelease() Speedup Chip Area (cores) MySQL (oltp-1)

  39. Accelerated Critical Sections 1. P2 encounters a critical section (CSCALL) 2. P2 sends CSCALL Request to CSRB 3. P1 executes Critical Section 4. P1 sends CSDONE signal • EnterCS() • PriorityQ.insert(…) • LeaveCS() P1 Core executing critical section P2 P3 P4 Critical SectionRequest Buffer (CSRB) Onchip-Interconnect

  40. Accelerated Critical Sections (ACS) • Suleman et al., “Accelerating Critical Section Execution with Asymmetric Multi-Core Architectures,” ASPLOS 2009. Small Core Small Core Large Core A = compute() PUSH A CSCALL X, Target PC A = compute() LOCK X result = CS(A) UNLOCK X print result CSCALL Request … … … Waiting in Critical Section Request Buffer (CSRB) … … … Send X, TPC, STACK_PTR, CORE_ID … … … … TPC: Acquire X POP A result = CS(A) PUSH result Release X CSRET X CSDONEResponse POP result print result

  41. Large core Large core ACS ACMP SCMP Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore ACS Comparison Points • One large core (area-equal 4 small cores) • Conventional locking • ACMP with a CSRB • Accelerates Critical Sections • All small cores • Conventional locking

  42. ACS Performance Chip Area = 32 small coresSCMP = 32 small coresACMP = 1 large and 28 small cores Equal-area comparisonNumber of threads = Best threads 269 180 185 Coarse-grain locks Fine-grain locks

  43. ACS Performance Tradeoffs • Fewer threads vs. accelerated critical sections • Accelerating critical sections offsets loss in throughput • As the number of cores (threads) on chip increase: • Fractional loss in parallel performance decreases • Increased contention for critical sections makes acceleration more beneficial • Overhead of CSCALL/CSDONE vs. better lock locality • ACS avoids “ping-ponging” of locks among caches by keeping them at the large core • More cache misses for private data vs. fewer misses for shared data

  44. Cache misses for private data PriorityHeap.insert(NewSubProblems) Shared Data: The priority heap Private Data: NewSubProblems Puzzle Benchmark

  45. ACS Performance Tradeoffs • Fewer threads vs. accelerated critical sections • Accelerating critical sections offsets loss in throughput • As the number of cores (threads) on chip increase: • Fractional loss in parallel performance decreases • Increased contention for critical sections makes acceleration more beneficial • Overhead of CSCALL/CSDONE vs. better lock locality • ACS avoids “ping-ponging” of locks among caches by keeping them at the large core • More cache misses for private data vs. fewer misses for shared data • Cache misses reduce if shared data > private data

  46. Large core Large core ACS ACMP SCMP Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore Niagara-likecore ACS Comparison Points • One large core (area-equal 4 small cores) • Conventional locking • ACMP with a CSRB • Accelerates Critical Sections • All small cores • Conventional locking

  47. ------ SCMP ------ ACMP ------ ACS Equal-Area Comparisons Number of threads = No. of cores (a) ep (b) is (c) pagemine (d) puzzle (e) qsort (f) tsp Speedup over a small core (g) sqlite (h) iplookup (i) oltp-1 (i) oltp-2 (k) specjbb (l) webcache Chip Area (small cores)

More Related