1 / 67

Amdahl’s Law in the Multicore Era

Amdahl’s Law in the Multicore Era. Mark D. Hill and Michael R. Marty University of Wisconsin—Madison January 2010 @ Other UW. IBM’s Dr. Thomas Puzak: Everyone knows Amdahl’s Law But quickly forgets it!. HPCA 2007 Debate [IEEE Micro 11-12/2007]. Single-Threaded vs. Multithreaded:

xiang
Download Presentation

Amdahl’s Law in the Multicore Era

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Amdahl’s Law in the Multicore Era Mark D. Hill and Michael R. Marty University of Wisconsin—Madison January 2010 @ Other UW IBM’s Dr. Thomas Puzak: Everyone knows Amdahl’s Law But quickly forgets it!

  2. HPCA 2007 Debate [IEEE Micro 11-12/2007] Single-Threaded vs. Multithreaded: Where Should We Focus? Yale Patt vs. Mark Hill w/ Joel Emer, moderator

  3. Executive Summary • Develop A Corollary to Amdahl’s Law • Simple Model of Multicore Hardware • Complements Amdahl’s software model • Fixed chip resources for cores • Core performance improves sub-linearly with resources • Research Implications (1) Need Dramatic Increases in Parallelism (No Surprise) • 99% parallel limits 256 (base) cores to speedup 72 • New Moore’s Law: Double Parallelism Every Two Years? (2) Many larger chips need increased core performance (3) HW/SW for asymmetric designs (one/few cores enhanced) (4) HW/SW for dynamic designs (serial  parallel) Wisconsin Multifacet Project

  4. Outline Multicore Motivation & Research Paper Trends Recall Amdahl’s Law A Model of Multicore Hardware Symmetric Multicore Chips Asymmetric Multicore Chips Dynamic Multicore Chips Caveats & Wrap Up 1/4/10 5 Wisconsin Multifacet Project

  5. Transistor1947 Integrated Circuit 1958 (a.k.a. Chip) Technology & Moore’s Law Moore’s Law 1964: # Transistors per Chip doubles every two years (or 18 months)

  6. 50M transistors ~2000  Architects & Another Moore’s Law Microprocessor 1971 Popular Moore’s Law: Processor (core) performance doubles every two years

  7. Multicore Chip (a.k.a. Chip Multiprocesors) Why Multicore? • Power  simpler structures • Memory  Concurrent accessesto tolerate off-chip latency • Wires  intra-core wires shorter • Complexity  divide & conquer But More cores; NOT faster cores Will effective chip performance keep doubling every two years? Eight 4-way cores 2006

  8. Slowerprograms Larger, morefeature-fullsoftware Largerdevelopmentteams Higher-levellanguages &abstractions Virtuous Cycle, circa 1950 – 2005 (per Larus) Increasedprocessor performance World-Wide Software Market (per IDC): $212b (2005)  $310b (2010) Wisconsin Multifacet Project

  9. Virtuous Cycle, 2005 – ??? World-Wide Software Market $212b (2005)  ? Slowerprograms Larger, morefeature-fullsoftware Largerdevelopmentteams Higher-levellanguages &abstractions X Increasedprocessor performance GAME OVER — NEXT LEVEL? Thread Level Parallelism & Multicore Chips 1/4/10 10 Wisconsin Multifacet Project

  10. How has Architecture Research Prepared? Lead up toMulticore What Next? SMP Bulge Percent Multiprocessor Papers in ISCA Source: Hill & Rajwar, The Rise & Fall of Multiprocessor Papers in ISCA, http://www.cs.wisc.edu/~markhill/mp2001.html (3/2001) 1/4/10 11 Wisconsin Multifacet Project

  11. How has Architecture Research Prepared? Reacted? Will Architecture Research Overreact? Multicore Ramp Percent Multiprocessor Papers in ISCA 2009 2008 Source: Hill, 2/2008 1/4/10 12 Wisconsin Multifacet Project

  12. What About PL/Compilers (PLDI) Research? What Next? Lead up toMulticore Percent Multiprocessor Papers Gentle Multicore Ramp End of Small SMP Bulge? PLDI Begins Source: Steve Jackson, 3/2008 1/4/10 14 Wisconsin Multifacet Project

  13. What About Systems (SOSP/OSDI) Research? Lead up toMulticore What Next? Percent Multiprocessor Papers Small SMP Bulge NO MulticoreRamp (Yet)  SOSP odd years only   ODSI even & SOSP odd  Source: Michael Swift, 3/2008 1/4/10 15 Wisconsin Multifacet Project

  14. Outline • Multicore Motivation & Research Paper Trends • Recall Amdahl’s Law • A Model of Multicore Hardware • Symmetric Multicore Chips • Asymmetric Multicore Chips • Dynamic Multicore Chips • Caveats & Wrap Up Wisconsin Multifacet Project

  15. Recall Amdahl’s Law • Begins with Simple Software Assumption (Limit Arg.) • Fraction F of execution time perfectly parallelizable • No Overhead for • Scheduling • Communication • Synchronization, etc. • Fraction1 – F Completely Serial • Time on 1 core = (1 – F) / 1 + F / 1 = 1 • Time on N cores = (1 – F) / 1 + F / N Wisconsin Multifacet Project

  16. 1 Amdahl’s Speedup = F 1 - F + 1 N Recall Amdahl’s Law [1967] • For mainframes, Amdahl expected 1 - F = 35% • For a 4-processor speedup = 2 • For infinite-processor speedup < 3 • Therefore, stay with mainframes with one/few processors • Amdahl’s Law applied to Minicomputer to PC Eras • What about the Multicore Era? Wisconsin Multifacet Project

  17. Designing Multicore Chips Hard • Designers must confront single-core design options • Instruction fetch, wakeup, select • Execution unit configuation & operand bypass • Load/queue(s) & data cache • Checkpoint, log, runahead, commit. • As well as additional design degrees of freedom • How many cores? How big each? • Shared caches: levels? How many banks? • Memory interface: How many banks? • On-chip interconnect: bus, switched, ordered? Wisconsin Multifacet Project

  18. Want Simple Multicore Hardware Model To Complement Amdahl’s Simple Software Model (1) Chip Hardware Roughly Partitioned into • Multiple Cores (with L1 caches) • The Rest (L2/L3 cache banks, interconnect, pads, etc.) • Changing Core Size/Number does NOT change The Rest (2) Resources for Multiple Cores Bounded • Bound of N resources per chip for cores • Due to area, power, cost ($$$), or multiple factors • Bound = Power? (but our pictures use Area) Wisconsin Multifacet Project

  19. Want Simple Multicore Hardware Model, cont. (3) Micro-architects can improve single-core performance using more of the bounded resource • A Simple Base Core • Consumes 1 Base Core Equivalent (BCE) resources • Provides performance normalized to 1 • An Enhanced Core (in same process generation) • Consumes R BCEs • Performance as a function Perf(R) • What does function Perf(R) look like? Wisconsin Multifacet Project

  20. More on Enhanced Cores • (Performance Perf(R) consuming R BCEs resources) • If Perf(R) > R Always enhance core • Cost-effectively speedups both sequential & parallel • Therefore, Equations Assume Perf(R) < R • Graphs Assume Perf(R) = Square Root of R • 2x performance for 4 BCEs, 3x for 9 BCEs, etc. • Why? Models diminishing returns with “no coefficients” • Alpha EV4/5/6 [Kumar 11/2005] & Intel’s Pollack’s Law • How to speedup enhanced core? • <Insert favorite or TBD micro-architectural ideas here> Wisconsin Multifacet Project

  21. Outline • Multicore Motivation & Research Paper Trends • Recall Amdahl’s Law • A Model of Multicore Hardware • Symmetric Multicore Chips • Asymmetric Multicore Chips • Dynamic Multicore Chips • Caveats & Wrap Up Wisconsin Multifacet Project

  22. How Many (Symmetric) Cores per Chip? • Each Chip Bounded to N BCEs (for all cores) • Each Core consumes R BCEs • Assume Symmetric Multicore = All Cores Identical • Therefore, N/R Cores per Chip —(N/R)*R = N • For an N = 16 BCE Chip: Sixteen 1-BCE cores Four 4-BCE cores One 16-BCE core Wisconsin Multifacet Project

  23. 1 Symmetric Speedup = F * R 1 - F + Perf(R) Perf(R)*N Enhanced Cores speed Serial & Parallel Performance of Symmetric Multicore Chips • Serial Fraction 1-F uses 1 core at rate Perf(R) • Serial time = (1 – F) / Perf(R) • Parallel Fraction uses N/R cores at rate Perf(R) each • Parallel time = F / (Perf(R) * (N/R)) = F*R / Perf(R)*N • Therefore, w.r.t. one base core: • Implications? Wisconsin Multifacet Project

  24. Symmetric Multicore Chip, N = 16 BCEs F=0.5, Opt. Speedup S = 4 = 1/(0.5/4 + 0.5*16/(4*16)) Need to increase parallelism to make multicore optimal! F=0.5 R=16, Cores=1, Speedup=4 (16 cores) (8 cores) (2 cores) (1 core) (4 cores) Wisconsin Multifacet Project

  25. Symmetric Multicore Chip, N = 16 BCEs At F=0.9, Multicore optimal, but speedup limited Need to obtain even more parallelism! F=0.9, R=2, Cores=8, Speedup=6.7 F=0.5 R=16, Cores=1, Speedup=4 Wisconsin Multifacet Project

  26. Symmetric Multicore Chip, N = 16 BCEs F1, R=1, Cores=16, Speedup16 F matters: Amdahl’s Law applies to multicore chips MANY Researchers should target parallelism F first Wisconsin Multifacet Project

  27. Need a Third “Moore’s Law?” Technologist’s Moore’s Law Double Transistors per Chip every 2 years Slows or stops: TBD Microarchitect’s Moore’s Law Double Performance per Core every 2 years Slowed or stopped: Early 2000s Multicore’s Moore’s Law Double Cores per Chip every 2 years & Double Parallelism per Workload every 2 years & Aided by Architectural Support for Parallelism = Double Performance per Chip every 2 years Starting now Software as Producer, not Consumer, of Performance Gains! 1/4/10 29 Wisconsin Multifacet Project

  28. Symmetric Multicore Chip, N = 16 BCEs Recall F=0.9, R=2, Cores=8, Speedup=6.7 As Moore’s Law enables N to go from 16 to 256 BCEs, More cores? Enhance cores? Or both? Wisconsin Multifacet Project

  29. Symmetric Multicore Chip, N = 256 BCEs As Moore’s Law increases N, often need enhanced core designs Some arch. researchers should target single-core performance F1 R=1 (vs. 1) Cores=256 (vs. 16) Speedup=204 (vs. 16) MORE CORES! F=0.99 R=3 (vs. 1) Cores=85 (vs. 16) Speedup=80 (vs. 13.9) MORE CORES& ENHANCE CORES! F=0.9 R=28 (vs. 2) Cores=9 (vs. 8) Speedup=26.7 (vs. 6.7) ENHANCE CORES! Wisconsin Multifacet Project

  30. Aside: Cost-Effective Parallel Computing • Isn’t Speedup(C) < C Inefficient? (C = #cores) • Much of a Computer’s Cost OUTSIDE Processor [Wood & Hill, IEEE Computer 2/1995] • Let Costup(C) = Cost(C)/Cost(1) • Parallel Computing Cost-Effective: • Speedup(C) > Costup(C) • 1995 SGI PowerChallenge w/ 500MB: • Costup(32) = 8.6 Cores Multicores haveeven lower Costups!!!

  31. How Might Servers/Clients/Embedded Evolve? Recall 1970s Watergate Secret Source Deep Throat (W. Mark Felt @ FBI) Helped Reporters Bob Woodward & Carl Bernstein Confirmed, but would not provide information Frequently recommended: Follow the Money Today I recommend: Follow the Parallelism! Where Parallelism Helps Performance Where Parallelism Helps Cost-Performance Servers can use vast parallelism Can clients & embedded? If not, computing’s center of gravity  server cloud 1/4/10 34 Wisconsin Multifacet Project

  32. Outline • Multicore Motivation & Research Paper Trends • Recall Amdahl’s Law • A Model of Multicore Hardware • Symmetric Multicore Chips • Asymmetric Multicore Chips • Dynamic Multicore Chips • Caveats & Wrap Up Wisconsin Multifacet Project

  33. Asymmetric (Heterogeneous) Multicore Chips • Symmetric Multicore Required All Cores Equal • Why Not Enhance Some (But Not All) Cores? • For Amdahl’s Simple Software Assumptions • One Enhanced Core • Others are Base Cores • How? • <fill in favorite micro-architecture techniques here> • Model ignores design cost of asymmetric design • How does this effect our hardware model? Wisconsin Multifacet Project

  34. How Many Cores per Asymmetric Chip? • Each Chip Bounded to N BCEs (for all cores) • One R-BCE Core leaves N-R BCEs • Use N-R BCEs for N-R Base Cores • Therefore, 1 + N - R Cores per Chip • For an N = 16 BCE Chip: Asymmetric:One 4-BCE core & Twelve 1-BCE base cores Symmetric: Four 4-BCE cores Wisconsin Multifacet Project

  35. 1 Asymmetric Speedup = F 1 - F + Perf(R) Perf(R) + N - R Performance of Asymmetric Multicore Chips • Serial Fraction 1-F same, so time = (1 – F) / Perf(R) • Parallel Fraction F • One core at rate Perf(R) • N-R cores at rate 1 • Parallel time = F / (Perf(R) + N - R) • Therefore, w.r.t. one base core: Wisconsin Multifacet Project

  36. Asymmetric Multicore Chip, N = 256 BCEs Number of Cores = 1 (Enhanced) + 256 – R (Base) How do Asymmetric & Symmetric speedups compare? (256 cores) (1+252 cores) (1+192 cores) (1 core) (1+240 cores) 1/4/10 39 Wisconsin Multifacet Project

  37. Recall Symmetric Multicore Chip, N = 256 BCEs Recall F=0.9, R=28, Cores=9, Speedup=26.7 Wisconsin Multifacet Project

  38. Asymmetric Multicore Chip, N = 256 BCEs Asymmetric offers greater speedups potential than Symmetric In Paper: As Moore’s Law increases N, Asymmetric gets better Some arch. researchers should target asymmetric multicores F=0.99 R=41 (vs. 3) Cores=216 (vs. 85) Speedup=166 (vs. 80) F=0.9 R=118 (vs. 28) Cores= 139 (vs. 9) Speedup=65.6 (vs. 26.7) Wisconsin Multifacet Project

  39. Asymmetric Multicore: 3 Software Issues • Schedule computation (e.g., when to use bigger core) • Manage locality (e.g., sending code or data can sap gains) • Synchronize (e.g., asymmetric cores reaching a barrier) At What Level? • Application Programmer • Library Author • Compiler • Runtime System • Operating System • Hypervisor (Virtual Machine Monitor) • Hardware More Leverage (?) More Info (?)

  40. Outline • Multicore Motivation & Research Paper Trends • Recall Amdahl’s Law • A Model of Multicore Hardware • Symmetric Multicore Chips • Asymmetric Multicore Chips • Dynamic Multicore Chips • Caveats & Wrap Up Wisconsin Multifacet Project

  41. Dynamic Multicore Chips, Take 1 • Why NOT Have Your Cake and Eat It Too? • N Base Cores for Best Parallel Performance • Harness R Cores Together for Serial Performance • How? DYNAMICALLY Harness Cores Together • <insert favorite or TBD techniques here> parallel mode sequential mode Wisconsin Multifacet Project

  42. Dynamic Multicore Chips, Take 2 Let POWER provide the limit of N BCEs While Area is Unconstrained (to first order) Result: N base cores for parallel; large core for serial [Chakraborty, Wells, & Sohi, Wisconsin CS-TR-2007-1607] When Simultaneous Active Fraction (SAF) < ½ parallel mode How to model these two chips? sequential mode 45 1/4/10 Wisconsin Multifacet Project

  43. 1 Dynamic Speedup = F 1 - F + Perf(R) N Performance of Dynamic Multicore Chips • N Base Cores with R BCEs used Serially • Serial Fraction 1-F uses R BCEs at rate Perf(R) • Serial time = (1 – F) / Perf(R) • Parallel Fraction F uses N base cores at rate 1 each • Parallel time = F / N • Therefore, w.r.t. one base core: Wisconsin Multifacet Project

  44. Recall Asymmetric Multicore Chip, N = 256 BCEs Recall F=0.99 R=41 Cores=216 Speedup=166 What happens with a dynamic chip? Wisconsin Multifacet Project

  45. Dynamic Multicore Chip, N = 256 BCEs Dynamic offers greater speedup potential than Asymmetric Arch. researchers should target dynamically harnessing cores F=0.99 R=256 (vs. 41) Cores=256 (vs. 216) Speedup=223 (vs. 166) Wisconsin Multifacet Project

  46. Dynamic Asymmetric Multicore: 3 Software Issues • Schedule computation (e.g., when to use bigger core) • Manage locality (e.g., sending code or data can sap gains) • Synchronize (e.g., asymmetric cores reaching a barrier) At What Level? • Application Programmer • Library Author • Compiler • Runtime System • Operating System • Hypervisor (Virtual Machine Monitor) • Hardware More Leverage (?) More Info (?) Dynamic Challenges > Asymmetric Ones Dynamic chips due to power likely

  47. Outline • Multicore Motivation & Research Paper Trends • Recall Amdahl’s Law • A Model of Multicore Hardware • Symmetric Multicore Chips • Asymmetric Multicore Chips • Dynamic Multicore Chips • Caveats & Wrap Up Wisconsin Multifacet Project

  48. 1 1 1 Symmetric Speedup = Dynamic Speedup = Asymmetric Speedup = F * R F F 1 - F 1 - F 1 - F + + + Perf(R) Perf(R) Perf(R) Perf(R)*N N Perf(R) + N - R Sequential Section 1 Enhanced Core Three Multicore Amdahl’s Law Parallel Section N/R Enhanced Cores 1 Enhanced & N-R Base Cores N Base Cores Wisconsin Multifacet Project

  49. Amdahl’s Software Model Charges Serial (parallel) fraction not totally serial (parallel) Can extend to model to tree algorithms (bounded parallelism) Serial/Parallel fraction changes Weak Scaling [Gustafson] Prudent architectures support Strong Scaling Synchronization, communication, scheduling effects? Can extend for overheads and imbalance Software challenges for asymmetric/dynamic worse Can extend to model overheads to facilitate Future software will be totally parallel (see “my work”) I’m skeptical; not even true for MapReduce 1/4/10 52 Wisconsin Multifacet Project

  50. Our Hardware Model Charges Naïve to bound Cores by one resource (esp. area) Can extend for Pareto optimal mix of area, dynamic/static power, complexity, reliability, … Naïve to ignore off-chip bandwidth & L2/L3 caching Can extend for modeling memory system Naïve to use performance = square root of resources Can extend as equations can use any function We architects can’t scale Perf(R) for very large R True, so what should we do about it? 1/4/10 53 Wisconsin Multifacet Project

More Related