1 / 16

Different Species of Parallel

Different Species of Parallel. Multiple Instructions, Data Sets. Some Common Organizations. Symmetric Multiprocessing (SMP) - multiple CPUs each with cache - common everything else Cluster and Grid Computing - multiple everything, networked together - common software control Blade Server

oki
Download Presentation

Different Species of Parallel

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Different Species of Parallel

  2. Multiple Instructions, Data Sets

  3. Some Common Organizations Symmetric Multiprocessing (SMP) - multiple CPUs each with cache - common everything else Cluster and Grid Computing - multiple everything, networked together - common software control Blade Server - multiple blades (CPU, cache, RAM, hard drive) - common everything else (network, other cards)

  4. Symmetric MultiProcessing (SMP)

  5. SMP and Cache Coherence Each processor has its own cache How do we keep them in sync? - if one changes its copy, how do others know? - one option: directory protocol (centralized) - typical option: snoopy protocol (decentralized) MESI: Modified/Exclusive/Shared/Invalid - track global status of cache word with 2 bits - coordinate via snoop signals (set bits in others' cache words)

  6. Cluster

  7. Blade

  8. CC-NUMA

  9. Hyperthreading and Multicore For a single CPU, pipelining and superscalar are the bread and butter However, we can do more We can grab multiple instruction streams • Each stream represents a different program • When one stream stalls, feed in another Hyperthreading switches among multiple instruction streams (called “threads”) Multicore does multiple streams in parallel

  10. Hyperthreading and Multicore

  11. Variations on Chip Organization

  12. Intel Multicore

  13. ARM Multicore

  14. Finer Distinctions Divide: ask who divides the work? Hide: easier to hide divisions with some Share: how much data sharing is needed? These issues are becoming more a part of the programmer's design decisions Concurrent programming is becoming a necessary skill; used to be extra A major topic covered in Operating Systems

  15. SMP vs Cluster vs CC-NUMA SMP versus Cluster • SMP has trouble scaling past 64 processors • SMP packaged more conveniently Cluster versus CC-NUMA • CC-NUMA scales like cluster • CC-NUMA data sharing more transparent • “NU” effect untried, uncharacterized

  16. Where do SMT, Multicore Fit? SMP versus Hyperthreading and Multicore • better packaged than SMP Hyperthreading versus Multicore • Hyperthreading more transparent • Multicore scales better, but with limits Real world: do 'em all at multiple levels How about their place in Flynn's taxonomy?

More Related