1 / 29

ICS 556 Parallel Algorithms

ICS 556 Parallel Algorithms. Ebrahim Malalla malalla@kfupm.edu.sa Office: Bldg 22, Room 124-8. Sequential Computers. "…sequential computers are approaching a fundamental physical limit on their potential power. Such a limit is the speed of light…". ``o'' are uniprocessors,

dagan
Download Presentation

ICS 556 Parallel Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ICS 556Parallel Algorithms Ebrahim Malalla malalla@kfupm.edu.sa Office: Bldg 22, Room 124-8

  2. Sequential Computers "…sequential computers are approaching a fundamental physical limit on their potential power. Such a limit is the speed of light…"

  3. ``o'' are uniprocessors, ``+''denotes modestly parallel computers with 4--16 processors,  ``x'' denotes massively parallel computers with 100s or 1000s of processors

  4. Parallelism • Is and can be done easily in real life (nature) • E.g., ants and pees colony. • So why not use it to solve human’s problems!

  5. Parallel Computers Supercomputers (high performance computers) are needed to • Speedup sequential algorithms • Solve large complex problems that are almost not practically solvable.

  6. Applications of Parallelism • Weather forecasting • Simulations (DNA, biomedical, flight control) • Scientific Computations • Fourier transform, wavelets • Applied linear algebra, matrix factorization and multiplication • Image and signal processing • Numerical analysis • PDEs • Huge Databases • Real-time systems with large live stream inputs

  7. Unfortunately .. • Not every algorithm can be parallelized “One woman can have a child in 9 months But 9 women can’t have a child in 1 month!”

  8. Courses in Parallelism • Parallel Architecture • Parallel Programming • Parallel Algorithms • Parallel Algorithms for scientific computations • Concurrent Programming • Distributed systems • Distributed Algorithms • Cluster Computing

  9. Parallel Architecture • Focuses on the design of hardwares • E.g., processors, memory, communications and interconnections, ...etc

  10. Parallel Programming • Code parallel programs using parallel programming languages, e.g., PVM (Parallel Virtual Machine), MPI (Massage Passing Interface), OpenMP • Study: Parallel machine and programming models, Programming techniques (partitioning, pipelined computations, synchronous computations, load balancing and termination detection) • Textbooks: • Parallel Computing, by Grama, Gupta, Karypis, Kumar, Addison-Wesley, 2003. • Parallel Programming, by Wilkinson and Allen, Pearson-Prentice Hall • Fundamentals of Parallel Processing by Jordan and Alaghband, Prentice Hall

  11. Parallel Algorithms • Focuses on the design and analysis of parallel algorithms • Given a problem • partition it into sub-problems, • Distribute these sub-problems to the processors • Solve the communication problem • Combine the solution

  12. Parallel Computation Models • Parallel algorithms are designed to run on a specific parallel computer • But there many different practical models of them (unlike the sequential computer)

  13. Shared Memory Model

  14. Interconnection Network Model Network structure could be array, tree, hypercube, or mesh based design

  15. Combinatorial Circuits Model

  16. Other options • Synchronous or asynchronous computation • SIMD MIDM • ....

  17. Massively Parallel Computer

  18. Top 10 Supercomputers (2007) Google Map of TOP 100 HPCs

  19. Intel Promised Teraflops chip Intel has introduced a prototype for A supercomputer Chip capable of teraflops Performance While consuming Very little power. It has 80 cores and consumes only 62 watts.

  20. Approaches for studying Parallel Algorithms • Model Approach: divide the subject according to the parallel computation models and for each study different parallel algorithms design techniques and consider different application problems

  21. Approaches for studying Parallel Algorithms • Problem Approach: Divide the subject according to the type of problems that can be solved by parallel algorithms like • Sorting and searching • Scientific computation • Graph problems • Computational geometry • Simulations, graphics and vision • Biomedical modeling For each type we study design techniques for models of computations that are best suited.

  22. Approaches for studying Parallel Algorithms • Models and Methods Approach: mix of the above two approaches. • Models: Combinatorial circuits, interconnected networks, shared memory • Design Methods: prefix computation, list ranking, divide and conquer, split and plan, .... etc

  23. Textbook • Parallel Computation By Selim Akl • Supplementary Lecture notes, current research papers

  24. Assessment • 4-5 assignments • 2 exams • Presentation and report based on research papers

More Related