Scaling Area Under a Curve

1. Scaling Area Under a Curve

2. Why do parallelism? • Speedup– solve a problem faster. • Accuracy – solve a problem better. • Scaling– solve a bigger problem.

3. Problem 1: communication overhead • Parallel algorithms that employ distributed memory have a certain amount of communication and a certain amount of computation. • As more processes are added, the communication overheadincreases. • At some number of processes, communication overhead will cause the algorithm to take longer in parallel than serially.

4. Problem 2: Amdahl’s law • Speedup is limited by the algorithm’s critical section– the part of the algorithm that cannot be parallelized. • An algorithm will never be faster than its critical section. Speedup = where P = the proportion of the program that can be made parallel 1 – P = the proportion of the program that cannot be made parallel N = the number of processors

5. Strong scaling • Strong scaling– increasing the number of processes but keeping the problem size constant.

6. The solution • Don’t solve a problem faster, instead solve a bigger problem.

7. Gustafson’s law • A bigger problem solved with more processors can be solved in the same amount of time it takes to solve a smaller problem with fewer processors. Speedup(N) = N – (1 – P) * (N – 1) where N = the number of processors (1 – P) = the proportion of the program that cannot be made parallel

8. Weak scaling • Weak scaling– increasing the problem size as the number of processes increases.