LECTURE #1

1. LECTURE #1 INTRODUCTON TO PARALLEL COMPUTING

2. What is parallel computing? • Why we need parallel computing? • Why parallel computing is more difficult? • What are uses for Parallel Computing in Science and Engineering? • What are uses for Parallel Computing in Industrial and Commercial? • How can we do Parallel computing in Laptop or Desktop? • Briefly write the Parallel Computer Examples?

3. LECTURE #2 PARALLEL COMPUTER ARCHITECTURES

4. Flynn's Classical Taxonomy • The 4 possible classifications according to Flynn: • Single Instruction, Single Data (SISD) • Single Instruction, Multiple Data (SIMD) • Multiple Instruction, Single Data (MISD) • Multiple Instruction, Multiple Data (MIMD)

5. 1. Single Instruction, Single Data (SISD)

6. 2. Single Instruction, Multiple Data (SIMD)

7. 3. Multiple Instruction, Single Data (MISD)

8. 4. Multiple Instruction, Multiple Data (MIMD)

9. Classification for MIMD Computers • MIMD Computers are classified into two types: • Shared Memory • Distributed Memory

10. Shared Memory

11. Distributed Memory

14. Assignment #1 Briefly Explain Systolic Architecture.(For Even Roll No.) Briefly Explain Vector Architecture.(For Odd Roll No.) Important Note: • Not more than 1 and Half paper. • Write your Name, Roll No., Section and Batch on top of paper. • Assignment must be submitted in next week. • Assignment must be Hand Written.

16. LECTURE #3 CONCEPTS AND TERMINOLOGY OF PARALLEL COMPUTING

17. CONCEPTS AND TERMINOLOGY OF PARALLEL COMPUTING • High Performance Computing (HPC) • Node • Task • Pipelining • Symmetric Multi-Processor (SMP) • Synchronization • Granularity • Multiprogramming • Multiprocessing • Multitasking • Simultaneous Multithreading (SMT)

18. LECTURE #4 PARALLEL COMPUTING MATRIC

19. Amdahl’s Law Amdahl’s Law calculates the speedup of parallel code based on three variables: Duration of running the application on a single-core machine. The percentage of the application that is parallel. The number of processor cores. Here is the formula, which returns the ratio of single-core versus Multicore performance. The variable P is the percent of the application that runs in parallel, and N is the number of processor cores.

20. Numerical # 1 • Suppose you have an application that is 75 percent parallel and runs on a machine with three processor cores. • Find Speedup by using Amdahl’s Law? • Numerical # 2 • If you have an algorithm in which only 50 percent (P = 0.50) of its total work is executed in parallel with two physical cores. An algorithm with 1,000 units of work split into 500 units of sequential work and 500 units of parallelized work. Find Speedup by using Amdahl’s Law? • If the sequential version of code takes 1,000 seconds to complete then how long, the new version with some parallelized code will take?

21. Numerical # 3 • The maximum speedup for the same algorithm on a microprocessor with eight physical cores. Find Speedup by using Amdahl’s Law? • If the sequential version of code takes 1,000 seconds to complete then how long, the new version with some parallelized code will take? • Numerical # 4 • Find the maximum speedup for the algorithm according to the number of physical cores, from 1 to 16. In algorithm in which 90 percent (P = 0.90) of its total work is executed in parallel?

22. Gustafson’s Law Gustafson’s Law provides the following formula with the focus on the problem size to measure the amount of work that can be performed in a fixed time: • Total work (in units) = S + (N × P) • where: • S represents the units of work that run with a sequential execution. • P is the size of each unit of work that runs completely in parallel. • N is the number of available execution units (processors or physical cores).

23. Numerical # 5 You can consider a problem composed of 50 units of work with a sequential execution. The problem can also schedule parallel work in 50 units of work for each available core. If you have a microprocessor with two physical cores, findthe maximum amount of work ?

24. Numerical # 6 The same algorithm can run on a microprocessor with eight physical cores. Findthe maximum amount of work ?

25. Assignment # 2 Suppose you have an application that is 45 percent parallel and runs on a machine with three processor cores. Find the Speed Up by using Amdahl’s Law. (For Even Roll No.) If you have an algorithm in which only 30 percent (P = 0.30) of its total work is executed in parallel, a microprocessor with two physical cores. Find the Speed Up by using Amdahl’s Law? (For Odd Roll No.) Important Note: Not more than 1 and Half paper. Write your Name, Roll No., Section and Batch on top of paper. Assignment must be submitted in next week. Assignment must be Hand Written.

26. LECTURE # 5 LEVEL OF PARALLELISM

27. Levels Of Parallelism • Instruction Level Parallelism  • Thread Level Parallelism • Process Level Parallelism • Message-Passing Parallelism

28. Assignment # 3 Why fine grained Parallel Processing is potentially faster ? Important Note: Not more than 1 paper. Write your Name, Roll No., Section and Batch on top of paper. Assignment must be submitted in next week. Assignment must be Hand Written.

29. LECTURE # 6 INTERCONNECTION NETWORKS - I

30. Interconnection Networks: • What is Interconnection Networks? • Types of Interconnection Networks Shared versus Switched Media

31. BUS:

32. CROSSBAR NETWORK:

33. MULTISTAGE NETWORKS:

34. LECTURE # 7 INTERCONNECTION NETWORKS - II

35. HYPERCUBE NETWORKS:

36. Mesh Networks:

37. TREE NETWORKS:

38. BUTTERFLY NETWORKS: :

39. PYRAMID NETWORKS:

40. STAR NETWORK

41. QUIZ # 2 Q1) What is Interconnection Networks? Q2) Briefly defined Cross Bar Network? Q3) Briefly defined Butterfly Network? Q4) Briefly defined Hypercube Network? Important Note: Write your Name, Roll No., Section and Batch on top of paper. Time :15 Minutes .