Lecture 12 Parallel Processing Architectures

1. Lecture 12Parallel Processing Architectures Parallel Processing Architectures

2. Lecture 12: Parallel Processing Architectures In this lecture, we will study • Pipeline and parallel processing • Pipeline Architecture and Vector Architecture • Classification of Computer Architectures • SISD • SIMD • MISD • MIMD Parallel Processing Architectures

3. A system with p copies of A will complete the jobs in m/p x n • seconds Parallel Processing • A job requires n seconds to complete using machine A: • There are m(m is a large number) jobs which need to be completed. • A will complete the jobs in m x n seconds • Machine A can be organized with h stages, A1, A2, … , Ah, with identical processing delay such that • When the task j in job i completes in stage Aj, • Job i enters the stage j+1 to execute the task j+1, and • Job i+1 enters from Aj-1 to Aj to execute the task j This is a pipeline execution and it will, on the average, complete a job in every delay of the pipeline stage. Thus, m jobs will be completed in (n/h) x (h-1) + (n/h) x m seconds Parallel Processing Architectures

4. P0 P1 Pn-1 . . . Parallel Processing Instruction execution time remain same as ordinary single computer. Throughput - amount work done in a unit time Throughput becomes n times the single processor Parallel Processing Architectures

5. Processor S0 S1 S2 S3 S4 Time 0 1 2 3 4 5 6 7 8 9 10 11 ... I0 I01 I02 I03 I04 I05 I1 I11 I12 I13 I14 I15 I2 I21 I22 I23 I24 I25 I3 I31 I32 I33 I34 I35 I4 I41 I42 I43 I44 I45 I5 I51 I52 I53 I54 I55 I6 I61 I62 I63 I64 I65 I7 I71 I72 I73 I74 I75 I8 I81 I82 I83 I84 ... ... . . . Completion I0 I1 I2 I3 I4 I5 I6 I7... Pipeline Prcessor Instruction execution time remain same as ordinary single computer. But effective instruction execution time becomes delay of a pipeline stage. Parallel Processing Architectures

6. Input/Output Data Bus Control Processor Processor Control Memory PE0 PE1 PEn-1 . . . Processor Processor Processor Local Memory Local Memory Local Memory Control Interconnection Network Vector Processor Parallel Processing Architectures

7. Scalar Processor SP1 SP2 Scalar Register File . . . Scalar Fetch Vector Fetch SPn Scalar Pipeline Memory IF ID OF Input/Output Vector Processor VP1 Vector Register File VP2 . . . VPn Vector Pipeline Pipeline Processor Parallel Processing Architectures

8. Classification of Computer Architecture • Flynn’s classification Instruction Stream • Continuous instruction sequence executed by a processor Data Stream • Continuous data sequence processed by a processor • SISD • SIMD • MISD • MIMD Parallel Processing Architectures

9. IS PU IS DS CU P M Common Memory DS PU1 MM1 DS PU2 MM2 IS CU . . . . . . DS PUn MMm IS SISD and SIMD • SISD • SIMD Parallel Processing Architectures

10. IS1 IS1 CU1 PU1 IS2 IS2 CU2 PU2 Common Memory ... ... ISn ISn MM1 MMm MM2 ... CUn PUn DS ISn ... IS2 IS1 IS1 IS1 DS1 Common M CU1 PU1 IS1 MM1 IS2 IS2 DS2 IS2 CU2 PU2 MM2 ... ... ... ISn ISn DSn CUn PUn ISn MMn MISD and MIMD • MISD • MIMD Parallel Processing Architectures