Pipeline Pattern

1. Pipeline Pattern ITCS 4/5145 Parallel Computing, UNC-Charlotte, B. Wilkinson, 2012 slides5.ppt Oct 24, 2013.

2. Pipeline pattern Task 2 Task 1 Task 3 Computation divided into a series of tasks that have to be performed one after the other, with the result of one task passed on to the next task. Basis of sequential programming. Result Pipeline pattern is rather specialized and applicable is certain problems and algorithms for good efficiency and speed up.

3. Effective Pipeline pattern Task 2 Task 1 Task 3 To use as a effective parallel pattern, each task executed by a separate process/processor and somehow processes/ processors have to operate at the same time to get increased speed Slave processes Result Master One way is the approach seen in a automobile assembly line, where a series cars are constructed, in this case a series of identical computations but with different data.

4. Seeds Pipeline Pattern See “Pipeline Template Tutorial.” by Jeremy Villalobos. http://coit-grid01.uncc.edu/seeds/docs/pipeline_tutorial.pdf Interface: public abstract class PipeLine extends BasicLayerInterface { public abstract Data Compute(int stage, Data input); public abstract Data DiffuseData(int segment); public abstract void GatherData(int segment, Data dat); public abstract int getDataCount(); public abstract int getStageCount(); }

5. Pipeline applications Broadly speaking, we can divide pipeline applications into two types: 1. More than one instance of complete problem is executed. Partial results passes from one pipeline stage to the next – example is an automobile assembly line building cars. 2. A single problem instance is executed that has a series of data items to be processed sequentially, each requiring multiple operations. Numbers pass from one stage to the next Although a pipeline application may exhibit features of both types. Sometimes a single problem instance requires a series of data items to be processed

6. 1. More than one instance of the complete problem is to be executed Pipeline Space-Time Diagram p stages m instances of problem Execution time = p + m – 1 steps

7. Speedup factor So if we provide a constant stream of instances of the problem, get maximum speedup.

8. 2. Single problem with a series of data items to be processed Pipeline Space-Time Diagram Execution time = p + n – 1 steps n data items

9. Speedup factor So if we provide a constant stream of data items, get maximum speedup. Processors can use this form of pipeline for streams of data (video etc.)

10. Example Pipelined Solutions

11. Matrix-Vector Multiplication c = A x b A is a matrix, b and c are vectors. Matrix-vector multiplication follows directly from definition of matrix-matrix multiplication by making B an n x1 matrix (vector). Result an n x 1 matrix (vector).

12. Matrix-Vector Multiplication A[0][3] A[0][3] A[0][3] A[0][3] A[0][2] A[0][2] A[0][2] A[0][2] A[0][1] A[0][1] A[0][1] A[0][1] A[0][0] A[0][0] A[0][0] A[0][0] . . . . . . delay a a a a B[3] B[2] B[1] B[0] sin sout sin sout sin sout P2 P3 P0 P1 C elements accumulate in pipeline stages C[0] C[1] C[2] C[3] B array elements pass through

13. Might be more easy to see drawn this way

14. Space-Time diagram P6 P5 P4 P3 A[3][3] A[3][2] A[3][1] A[3][0] P2 A[2][3] A[2][2] A[2][1] A[2][0] P1 A[1][3] A[1][2] A[1][1] A[1][0] P0 A[0][3] A[0][2] A[0][1] A[0][0] Time B[0] B[1] B[2] B[3]

15. Matrix Multiplication Two-dimensional pipeline

16. Another Example Add together all the elements of each row of an array: Add elements of each row j B[i] for (i = 0; i < N; i++) { B[i] = 0; for (j = 0; j < M; j++) { B[i] += A[i][j]; } } i A[i][j] row column

17. Pipeline A[i][4] A[i][2] A[i][1] A[i][3] A[i][0] Initialize B[i]=0 Final results a a a a a B[i] sin sin sin sin sin sout sout sout sout sout P2 P4 P3 P0 P1 0 < I < N Sout = Sin + a Note the value of i is needed in each stage Series of values, B[0], B[1], B[2], with partial accumulations of A rows pass through pipeline. While Pj is operating on B[i], Pj-1 is operating on B[i+1].

18. Unfolding loops We are essentially unfolding inner for loop and then each statement is done by a separate pipeline stage: for (i = 0; i < N; i++) { B[i] = 0; for (j = 0; j < M; j++) { B[i] += A[j][i]; } } for (j = 0; j < M; j++) { B[i] += A[0][i]; B[i] += A[1][i]; B[i] += A[2][i]; B[i] += A[3][i]; … } Unfold

19. Questions/discussion How does the previous pipeline compare to an automobile assembly line? Which of the following types is this pipeline: 1. More than one instance of complete problem is executed. 2. A single problem instance is executed that has a series of data items to be processed sequentially, each requiring multiple operations. Numbers pass from one stage to the next

20. Frequency filter - Objective to remove specific frequencies (f0, f1, f2,f3, etc.) from a digitized signal, f(t). Signal enters pipeline from left:

21. Sorting numbers with a pipeline pattern Parallel version of Insertion sort Numbers to sort 1 4, 3, 1, 2, 5 Insertion sort: Each number is inserted into the position such that all numbers before it are smaller, by comparing with all the numbers before it. 2 5 4, 3, 1, 2 5 2 3 4, 3, 1 2 5 4, 3 4 1 5 Time steps 3 2 1 5 4 4 3 1 2 5 6 5 4 3 1 7 2 4 3 5 1 8 2 5 4 3 1 9 2 5 4 3 10 1 2

22. Pipeline for sorting using insertion sort Receive x from Pi-1 if (stored_number < x) { send stored_number to Pi x = stored_number; } else send x to Pi The basic algorithm for process Pi is: Larger numbers P0 P0 P0 Series of number to sort xn-1, … x1, x0 Compare Compare Compare xmin Next smallest number Smallest number

23. Prime Number Generation: Sieve of Eratosthenes • Series of all integers generated from 2. • First number, 2, is prime and kept. • All multiples of this number deleted as they cannot be prime. • Process repeated with each remaining number. • The algorithm removes non-primes, leaving only primes.

24. The code for a process, Pi, could be based upon Receive x from P i-1 /* repeat following for each number */ Receive number from Pi-1 if ((number % x) != 0) send number to Pi+1 Each process will not receive the same number of numbers and the exact number is not known beforehand. Use a “terminator” message, which is sent at the end of the sequence: Receive x from P i-1 for (i = 0; i < n; i++) { Receive number from Pi-1 if (number == terminator) break; if ((number % x) != 0) send number to Pi+1 }

25. Other situations that a pipeline can be used effectively If information to start next process can be passed forward before all its internal operations have been completed.

26. Example: Solving Upper-triangular System of Linear Equations using Back Substitution (created after Gaussian Elimination step for a general system of linear equations) a’s and b’s are constants and x’s are unknowns to be found.

27. Back Substitution First, unknown x0 is found from last equation; i.e., Value obtained for x0 substituted into next equation to obtain x1; i.e., Values obtained for x1 and x0 substituted into next equation to obtain x2: and so on until all the unknowns are found.

28. Pipeline Solution First pipeline stage computes x0 and passes x0 onto the second stage, which computes x1 from x0 and passes both x0 and x1 onto the next stage, which computes x2 from x0 and x1, and so on.

29. Pipeline processing using back substitution

30. More complex pipelines Non-linear pipelines – those that features such as: Multiple paths Multifunction Multiple outputs Feedback and feedforward paths These types of pipelines are found in logic design courses

31. Example pipeline with two paths This is an application for a pattern operator

32. Structures that allow two-way movement between stages Not truly a pipeline We will look into these structures separately.

33. Questions