1 / 13

Bernstein’s Conditions

Bernstein’s Conditions. Techniques to Exploit Parallelism in Sequential Programming. Hierarchy of levels of parallelism: Procedure or Methods Statements Expressions. Techniques to Exploit Parallelism in Sequential Programming. Expression Level (Implicit Parallelism)

kordell
Download Presentation

Bernstein’s Conditions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Bernstein’s Conditions

  2. Techniques to Exploit Parallelism in Sequential Programming • Hierarchy of levels of parallelism: Procedure or Methods Statements Expressions

  3. Techniques to Exploit Parallelism in Sequential Programming • Expression Level (Implicit Parallelism) Using the associative, commutative, and distributive properties of arithmetic expressions, object code will be produced where more than one instruction can be executed in parallel. • Associativity: Z = ([(A+B)+C]+D)  Z = (A+B) + (C+D) + + + + + +

  4. Techniques to Exploit Parallelism in Sequential Programming • Statement Level This level is known as the D-structure level (for Dijkstra). Statements can be assigned into one of three categories: - Sequence: S1, S2 - Selection: IF b THEN S1 ELSE S2 - Iteration: (while, repeat, …) • Bernstein, A.J. (1966), “Analysis of Programs for Parallel Processing, IEEE transactions of computers”, EC-15, pg. 757-762

  5. Sequence Example S1: xa+b Each instruction has an input space and output space, IS and OS respectively. IS is defined as the variables in the instruction that do not change after executing the instruction. For the example above IS= {a,b}. OS is defined as the variables in the instruction that change after executing the instruction. For the example above OS={x}

  6. Berstein’s Conditions Given S1 and S2 IF IS(S1) ∩OS(S2) =  (Data Dependent) & OS(S1) ∩IS(S2) =  (Anti Dependent) & OS(S1) ∩OS(S2) =  (Output Dependent) THEN S1 and S2 can be executed in parallel

  7. Iteration Example for i 1 to 6 do A[i]B[i] + C[i] Using vectorization (expansion) we can write as A[1]B[1]+C[1] A[2]B[2]+C[2] A[3]B[3]+C[3]… Applying Berstein’s laws tell us that the above instructions can be executed in parallel.

  8. Selection Given AB+1 IF A>5 CA+1; Go to 5 ELSE DA+C; Go to 7 END IF 5: EA+1 6: FA+1 To exploit parallelism the program can be written as AB+1

  9. Selection To exploit parallelism the program can be rewritten as AB+1 b A > 5 CA+1 when b; DA+C when (not b); EA+1 when b; FA+1 when b;

  10. Program Transformation FOR I  1 TO n do X  A[I] + B[I] . . Y[I]  2 * X . . X  C[I] / D[I] . . P  X + 2 ENDFOR FOR I  1 TO n do X  A[I] + B[I] . . Y[I]  2 * X . . XX  C[I] / D[I] . . P  XX + 2 ENDFOR  removes data dependency data dependency

  11. Scalar Expansion FOR I  1 TO n do X  A[I] + B[I] . . Y[I]  2 * X . . ENDFOR FOR I  1 TO n do X  A[I] + B[I] . . Y[I]  2 * X[I] . . ENDFOR  removes data dependency data dependency

  12. Loop Unrolling FOR I  1 TO n do X[I]  A[I] * B[I] ENDFOR X[1]  A[1] * B[1] X[2]  A[2] * B[2] . . X[n]  A[n] * B[n] 

  13. Loop Fusion or Jamming FOR I  1 TO n do X[I]  Y[I] * Z[I] ENDFOR FOR I  1 TO n do M[I]  P[I] + X[I] ENDFOR a) FOR I  1 TO n do X[I]  Y[I] * Z[I] M[I]  P[I] + X[I] ENDFOR b) FOR I  1 TO n do M[I]  P[I] + Y[I] * Z[I] ENDFOR 

More Related