1 / 35

A Multiple Associative Model to Support Branches in Data Parallel Applications

A Multiple Associative Model to Support Branches in Data Parallel Applications. Wittaya Chantamas and Johnnie W. Baker Department of Computer Science Kent State University, Kent, OHIO 44242 USA Telephone: (330) 672-9055 Fax: (330) 672-7824 wchantam@cs.kent.edu and jbaker@cs.kent.edu.

zasha
Download Presentation

A Multiple Associative Model to Support Branches in Data Parallel Applications

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. A Multiple Associative Model to Support Branches in Data Parallel Applications Wittaya Chantamas and Johnnie W. Baker Department of Computer Science Kent State University, Kent, OHIO 44242 USA Telephone: (330) 672-9055 Fax: (330) 672-7824 wchantam@cs.kent.edu and jbaker@cs.kent.edu

  2. Outline • SIMD and Branches • Single Instruction Multiple Data (SIMD) • A data parallel program contains branches • MASC Computational Model • Multiple Associative Computing (MASC) Model • MASC model with manager-worker paradigm • The power of MASC model • With variations of MASC • With other models • ASC language compiler support for the MASC model • MASC Algorithm • Shapes algorithm • Modified ASC Quick Hull algorithm for the MASC model with manager-worker paradigm MASC - Spring 2007

  3. SIMD and Branches • Most SIMD computers allow masking of PEs while determining whether or not that PE should participate in the operation in a parallel IF-THEN-ELSE statement • The THEN and ELSE parts have to be executed sequentially MASC - Spring 2007

  4. SIMD and Branches • A traditional SIMD computer • IF ( a parallel condition) THEN Statement Block A ELSE Statement Block B • Suppose we have 14 PEs MASC - Spring 2007

  5. SIMD and Branches • A traditional SIMD computer • IF ( a parallel condition) THEN Statement Block A ELSE Statement Block B • Suppose we have 14 PEs MASC - Spring 2007

  6. SIMD and Branches • A traditional SIMD computer • IF ( a parallel condition) THEN Statement Block A ELSE Statement Block B • Suppose we have 14 PEs MASC - Spring 2007

  7. SIMD and Branches • A traditional SIMD computer • IF ( a parallel condition) THEN Statement Block A ELSE Statement Block B • Suppose we have 14 PEs MASC - Spring 2007

  8. SIMD and Branches • Question: How can we improve the execution of the branches if we can have more than one instruction stream? • One probable answer: Execute each part of the branches simultaneously by using each of the instruction streams MASC - Spring 2007

  9. SIMD and Branches • The MASC computational model • IF ( a parallel condition) THEN Statement Block A ELSE Statement Block B • Suppose we have 14 PEs MASC - Spring 2007

  10. SIMD and Branches • The MASC computational model • IF ( a parallel condition) THEN Statement Block A ELSE Statement Block B • Suppose we have 14 PEs MASC - Spring 2007

  11. Outline • SIMD and Branches • Single Instruction Multiple Data (SIMD) • A data parallel program contains branches • MASC Computational Model • Multiple Associative Computing (MASC) Model • MASC model with manager-worker paradigm • The power of MASC model • With variations of MASC • With other models • ASC language compiler support for the MASC model • MASC Algorithm • Shapes algorithm • Modified ASC Quick Hull algorithm for the MASC model with manager-worker paradigm MASC - Spring 2007

  12. MASC Computational Model • An extension of the associative computing model or ASC • ASC model was created to capture Aspro and Staran ‘s style of programming • The associative properties • Broadcast data in constant time • Constant time global reduction of • Boolean values using AND/OR • Integer values using MAX/MIN • Constant time data search • Provides content addressable data • Eliminates need for sorting and indexing. • Pick one responder in constant time • Supported in hardware with the broadcast and reduction network MASC - Spring 2007

  13. MASC Computational Model • The basic components of the model • A set of instruction streams • An array of cells (PE + Memory) • IS-to-Cell broadcast and reduction networks (one for each IS, preferable) • An IS-to-IS Network • A simple cell network MASC - Spring 2007

  14. MASC Using Manager-Worker Paradigm • A variation of the MASC model • Two types of ISs • A manager-IS (ID 0 ) • Managing the work pool of tasks • Coordinating and assigning subtasks (using a FORK operation) • Combining finished tasks (using a JOIN operation) from worker-ISs • Identical worker-ISs (ID 1 to ID m) • Executing tasks in an associative (e.g., data parallel, SIMD) fashion using the PEs currently assigned to it MASC - Spring 2007

  15. MASC Using Manager-Worker Paradigm • The IS network • An IS-to-IS broadcast/reduction network • The manager-IS can use the network to perform a Min/MAX or logical reduction on ISs in constant time (pick one idle worker-IS) • The cell network is optional MASC - Spring 2007

  16. MASC Using Manager-Worker Paradigm • A Cell is a simple ALU + its local memory • IS-Selector Register (an (lg m)-bit register if there are m ISs) • Holding the instruction stream ID, to which that PE is currently listening • The register can be set or reset by the instruction stream that the PE is listening to or by the data tested in that PE • Task-History Stack (use the memory in each cell) • Holding the task ID • The default content is empty • The top of the stack always shows • the current task the PE is currently executing, • the task that has just been finished, or • a new task that has not yet assigned to a worker-IS • At any point in time, each PE listens to exactly one IS MASC - Spring 2007

  17. MASC Using Manager-Worker Paradigm • A task is broken down into subtasks • No interaction between subtasks during their executions • A FORK operation • Generates one or more subtasks from a branch by partitioning PEs into group based on the parallel condition • New task id will be push into the Task-History Stack • Those subtasks will be assigned to worker-ISs to be executed concurrently by setting the IS-Selector register of PEs in the corresponding group to the ID of the worker-IS • A JOIN operation • Recombines subtasks into the original parent task (i.e., the one existing prior to the fork) after they have been successfully executed by popping top of the Task-History Stack MASC - Spring 2007

  18. MASC Using Manager-Worker Paradigm • A work pool (WP-Q) • Containing tasks ready to be executed Worker-IS Worker-IS Work Pool MASC - Spring 2007

  19. Fork Operation Wittaya Chantamas, 08/24/2004 MASC - Spring 2007

  20. Join Operation Wittaya Chantamas, 08/24/2004 MASC - Spring 2007

  21. The Power of MASC Model • Among the variations of the MASC model, the original MASC model with a simple cell network (1-d, 2-d, or hypercube) has the same power as • A MASC model without any cell network • 1-d cell network can be simulated in the MASC without any cell network in O(1) with a polynomial blow-up in size (PEs and ISs) • A proof of the 2-d and hypercube network case is similar to the case of 1-d cell network • A MASC model with manager-worker paradigm (We believed! Need further proof.) MASC - Spring 2007

  22. The Power of MASC Model • Comparing to other models, the MASC model • has the same power as • Basic and Segmenting Reconfigurable Multiple Bus Machine (RMBM) • CRCW-PRAM • A restriction version of RM • A Mesh with Multiple Broadcasting (MMB) • is less powerful than • Fused and Extended RMBM • Reconfigurable Mesh (RM) • Linear Mesh (LM) MASC - Spring 2007

  23. ASC Language Compiler Support for the MASC Model • The MASC model needs a multiple IS support from the ASC • An extension of the ASC language compiler for the MASC model • A MASC directive • Concurrent data parallel executions of different paths in a branch can be achieved by using the directive /* .masc fork*/ • A user has a tight control • Not all different paths in branches will be executed concurrently • Only those in branches with directives will • Considered as a comment by the ASC compiler (will show in .lst file, not show in .iob file) • No need for a new ASC compiler in order to run an ASC program in MASC system • Need another extension if wanted to add a parallel case statement support MASC - Spring 2007

  24. A parallel IF-THEN-ELSE statement in the ASC language IF condition expression THEN statement block A ELSE statement block B ENDIF; MASC - Spring 2007

  25. main test int parallel b[$], c[$], d[$]; logical parallel BCD[$]; associate b[$], c[$], d[$] with BCD[$]; read b[$] c[$] d[$] in BCD[$]; b[$] = c[$] + 2; c[$] = d[$] - 3; /* will be no fork here */ if (b[$] .lt. c[$]) then b[$] = c[$]; d[$] = 4; else c[$] = b[$]; b[$] = d[$]; endif; c[$] = d[$]; d[$] = c[$]; end; a structure code M100 0000 .MI_BEGIN W1100000 beg_of_stmt 1c00 6 0 beg_read 5a00 SYSOT BCD B,C,D, … beg_of_stmt 1c00 20 0 mvpa_ 4812 C D .MI_END W1100000 M100 0000 W110 0000 W110 0000 M111 0000 M111 0000 MASC - Spring 2007

  26. A parallel IF-THEN-ELSE statement in the ASC language /* .MASC fork */ IF condition expression THEN statement block A ELSE statement block B ENDIF; MASC - Spring 2007

  27. a structure code M100 0000 main test int parallel b[$], c[$], d[$]; logical parallel BCD[$]; associate b[$], c[$], d[$] with BCD[$]; read b[$] c[$] d[$] in BCD[$]; b[$] = c[$] + 2; c[$] = d[$] - 3; /*.MASC FORK */ if (b[$] .lt. c[$]) then b[$] = c[$]; d[$] = 4; else c[$] = b[$]; b[$] = d[$]; endif; c[$] = d[$]; d[$] = c[$]; end; M100 0000 W110 0000 .MI_BEGIN W1112000 beg_of_stmt 1c00 16 0 mvpa_ 4812 B C beg_of_stmt 1c00 17 0 mvpa_ 4812 D B .MI_END W1112000 W110 0000 M111 0000 M111 0000 W111 1000 W111 1000 W111 2000 W111 2000 W111 X100 W111 X100 M111 X110 M111 X110 MASC - Spring 2007

  28. Outline • SIMD and Branches • Single Instruction Multiple Data (SIMD) • A data parallel program contains branches • MASC Computational Model • Multiple Associative Computing (MASC) Model • MASC model with manager-worker paradigm • The power of MASC model • With variations of MASC • With other models • ASC language compiler support for the MASC model • MASC Algorithm • Shapes algorithm • Modified ASC Quick Hull algorithm for the MASC model with manager-worker paradigm MASC - Spring 2007

  29. Shape Problem • The testing problem • To compute area of basic shapes in a database • Can use the MASC model to solve this problem • Each type of shapes required different equation to compute the area • Areas of each shape types can be compute simultaneously by partitioning PEs in to groups (triangle, rectangle, or circle) and using one IS to compute the areas for each group MASC - Spring 2007

  30. MASC Quick Hull Algorithm • The convex hull problem • The convex hull of a set of points S is the smallest convex set containing S. In particular, each point of set S is either on the boundary of or in the interior of the convex hull • Modified ASC Quick Hull algorithm for the MASC model with a limited number of ISs and using manager-worker paradigm with work pool MASC - Spring 2007

  31. MASC Quick Hull Algorithm Algorithm MASC Quick Hull (for the upper hull) Input: A set of points S given as (x,y) coordinates, each PE holds one point in S Output: vertices of the upper convex hull • The manager assigns the initialization task (i.e., task 0) to a worker IS to find two extreme points, X-min point (w) and X-max point (e) • Two points (w and e) in the convex hull are identified • The manager creates task we and places it in the work pool. The PEs associated with this task are the ones whose point lies above segment we MASC - Spring 2007

  32. MASC Quick Hull Algorithm • The manager assigns each task pq in the work pool to a worker IS to find another point in the convex hull using the PEs assigned to this task. • Another point (r) in the convex hull is identified • The manager places task pr and task rq in the work pool. The PEs associated with each task are the ones whose point lies above corresponding line segment • The manager continues to execute 2 steps above until there are no active tasks and no tasks remain in the work pool, and then terminates the algorithm MASC - Spring 2007

  33. MASC Quick Hull Algorithm W-IS: Execute Task PR M-IS: Join Task PR M-IS: Fork Task 0 M-IS: Join Task 0 M-IS: Fork Task WE M-IS: Fork Task PR and Task RQ M-IS: Join Task WE T pr J T 0 T we F J F J F T rq J W-IS: Execute Task 0 W-IS: Execute Task WE W-IS: Execute Task RQ M-IS: Join Task RQ MASC - Spring 2007

  34. MASC Quick Hull Algorithm • Timing • n is the number of points in S and m is the number of instruction streams • Still O(n) in the worst case • If we assume that on the average O(lg n) is the number of convex hull points, the average case running time is O((lg lg n)(lg n)/m) • Producing a constant speedup of approximately m over the 1-IS version of the same algorithm for the average case MASC - Spring 2007

  35. Conclusion • Traditional SIMD executes each part of branches of a data parallel program sequentially • MASC can execute most or all parts of the branches simultaneously if there are enough instruction streams • The original MASC model with a simple cell network is as powerful as a model without any cell network or with manager/worker paradigm • The MASC model is as powerful as many computational models such as PRAM and some versions of RMBM • An extension of the ASC compiler is required to take the benefit of having multiple ISs • Some problems can take the advantage of having more than one instruction stream. Some do not. MASC - Spring 2007

More Related