Computer Science 320

Computer Science 320 Reduction Variables and Operators

Get a Histogram of Mandelbrot Set • Print to a file the number of points generated on each iteration • Same inputs as before, but the output text file will hold a table whose first column runs from 0 to N and whose second column includes the number of points for each I

MSHistogramSeq • Maintains an array of N + 1 integers • Increments the array element at position i after exiting the nested loop • Prints the array indexes and contents to a text file // Histogram (array of counters indexed by pixel value). static int[] histogram; … // Create histogram. histogram = new int [maxiter + 1]; … while (i < maxiter && zmagsqr <= 4.0){ // i will be the pixel “value” when the loop exits } // Increment histogram counter for pixel value. ++ histogram[i];

Parallelize without Reduction • All threads write to the same shared array • Use edu.rit.reduction.SharedIntegerArray for multiple-thread-safe incrementAndGet • Only minor changes to the structure of the Seq program

MSHistogramSmp import edu.rit.pj.reduction.SharedIntegerArray; // Histogram (array of counters indexed by pixel value). static SharedIntegerArray histogram; // Create histogram. histogram = new SharedIntegerArray (maxiter + 1); … execute (0, height-1, new IntegerForLoop() … while (i < maxiter && zmagsqr <= 4.0){ // i will be the pixel “value” when the loop exits } // Increment histogram counter for pixel value. histogram.incrementAndGet(i);

Reduction Pattern • Shared variable to hold the complete result of the whole program • Each thread has its own variable to hold the partial result it computes • The partial results are combined in the shared variable as each thread finishes

Reduction Variables and Operators A reduction variable holds the complete result A reduction operator combines this variable with a partial result The variable and operator are multiple-thread-safe

Using a Reduction Operator // Import integer and operator classes import edu.rit.pj.reduction.SharedInteger; import edu.rit.pj.reduction.IntegerOp; // Initialize an integer SharedInteger x = new SharedInteger(0); // Thread-safe increment x by 4 and return new value of x to result int result = x.reduce(4, IntegerOp.SUM); (x op y) = x.reduce(y, op) // Definition of SUM in IntegerOp public static final IntegerOp SUM = new IntegerOp(){ public in op(int x, int y){ return x + y; } };

Reduction Operator for Arrays // Import shared integer array and operator classes import edu.rit.pj.reduction.SharedIntegerArray; import edu.rit.pj.reduction.IntegerOp; // Initialize a shared integer array and a regular integer array SharedIntegerArray x = new SharedIntegerArray(1000); int[] y = new int[1000]; // Thread-safe increment each x[i] by y[i] x.reduce(y, IntegerOp.SUM); x.length <= y.length

Threads Reducing with an Array

MSHistogramSmp2 // In the IntegerForLoop // Reduce per-thread histogram into global histogram. public void finish(){ histogram.reduce(thr_histogram, IntegerOp.SUM); }

MSHistogramSmp2 // In the ParallelRegion // Per-thread histogram, plus extra padding. int[] thr_histogram = new int [maxiter + 1 + 32]; long p0, p1, p2, p3, p4, p5, p6, p7; long p8, p9, pa, pb, pc, pd, pe, pf; // In the IntegerForLoop // Reduce per-thread histogram into global histogram. public void finish(){ histogram.reduce(thr_histogram, IntegerOp.SUM); }

Critical Sections • Sometimes you have to do more than appl;y a binary reduction operator to combine results • You have to execute a more complex block of code that is multiple-thread-safe • Critical sections support the needed synchronization

Critical Sections • When a thread is executing code in a critical section and another thread enters the same critical section, the second thread blocks until the first thread completes the section • And so on, for all the other threads • Guarantees mutual exclusion within a critical section: only one thread at a time can be executing in the section

Critical Sections in Parallel Java new ParallelTeam().execute(new ParallelRegion(){ public void run(){ // Non-multiple-thread-safe code critical(new ParallelSection(){ public void run(){ // Code for the critical section (multiple-thread-safe) } }); // Non-multiple-thread-safe code } });

MSHistogramSmp3 // Create histogram. histogram = new int [maxiter + 1]; // Reduce per-thread histogram into global histogram. public void finish() throws Exception{ region().critical(new ParallelSection(){ public void run(){ for (inti = 0; i <= maxiter; ++ i){ histogram[i] += thr_histogram[i]; } } }); } Shared histogram is now a standard array, made thread-safe by being accessed within a critical section

Computer Science 320