Loop Parallelism and OpenMP CS433 Spring 2001

1. Loop Parallelism and OpenMPCS433Spring 2001 Laxmikant Kale

2. Loop parallelism • Observation: • Most programs that need to be parallelized derive their complexity (in terms of the number of operations executed) from iterative behavior, expressed in loops • If we can execute multiple iterations of a loop in parallel… • We can often, indeed, do that • What are the conditions? • Sometimes, the code needs a bit of restructuring • Several research efforts • Also, automating the parallelization of loops via compilers • But that is not our focus in this lecture

3. OpenMP evolution • Standardization effort: • Included shared memory programming as well • I.e. subsumes Pthreads • Involves many vendors: • Intel, SGI, HP, DEC/Compaq, .. • Language bindings: • Fortran, C, C++

4. Overview • Basic view: • The program is single threaded, but it spawns parallel subtasks (threads) once in a while • Note the distinction with Pthreads model • There, each thread could be executing completely different code • But typically, it doesn’t • Parallel threads • Worksharing: • parallel loops • sections • Shared variables: • shared, private, firstprivate, lastprivate, ..

5. Spawning threads • Executes a section of code, by spawning multiple threads to execute it. • Each thread gets a copy of all private variables • Separate from the “main program”’s copy • (so, there are k+1 copies if there are k threads fired) • Threads may uses locks and barriers • calls are different than PThreads double A[1000]; omp_set_num_threads(4); #pragma omp parallel { int ID = omp_thread_num(); pooh(ID,A); } Some Example/s from sc99 Tutorial at OpenMP web site.

6. Work-sharing: for construct • Do all the iterations of a loop in parallel • We will use Prof. Padua’s slides for this • Not just the syntax and semantics of the parallel loop construct • But how to write code, by transforming sequential loops, so they are parallel • Slides 20-45 (Section 4.1 to 4.4), and 55-59 (4.7) • Also read the OpenMP C manual • http://polaris.cs.uiuc.edu/~padua/cs320 • http://polaris.cs.uiuc.edu/~padua/cs320/guidec.pdf #pragma omp parallel #pragma omp for schedule(static) for(i=0;I<N;i++) { a[i] = a[i] + b[i];}

7. Work-sharing: Sections • Parallel Loops allow identical sections of code (with different loop indices) to execute in parallel • Sections allow different sections of code to be run by parallel threads #pragma omp parallel #pragma omp sections { X_calculation(); #pragma omp section y_calculation(); #pragma omp section z_calculation(); }