Parallel Compilers

1. Parallel Compilers Ian Lehmann Jordan Wright

2. What is it? • Find parallelism in sequential code and exploit it to speed up execution time without the programmer needing to modify their source code.

3. Why do we want it? • The goal of automatic parallelization is to relieve programmers from the tedious and error-prone manual parallelization process.

4. How does it work? • 2 Methods • cyclic multi-threading • pipelined multi-threading

5. Cyclic Multi-threading

6. Pipelined Multi-threading

7. “Embarrassingly Parallel” • Search for loops without dependences between iterations. • No synchronization needed • Speedup proportional to workers

8. “Embarrassingly Parallel” Before After void foo() { start( task(0,4)); start( task(1,4)); start( task(2,4)); start( task(3,4)); wait(); } voidtask(k, M) { for(int i=k;i< N;i+= M) array[i]= work(array[i]); } voidfoo() { for(inti=0; i < N; i++) array[i]= work(array[i]); }

9. Difficulties • Dependence analysis is hard for code using indirect addressing, pointers, recursion, and indirect function calls intfactorial(n) { if(n ==0) return1; else returnn * factorial(n -1); }

10. Difficulties • Loops have an unknown number of iterations for(inti =0; i < N; i++) { if(array[i]>5) break; array[i]+=5; }

11. Difficulties • Accesses to global resources are difficult to coordinate in terms of memory allocation, I/O, and shared variables. for(inti =0; i < N; i++) { array[i]= work(array[i-1]); }

12. Real World Projects • Intel Compiler • Harvard HELIX Research Project

13. Demo #include <cmath> intmain() { constint N =200000000; double*array =newdouble[N]; for(inti=0;i< N;i++){ array[i]=double(i)*double(i)+1.0; array[i]=sqrt(array[i]); } delete[] array; return0; }

14. Questions ?

15. Sources • http://en.wikipedia.org/wiki/Automatic_parallelization • http://helix.eecs.harvard.edu/index.php/Main_Page • http://software.intel.com/en-us/articles/automatic-parallelization-with-intel-compilers • http://en.wikipedia.org/wiki/Automatic_parallelization_tool