1 / 42

OpenMP for Task Decomposition

OpenMP for Task Decomposition. Introduction to Parallel Programming – Part 8. Review & Objectives. Previously: Defined deadlock and explained ways to prevent it At the end of this part you should be able to: Describe how the OpenMP task pragma is different from the for pragma

mira-bond
Download Presentation

OpenMP for Task Decomposition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. OpenMP for Task Decomposition Introduction to Parallel Programming – Part 8

  2. Review & Objectives • Previously: • Defined deadlock and explained ways to prevent it • At the end of this part you should be able to: • Describe how the OpenMP taskpragma is different from the forpragma • Code a task decomposition solution with the OpenMP task construct

  3. Pragma: single • Denotes block of code to be executed by only one thread • First thread to arrive is chosen • Implicit barrier at end #pragmaomp parallel { DoManyThings(); #pragmaomp single { printf(“Many Things done\n”); }// threads wait here for single DoManyMoreThings(); }

  4. New Addition to OpenMP • Tasks – Main change for OpenMP 3.0 • Allows parallelization of irregular problems • unbounded loops • recursive algorithms • producer/consumer

  5. What are tasks? • Tasks are independent units of work • Threads are assigned to perform the work of each task • Tasks may be deferred • Tasks may be executed immediately • The runtime system decides which of the above • Tasks are composed of: • code to execute • data environment • internalcontrol variables (ICV) Serial Parallel

  6. data data data data data next next next next next A Linked List Example node *p = head; while (p) { process(p); p = p->next; } head

  7. data data data data data next next next next next A Linked List Example node *p = head; while (p) { process(p); p = p->next; } p head

  8. data data data data data next next next next next A Linked List Example node *p = head; while (p) { process(p); p = p->next; } p head

  9. data data data data data next next next next next A Linked List Example node *p = head; while (p) { process(p); p = p->next; } p head

  10. data data data data data next next next next next A Linked List Example node *p = head; while (p) { process(p); p = p->next; } p head

  11. data data data data data next next next next next A Linked List Example node *p = head; while (p) { process(p); p = p->next; } p head

  12. data data data data data next next next next next A Linked List Example node *p = head; while (p) { process(p); p = p->next; } p head

  13. Task Construct – Explicit Task View node *p = head; #pragmaomp parallel { #pragmaomp single while (p) { #pragmaomp task process(p); p = p->next; } } • A team of threads is forked at the omp parallel construct • A single thread, T0, executes the while loop • Each time T0 crosses the omp task construct it generates a new task • Each task runs in a thread • All tasks complete at the barrier at the end of the parallel region’s single construct

  14. data data data data data next next next next next A Linked List Example node *p = head; #pragmaomp parallel { #pragmaomp single while (p) { #pragmaomp task process(p); p = p->next; } } p head

  15. data data data data data next next next next next A Linked List Example node *p = head; #pragmaomp parallel { #pragmaomp single while (p) { #pragmaomp task process(p); p = p->next; } } process() p p head

  16. data data data data data next next next next next A Linked List Example node *p = head; #pragmaomp parallel { #pragmaomp single while (p) { #pragmaomp task process(p); p = p->next; } } process() p process() p p head

  17. data data data data data next next next next next A Linked List Example node *p = head; #pragmaomp parallel { #pragmaomp single while (p) { #pragmaomp task process(p); p = p->next; } } process() p p head

  18. data data data data data next next next next next A Linked List Example node *p = head; #pragmaomp parallel { #pragmaomp single while (p) { #pragmaomp task process(p); p = p->next; } } process() p process() p p head

  19. data data data data data next next next next next A Linked List Example node *p = head; #pragmaomp parallel { #pragmaomp single while (p) { #pragmaomp task process(p); p = p->next; } } process() p p head

  20. data data data data data next next next next next A Linked List Example node *p = head; #pragmaomp parallel { #pragmaomp single while (p) { #pragmaomp task process(p); p = p->next; } } process() p process() p p head

  21. data data data data data next next next next next A Linked List Example node *p = head; #pragmaomp parallel { #pragmaomp single while (p) { #pragmaomp task process(p); p = p->next; } } process() p p head

  22. data data data data data next next next next next A Linked List Example node *p = head; #pragmaomp parallel { #pragmaomp single while (p) { #pragmaomp task process(p); p = p->next; } } process() p process() p p head

  23. data data data data data next next next next next A Linked List Example node *p = head; #pragmaomp parallel { #pragmaomp single while (p) { #pragmaomp task process(p); p = p->next; } } process() p p head

  24. data data data data data next next next next next A Linked List Example node *p = head; #pragmaomp parallel { #pragmaomp single while (p) { #pragmaomp task process(p); p = p->next; } } p head

  25. When are tasks gauranteed to be complete? • Tasks are gauranteed to be complete: • At thread or task barriers • At the directive: #pragma omp barrier • At the directive: #pragma omp taskwait

  26. Example: Naive Fibonacci Calculation • Recursion typically used to calculate Fibonacci number • Widely used as toy benchmark • Easy to code • Has unbalanced task graph long SerialFib( long n ) { if( n < 2 ) return n; else return SerialFib(n-1) + SerialFib(n-2); }

  27. SerialFib(2) SerialFib(4) SerialFib(2) SerialFib(3) SerialFib(3) SerialFib(1) SerialFib(1) SerialFib(2) SerialFib(0) SerialFib(1) SerialFib(0) SerialFib(2) SerialFib(0) SerialFib(1) SerialFib(0) SerialFib(1) SerialFib(1) Example: Naive Fibonacci Calculation • We can envision Fibonacci computation as a task graph

  28. Fibonacci - Task Spawning Solution • long ParallelFib(long n) • { long sum; • #pragmaomp parallel • { • #pragmaomp single • FibTask(n,&sum); • } • return sum; • } • Write a helper function to set up parallel region • Call FibTask() to do computation • Use sum return parameter in FibTask()

  29. Fibonacci - Task Spawning Solution • void FibTask(long n, long* sum) • { • if( n < CutOff ) { • *sum = SerialFib(n); • } • else { • long x, y; • #pragmaomp task • FibTask(n-1,&x); • #pragmaomp task • FibTask(n-2,&y); • #pragmaomptaskwait • *sum = x+y; • } • } • Thread will first check the value of n against CutOff • If the cutoff hasn’t been reached, the thread will create two new tasks • One to compute the n-1 Fib value • One to compute the n-2 Fib value • The computed values for these tasks will be returned through the private variables x and y, respectively • The #pragmaomptaskwaitis required to make sure that the values for x and y have been computed before they are added together into sum

  30. Fibonacci Task Solution Example FibTask(8,*sum) long x, y; FibTask(7,&x); FibTask(6,&y); *sum = x + y;

  31. Fibonacci Task Solution Example FibTask(8,*sum) FibTask(7,*sum) long x, y; long x, y; FibTask(7,&x); FibTask(6,&x); FibTask(6,&y); FibTask(5,&y); *sum = x + y; *sum = x + y;

  32. Fibonacci Task Solution Example FibTask(8,*sum) FibTask(7,*sum) FibTask(6,*sum) long x, y; long x, y; long x, y; FibTask(7,&x); FibTask(6,&x); FibTask(5,&x); FibTask(6,&y); FibTask(5,&y); FibTask(4,&y); *sum = x + y; *sum = x + y; *sum = x + y;

  33. Fibonacci Task Solution Example FibTask(6,*sum) FibTask(6,*sum) FibTask(8,*sum) FibTask(7,*sum) long x, y; long x, y; long x, y; long x, y; FibTask(5,&x); FibTask(7,&x); FibTask(5,&x); FibTask(6,&x); FibTask(4,&y); FibTask(6,&y); FibTask(4,&y); FibTask(5,&y); *sum = x + y; *sum = x + y; *sum = x + y; *sum = x + y;

  34. Fibonacci Task Solution Example FibTask(6,*sum) FibTask(6,*sum) FibTask(8,*sum) FibTask(7,*sum) long x, y; long x, y; long x, y; long x, y; FibTask(5,&x); FibTask(7,&x); FibTask(5,&x); FibTask(6,&x); FibTask(4,&y); FibTask(6,&y); FibTask(4,&y); FibTask(5,&y); *sum = x + y; *sum = x + y; *sum = x + y; *sum = x + y;

  35. Fibonacci Task Solution Example FibTask(6,*sum) FibTask(6,*sum) FibTask(8,*sum) FibTask(7,*sum) long x, y; long x, y; long x, y; long x, y; FibTask(5,&x); FibTask(7,&x); FibTask(5,&x); FibTask(6,&x); FibTask(4,&y); FibTask(6,&y); FibTask(4,&y); FibTask(5,&y); *sum = x + y; *sum = x + y; *sum = x + y; *sum = x + y;

  36. Fibonacci Task Solution Example FibTask(6,*sum) FibTask(6,*sum) FibTask(8,*sum) FibTask(7,*sum) long x, y; long x, y; long x, y; long x, y; FibTask(5,&x); FibTask(7,&x); FibTask(5,&x); FibTask(6,&x); FibTask(4,&y); FibTask(6,&y); FibTask(4,&y); FibTask(5,&y); *sum = x + y; *sum = x + y; *sum = x + y; *sum = x + y;

  37. Fibonacci Task Solution Example FibTask(8,*sum) FibTask(7,*sum) FibTask(6,*sum) long x, y; long x, y; long x, y; FibTask(7,&x); FibTask(6,&x); FibTask(5,&x); FibTask(6,&y); FibTask(5,&y); FibTask(4,&y); *sum = x + y; *sum = x + y; *sum = x + y;

  38. Fibonacci Task Solution Example FibTask(8,*sum) FibTask(7,*sum) FibTask(6,*sum) long x, y; long x, y; long x, y; FibTask(7,&x); FibTask(6,&x); FibTask(5,&x); FibTask(6,&y); FibTask(5,&y); FibTask(4,&y); *sum = x + y; *sum = x + y; *sum = x + y;

  39. Fibonacci Task Solution Example FibTask(8,*sum) FibTask(7,*sum) long x, y; long x, y; FibTask(7,&x); FibTask(6,&x); FibTask(6,&y); FibTask(5,&y); *sum = x + y; *sum = x + y;

  40. Fibonacci Task Solution Example FibTask(8,*sum) long x, y; FibTask(7,&x); FibTask(6,&y); *sum = x + y;

  41. References • OpenMP API Specification, www.openmp.org

More Related