1 / 13

Parallel Sorting Algorithms

Parallel Sorting Algorithms. Comparison Sorts if (A>B) { temp=A; A=B; B=temp; } Potential Speed-up Optimal Comparison Sort: O(N lg N) Optimal Parallel speed-up O(lg N) if P=N Huge Big Oh constant Source of problems Duplicate computations to reduce message passing

avidan
Download Presentation

Parallel Sorting Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Parallel Sorting Algorithms • Comparison Sorts if (A>B) { temp=A; A=B; B=temp; } • Potential Speed-up • Optimal Comparison Sort: O(N lg N) • Optimal Parallel speed-up O(lg N) if P=N • Huge Big Oh constant • Source of problems • Duplicate computations to reduce message passing • Different precision at different processors • Key • Processors independently working on data sections A B Max(A,B) Min(A,B) A B Max(A,B) Min(A,B)

  2. Data Partitioning P processors and n numbers Question: What is the complexity of finding the mid point? 88 50 28 25 88 50 28 25 98 88 80 50 43 42 28 25 98 88 80 50 43 42 28 25 98 88 80 50 43 42 28 25 98 80 43 42 98 80 43 42 P2 P1 P2 P1 43 42 28 25 98 80 43 42 88 50 28 25 88 50 28 25 Version 1 P2 Performs Comparisons Version 2 Duplicate Computations P1 Gets Smaller Numbers; P2 Gets Larger Numbers

  3. Sequential Bubble Sort Pass = 0; Do more = false; for (int i=0; i<n-pass; i++) if (array[i] > array[i+1]) {t=a[i]; a[i]=a[i+1]; a[i+1]=t; more=true;} pass++; Until (swaps = false); • Complexity: ∑i=1->ni = n(n-1)/2 • Example: Sort: 42785136 End of Pass 1: 24751386 End of Pass 2: 24513678 End of Pass 3: 24135678 End of Pass 4: 21345678 End of Pass 5: 12345678

  4. Parallel Bubble Sort • Pipeline Approach • Start the next pass before the previous iteration completes • Ensure that the next pass doesn’t overtake the previous pass Phase 4 Phase 3 Phase 2 Phase 1

  5. 0 1 2 3 4 5 6 7 6 2 6 5 8 4 1 1 8 6 6 3 2 1 6 7 3 3 5 4 6 7 1 3 2 3 5 3 5 6 1 1 2 3 2 5 7 8 4 7 5 5 8 8 4 4 7 7 4 5 4 4 7 3 6 8 2 7 2 2 8 1 8 1 Odd-Even Transposition Sort • Bubble Sort Modification • Operates in even and odd passes • Even passes: exchange with right neighbor • Odd passes: exchange with left neighbor Step P0 P1 P2 P3 P4 P5 P6 P7

  6. 4 2 7 8 5 1 3 6 4 2 7 8 5 1 3 6 4 2 7 8 5 1 3 6 4 2 7 8 5 1 3 6 2 4 7 8 1 5 3 6 2 4 7 8 1 3 5 6 1 2 3 4 5 6 7 8 Merge Sort • An optimal comparison sort • Complexity: O(n lg n) • Divide and conquer process allocation mergeSort(array1, array2, low, high) { middle = (low+high)/2; mergeSort(array, low, middle) mergeSort(array, middle+1, high) merge(array, low, middle+1, high) }

  7. Quick Sort • Perhaps the most popular sequential sorting method • Average optimal sequential complexity: O(n lg n) • Parallel efficiency limitations • Partitions are unbalanced • A single processor performs the initial partitioning • Work pool approach • One part given to a processor, another returned to work pool 4 2 7 8 5 1 3 6 3 2 1 4 5 7 8 6 2 1 3 4 5 7 8 6 1 2 3 6 7 8

  8. Bitonic Sequences • Monotonically increasing sequence • A sequence of increasing numbers (E.g.: 12345678) • Bitonic sequence • Two sequences, one increasing and one decreasing( E.g.: 56784321) • Note: a sequence is still considered Bitonic if the above condition occurs after performing a rotate operation • E.g.: 21567843 (Perform left rotate of two digits) • Special Characteristic of a Bitonic Sequence • Lg(n) compare and exchange operations on two Bitonic sequence results in a monotonic sequence • Notes: • The first half of the sequence is monotonically increasing • The second half of the sequence is monotonically decreasing

  9. Special Bitonic Properties • Assumptions • A[0], A[1], A[n-1] is a Bitonic sequence • A[0]  A[n/2] is monotonically increasing • A[n/2+1]  A[n] is monotonically decreasing • The value of n is n even power of 2 • Goal:Form a monotonic sequence of size n • Solution:Requires lg(n) steps • Steps: BitonicMerges of n/2, n/4, … , 1 • Bitonic k merge compares and exchanges A[k] with A[k+i]; 0<=k < n-i. 3 5 8 9 7 4 2 1 The original bitonic sequence 3 4 2 1 7 5 8 9 Bitonic Bitonic Example of a Bitonic 4 Merge where n=8

  10. 8 3 4 7 9 2 1 5 3 4 7 8 9 5 2 1 Step 1: k=2, lg 2 steps 3 4 2 1 9 5 7 8 2 9 5 1 3 8 7 4 2 1 3 4 7 5 9 8 3 4 7 8 5 9 2 1 1 2 3 4 5 7 8 9 3 4 7 8 9 5 2 1 Step 3: k=8, lg 8 steps Step 2: k=4, lg 4 steps Bitonic Merge Sort • Form Bitonic pairs (k=2) of adjacent numbers alternating between increasing and decreasing orders • At each step i, recursively form larger Bitonic sequences (|ki+1|= |2*ki|) • Remember to perform lg(i) Bitonic merges at each step • The sort completes in lg2n steps. Example

  11. Larger Bitonic Sort Example • Original Unsorted Numbers: 20,15,7,5,3,8,12,4,16,13,12,18,9,4,3,2 • After Sorting Pairs of Numbers: 15,20, 7,5, 3,8, 12,4, 13,16, 18,12, 4,9, 3,2 • Groups of four merged: 0->2,1->3; 4->6,5->7; 8->10, 9->11; 12->14, 13->157,5,15,20,12,8,3,4, 13,12,18,16,4,9,3,2 • Groups of two merged: 0->1; 2->3; 4->5; 6->7; 8->9; 10->11; 12->13; 14->155,7, 15,20,12,8, 4,3, 12,13, 16,18, 9,4, 3,2 • Groups of eight merged: 0->4,1->5,2->6,3->7, 8->12,9->13,10->14,11->155,7,4,3,12,8,15,20,12,13,16,18,9,4,3,2 • Groups of four merged: 0->2,1->3; 4->6,5->7; 8->10, 9->11; 12->14, 13->15 4,3,5,7, 12,8,15,20,16,18,12,13, 9,4,3,2 • Groups of two merged: 0->1; 2->3; 4->5; 6->7; 8->9; 10->11; 12->13; 14->15 3,4, 5,7, 8,12, 15,20,18,16, 13,12, 9,4, 3,2 • Group of sixteen merged: 0->8; 1->9; 2->10; 3->11; 4->12; 5->13; 6->14; 7->153,4, 5,7, 8,4,3,2,18,16,13,12,9,12,15,20 • Groups of eight merged: 0->4,1->5,2->6,3->7, 8->12,9->13,10->14,11->153,4, 5,7, 8,4,3,2, 9,12,13,12,18,16,15,20 • Groups of four merged: 0->2,1->3; 4->6,5->7; 8->10, 9->11; 12->14, 13->153,2, 3,4, 5,4, 8,7, 9,12, 13,12, 15,16, 18,20 • After final merge: 0->1; 2->3; 4->5; 6->7; 8->9; 10->11; 12->13; 14->152,3,3,4,4,5,7,8,9,12,12,13,15,16,18,20 Notes: Red indicates ascending; Blue indicates descending

  12. Batcher’s Odd-Even Merge Sort • Assumptions • A[0] to A[n-1] sorted • Array elements A[n] to A[2*n-1] are sorted • Goal: Form a sorted array of length 2*n • Procedure • Sort the even indices • Sort the odd indices • Perform an odd-even merge • Odd-even merge • Leave A[0] and A[2*n-1] unchanged • Compare & swap: A[i]&A[i+1] i=1;i<2*n-1;i+=2 Odd-even Merge Example: Form sorted array of 8 elements alow[] 1324 alow[] 1 2 34 alow[] 2458 ahigh[] 5768 ahigh[] 56 7 8 ahigh[] 1367 Two sorted lists Even and odd indices sorted Compare and exchange

  13. Larger Odd-Even Example • Original Unsorted Numbers 20,15,7,5,3,8,12,4,16,13,12,18,9,4,3,2 • Sort Pairs of Numbers 15,20, 5,7, 3,8, 4,12, 13,16, 12,18, 4,9, 2,3 • Odd-even sort of indices 0-3, 4-7, 8-11, 12-155,7,15,203,8,4,12, 12,16,13,182,3,4,9 • Odd-even merge5,7,15,203,4,8,12, 12,13,16,182,3,4,9 • Odd-even sort of indices 0-7, 8-153,4,5,7,8,12,15,20, 2,3,4,9,12,13,16,18 • Odd-even merge3,4,5,7,8,12,15,202,3,4,9,12,13,16,18 • Odd-even sort of indices 0-15 2,3,3,4,4,7,5,9,8,12,12,13,15,18,16,20 • Odd-even merge 2,3,3,4,4,5,7,8,9,12,12,13,15,16,18,20

More Related