1 / 33

Case Studies

Experiencing Cluster Computing. Case Studies. Class 7. Case 1: Number Guesser. Number Guesser. 2 players game ~ T hinker & Guesser Thinker thinks of a number between 1 & 100 Guesser guesses Thinker tells the guesser whether guess is high, low or correct Guesser’s best strategy

rcalderon
Download Presentation

Case Studies

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Experiencing Cluster Computing Case Studies Class 7

  2. Case 1:Number Guesser

  3. Number Guesser 2 players game ~ Thinker & Guesser • Thinker thinks of a number between 1 & 100 • Guesser guesses • Thinker tells the guesser whether guess is high, low or correct • Guesser’s best strategy • Remember high and low guesses • Guess the number in between • If guess was high, reset remembered high guess to guess • If guess was low, reset remembered low guess to guess  2 processes Source http://www.sci.hkbu.edu.hk/tdgc/tutorial/ExpClusterComp/guess.c

  4. Number Guesser

  5. Thinker #include <stdio.h> #include <mpi.h> #include <time.h> thinker() { int number,guess; char reply = ‘x’; MPI_Status status; srand(clock()); number = rand()%100+1; printf("0: (I'm thinking of %d)\n",number); while(reply!='c') { MPI_Recv(&guess,1,MPI_INT,1,0,MPI_COMM_WORLD,&status); printf("0: 1 guessed %d\n",guess); if(guess==number)reply = 'c'; else if(guess>number)reply = 'h'; else reply = 'l'; MPI_Send(&reply,1,MPI_CHAR,1,0, MPI_COMM_WORLD); printf("0: I responded %c\n",reply); } }

  6. Thinker (processor 0) clock() returns time in CLOCKS_PER_SEC since process started srand() seeds random number generator rand() returns next random number MPI_Recv receives in guess one int from processor 1 MPI_Send sends from reply one char to processor 1

  7. Guesser guesser() { char reply; MPI_Status status; int guess,high,low; srand(clock()); low = 1; high = 100; guess = rand()%100+1; while(1) { MPI_Send(&guess,1,MPI_INT,0,0,MPI_COMM_WORLD); printf("1: I guessed %d\n",guess); MPI_Recv(&reply,1,MPI_CHAR,0,0,MPI_COMM_WORLD,&status); printf("1: 0 replied %c\n",reply); switch(reply) { case 'c': return; case 'h': high = guess; break; case 'l': low = guess; break; } guess = (high+low)/2; } }

  8. Guesser (processor 1) MPI_Send sends from guess one int to processor 0 MPI_Recv receives in reply one char from processor 0

  9. main main(argc,argv) int argc; char ** argv; { int id,p; MPI_Init(&argc,&argv); MPI_Comm_rank(MPI_COMM_WORLD,&id); if(id==0) thinker(); else guesser(); MPI_Finalize(); }

  10. Number Guesser Process 0 is thinker & Process 1 is guesser % mpicc –O –o guess guess.c % mpirun –np 2 guess Output: 0: (I'm thinking of 59) 0: 1 guessed 46 0: I responded l 0: 1 guessed 73 0: I responded h 0: 1 guessed 59 0: I responded c 1: I guessed 46 1: 0 replied l 1: I guessed 73 1: 0 replied h 1: I guessed 59 1: 0 replied c

  11. Case 2: Parallel Sort

  12. Parallel Sort • Sort a file of n integers on p processors • Generate a sequence of random numbers • Pad the numbers and make its length a multiple of p • n+p-n%p • Scatter sequences of n/p+1 to the p processors • Sort the scattered sequences in parallel on each processor • Merge sorted sequences from neighbors in parallel • log2 p steps are needed

  13. Parallel Sort

  14. Parallel Sort e.g. Sort 125 integers with 8 processors Pad: 125+8-125%8 = 125+8-5 = 125+3 = 128 Scatter: 16 integers on each proc 0 – proc 7 Sorting: each proc sorts its 16 integers. Merge (1st step): 16 from P0 & 16 from P1  P0 == 32 16 from P2 & 16 from P3  P2 == 32 16 from P4 & 16 from P5  P4 == 32 16 from P6 & 16 from P7  P6 == 32 • Merge (2nd step): • 32 from P0 & 32 from P2  P0 == 64 • 32 from P4 & 32 from P6  P4 == 64 • Merge (3rd step): • 64 from P0 & 64 from P4  P0 == 128

  15. Algorithm • Root • Generate a sequence of random numbers • Pads data to make size a multiple of number of processors • Scatters data to all processors • Sorts one sequence of data • Other processes • receive & sort one sequence of data Sequential Sorting Algorithm: Quick sort, bubble sort, merge sort, heap sort, selection sort, etc

  16. Algorithm • Each processor is either a merger or sender of data • Keep track of distance (step) between merger and sender on each iteration • double step each time • Merger rank must be a multiple of 2*step • Sender rank must be merger rank + step • If no sender of that rank then potential merger does nothing • Otherwise must be a sender • send data to merger on left • at sender rank - step • terminate • Finished, root print out the result

  17. Example Output $ mpirun -np 5 qsort 0 about to broadcast 20000 0 about to scatter 0 sorts 20000 1 sorts 20000 2 sorts 20000 3 sorts 20000 4 sorts 20000 step 1: 1 sends 20000 to 0 step 1: 0 gets 20000 from 1 step 1: 0 now has 40000 step 1: 3 sends 20000 to 2 step 1: 2 gets 20000 from 3 step 1: 2 now has 40000 step 2: 2 sends 40000 to 0 step 2: 0 gets 40000 from 2 step 2: 0 now has 80000 step 4: 4 sends 20000 to 0 step 4: 0 gets 20000 from 4 step 4: 0 now has 100000

  18. Quick Sort • The quick sort is an in-place, divide-and-conquer, massively recursive sort. • Divide and Conquer Algorithms • Algorithms that solve (conquer) problems by dividing them into smaller sub-problems until the problem is so small that it is trivially solved. • In Place • In place sorting algorithms don't require additional temporary space to store elements as they sort; they use the space originally occupied by the elements. Reference http://ciips.ee.uwa.edu.au/~morris/Year2/PLDS210/qsort.html Source http://www.sci.hkbu.edu.hk/tdgc/tutorial/ExpClusterComp/qsort/qsort.c

  19. Quick Sort • The recursive algorithm consists of four steps (which closely resemble the merge sort): • If there are one or less elements in the array to be sorted, return immediately. • Pick an element in the array to serve as a "pivot" point. (Usually the left-most element in the array is used.) • Split the array into two parts - one with elements larger than the pivot and the other with elements smaller than the pivot. • Recursively repeat the algorithm for both halves of the original array.

  20. Quick Sort • The efficiency of the algorithm is majorly impacted by which element is chosen as the pivot point. • The worst-case efficiency of the quick sort, O(n2), occurs when the list is sorted and the left-most element is chosen. • If the data to be sorted isn't random, randomly choosing a pivot point is recommended. As long as the pivot point is chosen randomly, the quick sort has an algorithmic complexity of O(n log n). Pros: Extremely fast. Cons: Very complex algorithm, massively recursive.

  21. Quick Sort Performance

  22. Quick Sort Speedup

  23. Discussion • Quicksort takes time proportional to N*N for N data items • for 1,000,000 items, Nlog2N ~ 1,000,000*20 • Constant communication cost – 2*N data items • for 1,000,000 must send/receive 2*1,000,000 from/to root • In general, processing/communication proportional to N*log2N/2*N = log2N/2 • so for 1,000,000 items, only 20/2 =10 times as much processing as communication • Suggests can only get speedup, with this parallelization, for very large N

  24. Bubble Sort • The bubble sort is the oldest and simplest sort in use. Unfortunately, it's also the slowest. • The bubble sort works by comparing each item in the list with the item next to it, and swapping them if required. • The algorithm repeats this process until it makes a pass all the way through the list without swapping any items (in other words, all items are in the correct order). • This causes larger values to "bubble" to the end of the list while smaller values "sink" towards the beginning of the list.

  25. Bubble Sort The bubble sort is generally considered to be the most inefficient sorting algorithm in common usage. Under best-case conditions (the list is already sorted), the bubble sort can approach a constant O(n) level of complexity. General-case is O(n2). Pros: Simplicity and ease of implementation. Cons: Horribly inefficient. Reference http://math.hws.edu/TMCM/java/xSortLab/ Source http://www.sci.hkbu.edu.hk/tdgc/tutorial/ExpClusterComp/sorting/bubblesort.c

  26. Bubble Sort Performance

  27. Bubble Sort Speedup

  28. Discussion • Bubble sort takes time proportional to N*N/2 for N data items • This parallelization splits N data items into N/P so time on one of the P processors now proportional to (N/P*N/P)/2 • i.e. have reduced time by a factor of P*P! • Bubble sort is much slower than quick sort! • better to run quick sort on single processor than bubble sort on many processors!

  29. Merge Sort • The merge sort splits the list to be sorted into two equal halves, and places them in separate arrays. • Each array is recursively sorted, and then merged back together to form the final sorted list. • Like most recursive sorts, the merge sort has an algorithmic complexity of O(n log n). • Elementary implementations of the merge sort make use of three arrays - one for each half of the data set and one to store the sorted list in. The below algorithm merges the arrays in-place, so only two arrays are required. There are non-recursive versions of the merge sort, but they don't yield any significant performance enhancement over the recursive algorithm on most machines.

  30. Merge Sort Pros: Marginally faster than the heap sort for larger sets. Cons: At least twice the memory requirements of the other sorts; recursive. Reference http://math.hws.edu/TMCM/java/xSortLab/ Source http://www.sci.hkbu.edu.hk/tdgc/tutorial/ExpClusterComp/sorting/mergesort.c

  31. Heap Sort • The heap sort is the slowest of the O(n log n) sorting algorithms, but unlike the merge and quick sorts it doesn't require massive recursion or multiple arrays to work. This makes it the most attractive option for very large data sets of millions of items. • The heap sort works as it name suggests • It begins by building a heap out of the data set, • Then removing the largest item and placing it at the end of the sorted array. • After removing the largest item, it reconstructs the heap and removes the largest remaining item and places it in the next open position from the end of the sorted array. • This is repeated until there are no items left in the heap and the sorted array is full. Elementary implementations require two arrays - one to hold the heap and the other to hold the sorted elements.

  32. Heap Sort To do an in-place sort and save the space the second array would require, the algorithm below "cheats" by using the same array to store both the heap and the sorted array. Whenever an item is removed from the heap, it frees up a space at the end of the array that the removed item can be placed in. Pros: In-place and non-recursive, making it a good choice for extremely large data sets. Cons: Slower than the merge and quick sorts. Reference http://ciips.ee.uwa.edu.au/~morris/Year2/PLDS210/heapsort.html Source http://www.sci.hkbu.edu.hk/tdgc/tutorial/ExpClusterComp/heapsort/heapsort.c

  33. End

More Related