Complexity Analysis

1. Complexity Analysis

2. Basics • Ordo, Omega, Theta • Express the maximum, minimum and average complexity • The complexity of function f(x) can be expressed by using one of the above terms

4. Examples a) T(n) = (n+1)2, O(n2) Choose constant c=4. Examine if 4n2≥ (n+1) 2. Examination reveals that starting from value 1 the argument holds, therefore T(n) = O(n2), while n ≥ 1 b) T(n) = 3n3 + 2n2, Ω(n3) Choose c=1. Then 3n3 + 2n2≥ n3, i.e. T(n) = Ω(n3)

5. procedure bubble (int A[ 1...n] ) { int i,j,temp;     for i=1 to n-1 {         for j = n downto i+1 {             if A[ j-1] > A[ j] {             temp = A[ j-1] ; A[ j-1] = A[ j] ; A[ j] = temp;         }     } }

6. What about parallel • Calculation divided by number of processors • Algorithms usually contain both calculation and communication • Shared memory approach • More congestion in the bus • Distributed memory approach • Message passing over network

7. Result • Good • Calculation f(n / p) • Might be n2 / p, nn / p, optimal n=p • Bad • Communication m * p • If p = n, communication cost high

9. Typical communication • Master / Slave • Distribute / collect data (One-to-one sends, broadcast etc) • Shifting • One-to-one communication between neighbouring nodes • Collective communication • Scatter, Gather etc

10. Result • Complexity of the program not by one f(n) • Rather Ttotal = Tcalc (n/p) + Tcomm (p) • Complexity max {Tcalc, Tcomm}

11. procedure sum( int A[0..n-1] ) { private int i, id; global int sum; for (i = id*n/p ; i < (id+1)*n/p ; i++) { sum = sum + A[i]; } }

12. procedure sum_master( int A[0..n-1] ) { int sum; for (i = 0 ; i < p ; i++) { send( A[i*p .. (i+1)*p , i+1) } for (i = 0 ; i < p ; i++) { recv( tempsum , i+1); sum = sum + tempsum } }

13. procedure sum_slave() { int A[n/p], localsum; recv ( A[] , master ) for (i = 0 ; i < n/p ; i++) { localsum = localsum + A[i]; } send ( localsum, master ) }