1 / 88

Chapter 2

Chapter 2. The Complexity of Algorithms and the Lower Bounds of Problems. S. J. Shyu. Main Themes. （ 1 ） the goodness of algorithms （ 2 ） the difficulty of a problem （ 3 ） an optimal algorithm for a problem. 2.1 Time Complexity of an algorithm. Time vs. Space

uri
Download Presentation

Chapter 2

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 2 The Complexity of Algorithms and the Lower Bounds of Problems S. J. Shyu

  2. Main Themes （1） the goodness of algorithms （2） the difficulty of a problem （3） an optimal algorithm for a problem

  3. 2.1 Time Complexity of an algorithm • Time vs. Space • How to measure the time-complexity of an algorithm？ • Choose a particular step • Summation — # of additions, • Sorting — comparison data movement, • Matrix multiplication — # of multiplication, ... • Measure the number of steps in terms of the size of the problem

  4. c∣g(n)∣ ∣f(n)∣ f(n) n n0 Big-O Def. f(n)＝ O(g(n)) if and only if there exist two positive constants c and n0 such that ︱f(n)︱c︱g(n)︱ for all n  n0. 當 n 大到一個程度 (nn0) 後，f(n)會被g(n)的某個constant倍bound住！

  5. Time Complexity Functions vs. Problem size

  6. Rate of growth of common computing time functions

  7. Common computing time functions • O(1) O(log n) O(n) O(n log n) O(n2) O(n3) O(2n) O(n!) O(nn) • polynomial algorithms • exponential algorithms

  8. 2.2 The Best-, Average- and Worst-Case analysis of algorithms • Best case • Worst case • Average case

  9. input: 7,5,1,4,3,2,6 7,5,1,4,3,2,6 5,7,1,4,3,2,6 1,5,7,4,3,2,6 1,4,5,7,3,2,6 1,3,4,5,7,2,6 1,2,3,4,5,7,6 1,2,3,4,5,6,7 Ex.2.1 Insertion Sort (example) 觀察 data movement 用 inversion table來計算 data movement 的次數。 原序列: 7 5 1 4 3 2 6 Inversion 2 4 3 2 1 1 0 table ( 1 之前有 2 個數比 1 大， 2 之前有 4 個數比 1 大，… ) 將數列的值加總即為 data movement 的次數！

  10. Ex.2.1 Insertion Sort (Algorithm) • Input：x1, x2, x3, …, xn • Output： The sorted sequence of x1 ,x2 ,x3 , …,xn for j := 2 to n do begin i := j-1 x:= xj while (x＜xiand i＞0) do begin xi+1:=xj i:= i-1 end xi+1 := x end

  11. xj = 3 xi = 6 xi = 4 xi = 2 ex. x1x2 …xj1xj 1 2 4 63 ↘ 1 2 4 6 3 ↘ 1 2 4 6 3 1 2 3 4 6 觀察此 algorithm，其評估的重點在於 — data movement ﹙或 comparison﹚ 凡比xj大者就往右移一位！ xi < xj遂將 xj放在位置 j+1 之處

  12. Ex.2.1 Insertion Sort ( Analysis – best & worst case) Best case ＝ 0 ex. instance: 1 2 3 4 5 6 7 Worst case ｛n-1, n-2, ..., 1, 0｝ex. instance: 7 6 5 4 3 2 1 ＝ d1 = n 1 d2 = n 2 : di = ni dn = 0

  13. Ex.2.1 Insertion Sort( Analysis - average case) Average case :Suppose that x1, x2, …, xj-1, are sorted. , xj 是 x1～ xj 中第□大 1 2 3 … j steps needed 2 3 4 … j+1 probability … 於是將xj insert 至 x1, x2, …, xj-1中平均steps: 2 + 3 + ... +(j+1) ＝ 而 j從 2 ~ n都得執行，所以總共步驟為： = = (n+8)(n-1) = O(n2)

  14. Ex.2.2 The binary Search (Example) • Sorted sequence : (search 9) 1 4 5 7 9 10 12 15 Step 1  Step 2  Step 3 

  15. Ex.2.2 The binary Search (Algorithm) Input：a1, a2, … ,an , n > 0, with a1a2 … an, and x Output： j if aj= X and 0 if no j exists such that aj = X. i := 1 m := n while (im) do begin j := (i+m)/2 if (x＝ aj) then output j & stop if (x < aj) then m := j1 else i := j+1 end j := 0 output j

  16. Ex.2.2 The binary Search (Example) ← step 1 <= step 2 ＜≡ step 3 ＜ step 4 1 ← i x＝ 10 2 3 4 5 → 6 j 7 <=i 8 => 9 j 10 ＜≡i＜ i,j,m≡> 11 j 12 ← m<= m＜≡ m

  17. Ex.2.2 The binary Search (Analysis) 具「決定性」，足以評估方法優劣的步驟是 comparison！ • best case: 1 step = O(1) • worst case: (log2 n+1) steps = O(log n) • average case: 所有的情況：

  18. Ex.2.2 The binary Search (Analysis- Average case)＊ 找得到的情況： 計有 1 個情況，是找了 1 次即得 2 2 4 3 ︰ : 2lognlogn + 1

  19. Ex.2.2 The binary Search (Analysis- Average case)＊ 找不到的情況： 在 (n+1) 種情況裡，每一種都得找 logn + 1 次方可確定。 ∴ 平均 “找”的次數 A(n)＝ （令 ＝ logn + 1） 利用歸納法 (induction) 可得： A(n) < k＝ Ｏ( logn )

  20. n cases for successful search • n+1 cases for unsuccessful search • Average # of comparisons done in the binary tree: • A(n) = , where k = log n+1

  21. Ex 2.3 Straight Selection Sort (Example) 7 5 1 4 3 1 5 7 4 3 1 3 7 4 5 1 3 4 7 5 1 3 4 5 7

  22. Ex 2.3 Straight Selection Sort (Algorithm) • Input: a1,a2, …,an. • Output: The sorted sequence of a1,a2, …,an. For j :=1 to n1 do Begin f:= j For k := j+1 to n do If ak < afthen f:= k ajaf End

  23. Ex 2.3 Straight Selection Sort (Analysis) • We consider the # of changes of the flag which is used for selecting the smallest number in each iteration. • best case: O(1) • worst case: O(n2) • average case: O(n log n)

  24. Ex 2.4 Quicksort (Example) ex. x = 6 6 4 3 7 1 2 5 8 9 5 4 3 7 1 2 6 8 9 5 4 3 6 1 2 7 8 9 5 4 3 2 1 6 7 8 9 j i j i j i <6 >6 • Recursively apply the same procedure.

  25. Ex 2.4 Quicksort(f, l) (Algorithm) Input：af, af+1, …, al Output：The sorted sequence of af, af+1, …, al If (fl) then return x:=af； i:=f；j:=l While (i<j) do begin while af  x do j:=j 1 ai ←→ aj while ai x do i:=i+1 ai ←→ aj end Quicksort(f, j-1) Quicksort(j+1, l)

  26. Ex 2.4 Quicksort ( Analysis Best Case) 每次都能”均分 ”input之數列 比較的次數 n (n/2)2 (n/4)4 . . . 數列能切logn 次，而每次都比較 n次：O(nlogn)

  27. Ex 2.4 Quicksort ( AnalysisWorst Case) 比較的次數 n n1 n2 . . . (原數列已sorted，或reversely sorted) 這種數列得進行 n 次，

  28. Ex 2.4 Quicksort ( AnalysisAverage Case) 可能的”分割 ”方式： 每次分割均花n次comp n n n n T(n)＝Ave（T(S)＋T(n-S)）＋cn => O(nlogn) 1 S  n

  29. Harmonic number T(n) n

  30. D B C A E Ex. 2.5 The Problem of Rank Finding 2-D ranking finding • Def. Given two points A＝(a1, a2) and B＝(b1, b2). A dominates B iff ai＞bi , i=1,2 (a1> b1 and a2 > b2) • Def. Given a set S of n points, the rank of a point x is the number of points dominated by x. ＊一個直覺的解法： 每二點的pair都分別找 dominate的關係 => O(n2) rank(A)= 0 rank(B) = 1 rank(C) = 1 rank(D) = 3 rank(E) = 0

  31. Ex. 2.5 Rank Finding試試 Divide & Conquer 動動腦

  32. Ex. 2.5 A Rank Finding Algorithm Input： A set S of planar point p1,p2,...pn Output： The rank of every point in S 若S只有1點，則return rank = 0 Step1. 以垂直X軸的直線L，將平面切成兩半， 使左右各含 n/2個點 (右半X座標＞L，左半X座標＜L) Step2. Recursively 用本方法求出A，B內各點之 rank Step3. 將A，B內點依y座標大小排序，對B內每一點 都查是否有A中的點，其y座標小於自己的y值 rank即自己的加上A中比自己小的點個數。

  33. Straightforward algorithm: Compare all pairs of points : O(n2)

  34. Ex. 2.5 Rank FindingTime-complexity analysis Step1. 求median：O(n) Step2. 計算A，B的rank，recursive Step3. sorting & scanning：O(nlogn) & O(n) T(n) 2T(n/2)＋c1．nlogn＋c2．n設n＝2P  2T(n/2)＋c．nlogn  2(2T(n/4))＋c(n/2)log(n/2)＋cnlogn = 4T(n/4)＋cnlog(n/2)＋cnlogn nT(1)＋c(nlogn＋nlog(n/2)＋nlog(n/4)＋...＋nlog2) ＝ nT(1)＋cn（((1＋logn)/2)logn） ＝ c'n＋cn/2logn＋cn/2log2n ＝ O(nlog2n) 比 check all pairs 好！ …

  35. 2.3 The Lower Bound of a Problem 為什麼要找 Lower Bound • 利用對algorithm的time-complexity analysis可以評判algorithm的好壞！ • 是不是也可以用time-complexity 分析problem本身的難易度 (difficulty)？ (yes！) • 若一個problem有lower time-complexity 的algorithm，它就是easy，否則該problem是difficult。 這樣的觀察將引導我們想到該問題是否有 lower bound？

  36. Lower bound • Def : A lower bound of a problem is the least time complexity required for any algorithm which can be used to solve this problem. • ☆worst case lower bound ☆average case lower bound •  The lower bound for a problem is not unique. • e.g. (1), (n), (n log n) are all lower bounds for sorting. • ((1), (n) are trivial)

  37. Lower boundΩ • Def. f(n)＝Ω(g(n))if and only if there exists positive constant c and n0 , such that f(n)  c|g(n)| for all n ＞ n0

  38. trivial lowerbound • ex.: sorting Ω(1),Ω(n)均為trivial lowerbound，討論它們沒有意義！ Ω(n2)如何？已有heapsort 其worst case為Ω(nlogn)由 Def. 可知 lower bound 必須是所有 algorithms 中最小者，所以 Ω(n2) 也不對！lower bound 至多是Ω(nlogn)。

  39. 若目前 problem 之 highest lower bound 為 Ω(nlogn) 而找到的algorithm 最快的是O(n2)，則： • 若問題的 lower bound 為Ω(nlogn)且找到的 algorithm 的 time-complexity 為 O(nlogn) 則 optimal algorithm of this problem 即已找到！ lower bound 與 algorithm 都無法再 improve。

  40. 既然在「所有」 algorithms 中挑最少不可能，那麼設法把problem 轉換了一種替代的形式，而該形式「蘊藏 」著該 problem 至少要花的代價。

  41. Lower Bound VS Algorithm (upperbound) • 希望能找到解決 problem P的optimal algorithm A。但在 A尚未找到之前，目前可解P 的algorithm 中，若time-complexity 最少的為 Ai，如何確定 Ai 是最好的方法呢？ • 証明 P的 lower bound, LB，即為 Ai的追尋定出下界。 我們找到的 algorithm Ai‘的 complexity ；而我們能証得的 LB（凡合理要花的成本愈高愈有兩者重合之時，就是 optimal algorithm 誕生之日！

  42. Lower Bound觀念整理-1 • An algorithm is optimal if its time-complexity is equel to a known found lower bound of this problem. • 一個問題(problem)的lower bound是所有可解該問題的algorithm中time-complexity最少者。 • (由於無法舉証「所有」解該問題algorithm中最少者，我們都試圖「証明」該問題的lower bound愈高愈好！

  43. Lower Bound觀念整理-2 • 若目前得到的lower bound低於找得algorithm中time-complexity最少者，則可能： (1)lower bound可以提高 (2)algorithm可以改進，使time-complexity降低 (3) (1)與(2)同時進行。 • 若已知的lower bound恰與已得algorithm的time-complexity相同，(兩者均無法再improve)則此algorithm正為optimal也；此lower bound亦為 the highest lower bound(the truly significant one.)

  44. 2.4 The Worst Case Lower Bound of Sorting • 如何找到sorting的lower bound？ • 在所有解sorting的algorithm中找最少？ • 將sorting轉成另一種形式， 其LB可掌握！

  45. Sorting 中的compare ＆ exchange可以容易地以binary decision tree表示 整個binary tree中最長的path即對應該 sorting algorithm 之 worst case time-complexity

  46. Decision tree for bubble sort

  47. 觀察並思考下面描述： （1）不管那一種sorting algorithm，它要面對 input instance的種類皆有n!；即對應 binary decision tree之leaf node皆有n!個。 （2）不管binary decision tree長相為何，唯在 balanced tree上方有最少的depth。 （3）若binary tree 是balanced，則其depth為 [logX]，X為leaf node個數。 • sorting problem 的 lower bound為 [logn!] ∵ [logn!]＝[nlogn]。 ∴ sorting 所需最少comparison的數目為Ω(nlogn)， in the worst case.

  48. Lower bound of sorting • To find the lower bound, we have to find the smallest depth of a binary tree. • n! distinct permutations n! leaf nodes in the binary decision tree. • balanced tree has the smallest depth: log(n!) = (n log n) lower bound for sorting: (n log n) (See the next page.)

More Related