1 / 20

References

„Introduction to Algorithms“. References. Cormen/Leiserson/Rivest/Stein: Introduction to Algorithms Knuth: The Art of Computer Programming Sipser: Introduction to the Theory of Computation. Course page http://kaist.theoryofcomputation.asia/18cs300

leahm
Download Presentation

References

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. „Introduction to Algorithms“ References • Cormen/Leiserson/Rivest/Stein: Introduction to Algorithms • Knuth: The Art of Computer Programming • Sipser: Introduction to the Theory of Computation Course page http://kaist.theoryofcomputation.asia/18cs300 E-Learning: http://klms.kaist.ac.kr Homework: weekly assignments, individually, hand-written; random grading + programming in ELICEhttp://kaist.elice.io/ Pledge for integrity and academic honesty to be signed and submitted! No email requests, please!

  2. „Introduction to Algorithms“ Syllabus 1. Computational problems and algorithmic solutions • Virtues of Computer Science • Specification • Algorithm design • Efficiency analysis • Optimality, Example: Repeated Squaring • Effect of Different Primitives: Fibonacci Numbers • Recurrences and the Master Theorem • Polynomial Multiplication: Long, Karatsuba, Toom, Cook • Matrix Multiplication: open problem

  3. 1. Computational problems and algorithmic solutions “Virtues” of ComputerScience • problem specification • formal semantics • algorithm design • and analysis (correctness, efficiency) • optimality proof

  4. 1. Computational problems and algorithmic solutions specification • Find me a girlfriend/boyfriend. • Does God exist? • How many angels can dance on the head of a pin? • What is the best movie of all time? • How many twin primes are there? • Is „no“ the only correct answer to this question? • What is the smallest positive integer that cannot be described in English using 100 characters?

  5. 1. Computational problems and algorithmic solutions semantics • Find me a girlfriend/boyfriend. • Does God exist? • How many angels can dance on the head of a pin? • What is the best movie of all time? Kiss your seat neighbor! Yes! Try! Watch „Carmencita“ (1894) ; let B:=1 For each N=2,…8880980 do Compare movie #Nin IMDb to movie #B: If you like #N better than #B, let B:=N. Return B.

  6. 1. Computational problems and algorithmic solutions algorithm vs. code mov esi, offset list top: mov edi, esi inner: mov eax, [edi] mov edx, [edi+4] cmp eax, edx jle no_swap mov [edi+4], eax mov [edi], edx no_swap: add edi, 4 cmp edi, list_end - 4 jb inner add esi, 4 cmp esi, list_end - 4 jb top • primitive operations • their semantics • their costs # vowels list vowels = ['e', 'a', 'u', 'o', 'i'] # sort the vowels vowels.sort() # print vowels print('Sorted list:', vowels)

  7. 1. Computational problems and algorithmic solutions pseudo-code bubbleSort ( A[n] ) while n≠0 do newn := 0 for i := 1 to n-1 do if A[i-1] > A[i] then swap(A[i-1], A[i]) newn := i endif endfor n := newn endwhile • primitive operations • their semantics • their costs „Premature optimization is the root of all evil“ (D.Knuth)

  8. 1. Computational problems and algorithmic solutions asymptoticefficiency • Algorithm analysis, as opposed to empirical program evaluation „Premature optimization is the root of all evil“ (D.Knuth)

  9. 1. Computational problems and algorithmic solutions asymptotic growth Landau Big-Oh notation f = O(g) • constant 17 • logarithmic log(n) • square root √n • linear n • quasilinear n·log(n) • quadratic n² • cubic n³ • polynomial c·nc • superpolynomial nlog(n) • exponential 2n • doubly exponential 22n • tetration 22…2ⁿ

  10. 1. Computational problems and algorithmic solutions Optimality Example Powering Problem: Fix some monoid M. Input:XM and nN. Output:Xn. Cost: #multiplications • Xn = X· X · … · X : n-1 multiplications • Recursive algorithm: compute • Xn= (X n/2)2 if n even • Xn= (X(n-1)/2)2 · Xif n odd • #multiplications T(1) =0, T(n)  T(n/2)+2, • Proof by induction: T(n)  2·log2(n) T(n)  T(n/2)+1 T(n)  T(n/2)+2

  11. 1. Computational problems and algorithmic solutions Optimality Example Powering Problem: Fix some monoid M. Input:XM and nN. Output:Xn. Cost: #multiplications Xn = en·ln(X) 3 operations! Devised & analysed algorithm using ≤2log2n multiplications Lemma a) Any algorithm starting with input X, and using only multiplications, produces only monomials in X as (intermediate) results. b) After ℓ multiplications, all intermediate results (monomials) have degree n≤2ℓ. So computing Xn requires ℓ≥log2n multiplications.

  12. 1. Computational problems and algorithmic solutions Example: FibonacciNumbers F0=0 F1=1 Fn = Fn-1 + Fn-2 FibIter(n) if n=0 return 0; fib := 1; fibL := 0; while n>1 do tmp:=fibL; fibL := fib; fib := fibL+tmp; n := n-1 ; end return fib; ≥ 2n/2 −1, n≥1 FibRek(n) if n=0 return 0; if n=1 return 1; return FibRek(n-1)+FibRek(n-2); k = · · = k := n-1

  13. 1. Computational problems and algorithmic solutions Example: FibonacciNumbers F0=0 F1=1 Fn = Fn-1 + Fn-2 φ := (1+√5)/2Golden Ratio ≥ 2n/2 −1, n≥1 Prove by induction: Fn=(φn-(-1/φ)n) / √5 φn = exp(n·ln φ) Five different algorithms, from exponential to constant “time”! k = · · = k := n-1

  14. 1. Computational problems and algorithmic solutions recurrences Master Theorem: Let f:N→R be increasing and satisfy the recurrence relation f(n) = a · f(n/b) + O(nd) whenever n is an integral power of b, where a,b≥1 are integers and d>0. Then

  15. 1. Computational problems and algorithmic solutions LongMultiplication Input: coefficients of polynomials A(x)=a0+a1x+a2x²+…+aN-1xN-1 and B(x) of degree <N. Output: coefficients of polynomial C(x) := A(x)· B(x) of degree ≤K:=2N-2. T(N):=#arithmetic operations (multiplications, linear combinat.s) w.l.o.g. 2|N e.g.(-5) or + Recursive algorithm, Distributive law: T(N) = 4· T(N/2) + O(N) "Naîve“ ck= j aj·bk-j (A0(x)+ A1(x)·xN/2) · (B0(x)+ B1(x)·xN/2) = C0(x) + C1(x) · xN/2+ C2(x) · xN C1(x)= A0(x)·B1(x) + A1(x)·B0(x) C2(x)= A1(x)·B1(x) C0(x)= A0(x)·B0(x) C(x)= c0 ,… cN/2-1 cN/2 ,… cN-1 cN-1 ,… c3/2·N-1 c3/2·N-1 ,… c2N-2

  16. 1. Computational problems and algorithmic solutions KaratsubaMultiplication Input: coefficients of polynomials A(x)=a0+a1x+a2x²+…+aN-1xN-1 and B(x) of degree <N. Output: coefficients of polynomial C(x) := A(x)· B(x) of degree ≤K:=2N-2. T(N):=#arithmetic operations (multiplications, linear combinat.s) w.l.o.g. 2|N e.g.(-5) or + Recursive algorithm, based on: Distributive law: T(N) = 4· T(N/2) + O(N) „Karatsuba law“: T(N) = 3· T(N/2) + O(N) (A0(x)+ A1(x)·xN/2) · (B0(x)+ B1(x)·xN/2) = C0(x) + C1(x) · xN/2+ C2(x) · xN C0(x) := A0(x) · B0(x) , C2(x) := A1(x) · B1(x) C1(x) := (A0(x) + A1(x) ) · ( B0(x) + B1(x) )−C0(x) −C2(x)

  17. 1. Computational problems and algorithmic solutions ToomMultiplication o T1 := (A0+2A1+4A2) · (B0+2B1+4B2) T2 := (A0 + A1 + A2) · (B0 + B1 + B2) T3 := (4A0+2A1+A2) · (4B0+2B1+B2) o o w.l.o.g. 3|N C3 = −C0 + ⅓T1−2T2 + ⅙T3 − 3½C4 C2 = 3½C0 −½T1 +5T2 −½T3 + 3½C4 C1 = −3½C0 + ⅙T1 −2T2 +⅓T3 −C4 C0 = A0 · B0 , C4 = A2 · B2 Distributive law: T(N) = 4· T(N/2) + O(N) o o „Karatsuba law“: T(N) = 3· T(N/2) + O(N) „Toom‘s law“: T(N) =5· T(N/3) + O(N) (A0(x)+A1(x)·xN/3 + A2(x)·x2N/3)(B0(x)+B1(x)·xN/3 +B2(x)·x2N/3) = C0(x) + C1(x)·xN/3 + C2(x)·x2N/3 + C3(x)·x3N/3 + C4(x)·x4N/3

  18. A(x)·B(x) A(x)·B(x) | : | A(x)·B(x) A(x)·B(x) A(x)·B(x) | : | A(x)·B(x) 0 0 0 0 1 xx² … xK 1 xx²… xK   :  1 xx² … xK 1 xx² … xK 1 xx² … xK   :  1 x x² … xK 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 K k k K K k k K k K 1. Computational problems and algorithmic solutions Toom-Cook Input: coeff. of A(x)=a0+a1x+a2x²+… of deg<N,B(x) of deg<M Output: coeffients c0,…cK of C(x) := A(x) · B(x), deg(C)≤K:= N+M-2 Long Multiplication: N·Mproducts & linear combinations -1 A(x)· B(x) = C(x) = (1 x x²… xK) Vandermonde Matrix invertible for any distinct fixedx0,…xK c0 c1  | : | cK  = =   Evaluation/Interpolation: K+1products, O(N²+M²)linear comb.s T(N,M) = (n+m-1) · T(N/n,M/m) + O((N+M)·(n+m)) (A0(x) + A1(x)·xN/n+ A2(x)·x2N/n+ ...+ An-1(x)·x(n-1)·N/n)(B0(x) + B1(x)·xM/m+ B2(x)·x2M/m+ ...+ Bm-1(x)·x(m-1)·M/m)

  19. 1. Computational problems and algorithmic solutions Input: entries of nn-matrices A,B Output: entries of C := A +B A1,1 A1,2 B1,1 B1,2 C1,1 C1,2 · = A2,1 A2,2 B2,1 B2,2 C2,1 C2,2 matrix multiplication T(n) = #arithmetic operations  O(n²) O(n³) +18 lin.comb.s 7 multiplications of (n/2)(n/2)-matrizes T1:=(A2,1+A2,2)·B1,1 T2:=(A1,1+A1,2)·B2,2 T3:=A1,1·(B1,2-B2,2) T4:=A2,2·(B2,1-B1,1) T5:=(A1,1+A2,2)·(B1,1+B2,2) T6:=(A2,1-A1,1)·(B1,1+B1,2) T7:=(A1,2-A2,2)·(B2,1+B2,2) C1,1=T5+T4-T2+T7 , C1,2=T3+T2 C2,1=T1+T4 , C2,2=T5-T1+T3+T6 T(n) = 7·T(n/2) +18·(n/2)² = O(nlog27) World record:O(n2.37) [F.Le Gall'14]

  20. 1. Computational problems and algorithmic solutions pedagogy This course focuses on understanding, less about knowledge! Bloom's Taxonomy

More Related