290 likes | 425 Views
This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License . CS 312: Algorithm Analysis. Lecture #13: Strassen’s Algorithm for Matrix Multiplication; Convolution. Slides by: Eric Ringger, with contributions from Mike Jones, Eric Mercer, Sean Warnick.
E N D
This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License. CS 312: Algorithm Analysis Lecture #13: Strassen’s Algorithm for Matrix Multiplication; Convolution Slides by: Eric Ringger, with contributions from Mike Jones, Eric Mercer, Sean Warnick
Announcements • Homework #8.5 • Due Now • Project #3 • You sent Dr. Ringger your chosen theorem, right?
Objectives • Apply divide and conquer to matrix multiplication • Analyze using the Master Theorem • Introduce you to the “convolution” operation. • See an algorithm for convolution, inspired by polynomial multiplication
Matrix Multiplication • Who cares? • In this setting, • Scalar operations +, –, and x are elementary ops • Assume fixed precision • Matrix operations +, -, and x are NOT elementary ops • What about the commutative property of matrix multiplication?
Naïve Algorithm 5 6 7 8 1 2 3 4 = x
Strassen’s Algorithm e f g h m2+m3 m1+m2+m5+m6 m1+m2+m4-m7 m1+m2+m4+m5 a b c d = x m1 = (c + d - a) x (h – f + e) m2 = (a x e) m3 = (b x g) m4 = (a - c) x (h - f) m5 = (c + d) x (f - e) m6 = (b - c + a - d) x h m7 = d x (e + h – f - g) • Note: this solution was obtained in a manner similar to the Karatsuba Divide and Conquer algorithm for scalar multiplication
Strassen’s on our Example 5 6 7 8 m2+m3 m1+m2+m5+m6 m1+m2+m4-m7 m1+m2+m4+m5 1 2 3 4 = x 5 + 14 42+5+7+(-32) 42+5+(-4)-0 42+5+(-4)+7 19 22 43 50 = = m1 = (3+4-1) x (8-6+5) = 6 x 7 = 42 m2 = (1 x 5) = 5 m3 = (2 x 7) = 14 m4 = (1-3) x (8-6) = -2 x 2 = -4 m5 = (3+4) x (6-5) = 7 x 1 = 7 m6 = (2-3 + 1-4) x 8 = -4 x 8 = -32 m7 = 4 x (5+8-6-7) = 4x0 = 0
Recursive Application A B E F x G H C D Divide each matrix into fourths, and apply Strassen’s algorithm. Note: divide & conquer without Strassen’s algebraic insight is still
Again: Strassen’s Algorithm A B E F M2+M3 M1+M2+M5+M6 M1+M2+M4-M7 M1+M2+M4+M5 = x C D G H What’s the efficiency? M1 = (C + D - A) x (H – F + E) M2 = (A x E) M3 = (B x G) M4 = (A - C) x (H - F) M5 = (C + D) x (F - E) M6 = (B - C + A - D) x H M7 = D x (E + H – F - G) • 7 recursive calls • Subproblem matrices are half original width • 24 matrix additions to divide/merge Recall: Matrix addition (and subtraction) is
Efficiency Analysis a = number of subinstances n = original instance size n/b = size of subinstances d = polynomial order of g(n) where g(n) is cost of doing the divide (& combine) Therefore:
Can we do better? • Divide matrix width by b=2: • Naïve: multiplications • Only 7 multiplications • Strassen: O(n2.81) • Divide matrix width by b=3: • Naïve: multiplications • Only 21 multiplications • 1979: O(n2.521813) • 1980: O(n2.521801) • Divide matrix width by b=70: • Naïve: • Only 143,640 multiplications (Pan) • 1986: O(n2.376)
Note • Divide and conquer without Strassen’s algebraic insight:
Is it correct? What does it mean for Strassen’s to be correct? How can you prove it in general?
Correctness e f g h m2+m3 m1+m2+m5+m6 m1+m2+m4-m7 m1+m2+m4+m5 a b c d = x m1 = (c + d - a) x (h – f + e) m2 = (a x e) m3 = (b x g) m4 = (a - c) x (h - f) m5 = (c + d) x (f - e) m6 = (b - c + a - d) x h m7 = d x (e + h – f - g)
Polynomial Multiplication • Example: • What can you say about the degree of the product of two degree polynomials? • What will the coefficients of C look like?
Polynomial Multiplication is Convolution • More generally: • Then: • Where: for all • For , use and
Questions • Is it Correct? • How long does it take (assuming multiplication and additional are elementary ops)? • Can we do better? To find this we treated the math on the preceding slide as terse pseudocode, and reasoned that there are order constants , and calculating each requires up to additions.
More Questions • Why is convolution interesting? • Where else is it used? • Can we generalize the idea of a sequence of coefficients on a polynomial?
Signals Signal: An impulse or a fluctuating quantity, such as voltage, current, or EM field strength, whose variations represent some sort of information.
Signals • Naturally Occurring: • Light / Electromagnetic radiation • Sound (pressure) waves (e.g., in air or water) • Man-made: Encoded information • In coax, CAT-5, etc. • AM/FM Radio broadcast • Radar pulses • Mathematically: Represented by a function of one or more independent variables.
Signals Speech: acoustic pressure as a function of time,digitized as sequence of samples f(t) t
1-D Convolution = Noise filtering!
Signals Image: brightness as a function of 2 spatial variables x f(x,y) y
-1 0 1 -1 0 1 -1 0 1 2-D Convolution • A 2-D signal (an Image) is convolved with a second Image (the filter, or convolution “Kernel”). =
Assignment Homework: HW #9 • Do the upper right part of the proof of correctness for Strassen’s algorithm • Some more probability exercises Read Section 2.6 on the FFT! • Probably the most famous Divide and Conquer algorithm