1 / 29

CS 312: Algorithm Analysis

This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License . CS 312: Algorithm Analysis. Lecture #13: Strassen’s Algorithm for Matrix Multiplication; Convolution. Slides by: Eric Ringger, with contributions from Mike Jones, Eric Mercer, Sean Warnick.

gavivi
Download Presentation

CS 312: Algorithm Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License. CS 312: Algorithm Analysis Lecture #13: Strassen’s Algorithm for Matrix Multiplication; Convolution Slides by: Eric Ringger, with contributions from Mike Jones, Eric Mercer, Sean Warnick

  2. Announcements • Homework #8.5 • Due Now • Project #3 • You sent Dr. Ringger your chosen theorem, right?

  3. Objectives • Apply divide and conquer to matrix multiplication • Analyze using the Master Theorem • Introduce you to the “convolution” operation. • See an algorithm for convolution, inspired by polynomial multiplication

  4. Part 1: Matrix Multiplication

  5. Matrix Multiplication • Who cares? • In this setting, • Scalar operations +, –, and x are elementary ops • Assume fixed precision • Matrix operations +, -, and x are NOT elementary ops • What about the commutative property of matrix multiplication?

  6. Naïve Algorithm 5 6 7 8 1 2 3 4 = x

  7. Strassen’s Algorithm e f g h m2+m3 m1+m2+m5+m6 m1+m2+m4-m7 m1+m2+m4+m5 a b c d = x m1 = (c + d - a) x (h – f + e) m2 = (a x e) m3 = (b x g) m4 = (a - c) x (h - f) m5 = (c + d) x (f - e) m6 = (b - c + a - d) x h m7 = d x (e + h – f - g) • Note: this solution was obtained in a manner similar to the Karatsuba Divide and Conquer algorithm for scalar multiplication

  8. Strassen’s on our Example 5 6 7 8 m2+m3 m1+m2+m5+m6 m1+m2+m4-m7 m1+m2+m4+m5 1 2 3 4 = x 5 + 14 42+5+7+(-32) 42+5+(-4)-0 42+5+(-4)+7 19 22 43 50 = = m1 = (3+4-1) x (8-6+5) = 6 x 7 = 42 m2 = (1 x 5) = 5 m3 = (2 x 7) = 14 m4 = (1-3) x (8-6) = -2 x 2 = -4 m5 = (3+4) x (6-5) = 7 x 1 = 7 m6 = (2-3 + 1-4) x 8 = -4 x 8 = -32 m7 = 4 x (5+8-6-7) = 4x0 = 0

  9. Recursive Application x

  10. Recursive Application x

  11. Recursive Application A B E F x G H C D Divide each matrix into fourths, and apply Strassen’s algorithm. Note: divide & conquer without Strassen’s algebraic insight is still

  12. Again: Strassen’s Algorithm A B E F M2+M3 M1+M2+M5+M6 M1+M2+M4-M7 M1+M2+M4+M5 = x C D G H What’s the efficiency? M1 = (C + D - A) x (H – F + E) M2 = (A x E) M3 = (B x G) M4 = (A - C) x (H - F) M5 = (C + D) x (F - E) M6 = (B - C + A - D) x H M7 = D x (E + H – F - G) • 7 recursive calls • Subproblem matrices are half original width • 24 matrix additions to divide/merge Recall: Matrix addition (and subtraction) is

  13. Efficiency Analysis a = number of subinstances n = original instance size n/b = size of subinstances d = polynomial order of g(n) where g(n) is cost of doing the divide (& combine) Therefore:

  14. Can we do better? • Divide matrix width by b=2: • Naïve: multiplications • Only 7 multiplications • Strassen: O(n2.81) • Divide matrix width by b=3: • Naïve: multiplications • Only 21 multiplications • 1979: O(n2.521813) • 1980: O(n2.521801) • Divide matrix width by b=70: • Naïve: • Only 143,640 multiplications (Pan) • 1986: O(n2.376)

  15. Note • Divide and conquer without Strassen’s algebraic insight:

  16. Is it correct? What does it mean for Strassen’s to be correct? How can you prove it in general?

  17. Correctness e f g h m2+m3 m1+m2+m5+m6 m1+m2+m4-m7 m1+m2+m4+m5 a b c d = x m1 = (c + d - a) x (h – f + e) m2 = (a x e) m3 = (b x g) m4 = (a - c) x (h - f) m5 = (c + d) x (f - e) m6 = (b - c + a - d) x h m7 = d x (e + h – f - g)

  18. Part 2: Convolution

  19. Polynomial Multiplication • Example: • What can you say about the degree of the product of two degree polynomials? • What will the coefficients of C look like?

  20. Polynomial Multiplication is Convolution • More generally: • Then: • Where: for all • For , use and

  21. Questions • Is it Correct? • How long does it take (assuming multiplication and additional are elementary ops)? • Can we do better? To find this we treated the math on the preceding slide as terse pseudocode, and reasoned that there are order constants , and calculating each requires up to additions.

  22. More Questions • Why is convolution interesting? • Where else is it used? • Can we generalize the idea of a sequence of coefficients on a polynomial?

  23. Signals Signal: An impulse or a fluctuating quantity, such as voltage, current, or EM field strength, whose variations represent some sort of information.

  24. Signals • Naturally Occurring: • Light / Electromagnetic radiation • Sound (pressure) waves (e.g., in air or water) • Man-made: Encoded information • In coax, CAT-5, etc. • AM/FM Radio broadcast • Radar pulses • Mathematically: Represented by a function of one or more independent variables.

  25. Signals Speech: acoustic pressure as a function of time,digitized as sequence of samples f(t) t

  26. 1-D Convolution = Noise filtering!

  27. Signals Image: brightness as a function of 2 spatial variables x f(x,y) y

  28. -1 0 1 -1 0 1 -1 0 1 2-D Convolution • A 2-D signal (an Image) is convolved with a second Image (the filter, or convolution “Kernel”). =

  29. Assignment Homework: HW #9 • Do the upper right part of the proof of correctness for Strassen’s algorithm • Some more probability exercises Read Section 2.6 on the FFT! • Probably the most famous Divide and Conquer algorithm

More Related