Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
About this lecture • Recall the divide-and-conquer paradigm, which we used for merge sort: • Divide the problem into a number of sub-problems that are smaller instances of the same problem. • Conquer the sub-problems by solving them recursively. • Base case: If the sub-problems are small enough, just solve them by brute force. • Combine the sub-problem solutions to give a solution to the original problem. • We look at two more algorithms based on divide-and-conquer.
About this lecture • Analyzing divide-and-conquer algorithms • Introduce some ways of solving recurrences • Substitution Method (If we know the answer) • Recursion Tree Method (Very useful !) • Master Theorem (Save our effort)
Maximum-subarray problem • Input: an array A[1..n] of n numbers • Assume that some of the numbers are negative, because this problem is trivial when all numbers are nonnegative • Output: a nonempty subarray A[i..j]having the largest sum S[i, j] = ai + ai+1 +... + aj 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 A maximum subarray
A brute-force solution • Examine all possible S[i, j] • Two implementations: • compute each S[i, j] in O(n) time O(n3) time • compute each S[i, j+1] from S[i, j] in O(1) time • (S[i, i] = A[i] and S[i, j+1] = S[i, j] + A[j+1]) • O(n2) time Ex: i 1 2 3 4 5 6 A[i] 13 -3 -25 20 -3 -16 S[1, j] = 13 10 -15 5 2 -14 S[2, j] = -3 -28 -8 -11 -27
A divide-and-conquer solution • Possible locations of a maximum subarray A[i..j] of A[low..high], where mid = (low+high)/2 • entirely in A[low..mid] (lowi j mid) • entirely in A[mid+1..high] (mid < i j high) • crossing the midpoint (lowi mid < j high) crossing the midpoint low mid high mid +1 entirely in A[low..mid] entirely in A[mid+1..high] Possible locations of subarrays of A[low..high]
A[mid+1..j] i low mid high mid +1 j A[i..mid] A[i..j] comprises two subarrays A[i..mid] and A[mid+1..j]
FIND-MAX-CROSSING-SUBARRAY (A,low, mid, high) left-sum = - // Find a maximum subarray of the form A[i..mid] sum = 0 fori = mid downto low sum = sum + A[i ] if sum > left-sum left-sum = sum max-left = i right-sum = - // Find a maximum subarray of the form A[mid + 1 .. j ] sum =0 for j = mid +1 to high sum = sum + A[j] ifsum > right-sum right-sum = sum max-right = j // Return the indices and the sum of the two subarrays Return (max-left, max-right, left-sum + right-sum)
Example: mid =5 A S[5 .. 5] = -3 S[4 .. 5] = 17 (max-left = 4) S[3 .. 5] = -8 S[2 .. 5] = -11 S[1 .. 5] = 2 mid =5 A S[6 .. 6] = -16 S[6 .. 7] = -39 S[6 .. 8] = -21 S[6 .. 9] = (max-right = 9) -1 S[6..10] = -8 maximum subarray crossing mid is S[4..9] = 16
FIND-MAXIMUM-SUBARRAY (A, low, high) if high == low Return (low, high, A[low])// base case: only one element else mid = (left-low, left-high, left-sum) = FIND-MAXIMUM-SUBARRAY(A, low, mid) (right-low, right-high, right-sum)= FIND-MAXIMUM-SUBARRAY(A, mid + 1, high) (cross-low, cross-high, cross-sum) = FIND-MAX-CROSSING-SUBARRAY(A, low, mid, high) if left-sum ≧right-sum and left-sum ≧ cross-sum return (left-low, left-high, left-sum) elseif right-sum ≧ left-sum and right-sum ≧ cross-sum return (right-low, right-high, right-sum) else return (cross-low, cross-high, cross-sum) Initial call: FIND-MAXIMUM-SUBARRAY (A, 1, n)
Analyzing time complexity • FIND-MAX-CROSSING-SUBARRAY : (n), where n = highlow + 1 • FIND-MAXIMUM-SUBARRAY T(n) = 2T(n/2) + (n) (with T(1) = (1)) = (nlg n) (similar to merge-sort)
Matrix multiplication • Input: two nn matrices A and B • Output: C= AB, An O(n3) time naive algorithm SQUARE-MATRIX-MULTIPLY(A, B) n A.rows let C be an nn matrix fori 1 ton forj 1 ton cij 0 fork 1 ton cijcij + aikbkj returnC
Divide-and-Conquer Algorithm • Assume that n is an exact power of 2 (4.1)
Divide-and-Conquer Algorithm • A straightforward divide-and-conquer algorithm • T(n) = 8T(n/2) + (n2) • = (n3) Computing A+B O(n2)
Strassen’s method (4.2)
Strassen’s method (4.4) (4.3)
Strassen’s divide-and-conquer algorithm • Step 1: Divide each of A, B, and C into four sub-matrices as in (4.1) • Step 2: Create 10 matrices S1, S2, …, S10 as in (4.2) • Steep 3: Recursively, compute P1, P2, …, P7 as in (4.3) • Step 4: Compute according to (4.4)
Time complexity T(n) = 7T(n/2) + (n2) = (nlg 7 ) (why?) = (n2.81)
Discussion • Strassen’s method is largely of theoretical interest for n 45 • Strassen’s method is based on the fact that we can multiply two 2 2 matrices using only 7 multiplications (instead of 8). • It was shown that it is impossible to multiply two 2 2 matrices using less than 7 multiplications.
Discussion • We can improve Strassen’s algorithm by finding an efficient way to multiply two k k matrices using a smaller number q of multiplications, where k > 2. The time is T(n) = qT(n/k) + θ(n2). • A trivial lower bound for matrix multiplication is (n2). The current best upper bound known is O(n2.376). • Open problems: • Can the upper bound O(n2.376) be improved? • Can the lower bound (n2) be improved?
Substitution Method(if we know the answer) How to solve this? T(n) = 2T( ) + n, with T(1) = 1 • Make a guess e.g., T(n) = O(n log n) 2. Show it by induction • e.g., to show upper bound, we find constants c and n0 such that T(n) cf(n) for n = n0, n0+1, n0+2, …
Substitution Method(if we know the answer) How to solve this? T(n) = 2T( ) + n, with T(1) = 1 • Make a guess e.g., T(n) = O(n log n) • Show it by induction • Firstly, T(2) = 4, T(3) = 5. We want to have T(n) cn lg n Let c = 2 T(2) and T(3) okay • Other Cases ?
Substitution Method(if we know the answer) • Induction Case: Assume the guess is true for all n = 2, 3,…, k For n = k+1, we have: T(n) = 2T( ) + n = cn lg n – cn + ncn log n Induction case is true
Substitution Method(if we know the answer) Q. How did we know the value of c and n0 ? • If induction works, the induction case must be correct c ≥1 Then, we find that by setting c= 2, our guess is correct as soon as n0 = 2 Alternatively, we can also use c= 1.5 Then, we just need a larger n0 = 4 (What will be the new base cases? Why?)
Substitution Method(New Challenge) How to solve this? 1. Make a guess (T(n) = O(n)), and • Show T(n) ≤ cn by induction • What will happen in induction case?
Substitution Method(New Challenge) Induction Case: (assume guess is true for some base cases) This term is not what we want …
Substitution Method(New Challenge) • The 1st attempt was not working because our guess for T(n) was a bit “loose” Recall: Induction may become easier if we prove a “stronger” statement 2nd Attempt: Refine our statement Try to show T(n) ≤ cn- b instead
Substitution Method(New Challenge) Induction Case: We get the desired term (when b 1) It remains to find c and n0, and prove the base case(s), which is relatively easy Can you prove T(n) ≤ cn2 by induction?
Avoiding Pitfalls • For T(n) = 2T( ) + n, we can falsely prove T(n) = O(n) by guessing T(n) cn and then arguing Wrong!!
Substitution Method(New Challenge 2) How to solve this? T(n) = 2T( ) + lg n ? Hint: Change variable: Set m = lg n
Substitution Method(New Challenge 2) Set m = lg n , we get T(2m) = 2T(2m/2) + m Next, set S(m) = T(2m) = T(n) S(m) = 2S(m/2) + m We solve S(m) = O(m lg m) T(n) = O(lg n lg lg n)
Recursion Tree Method( Nothing Special… Very Useful ! ) How to solve this? T(n) = 2T(n/2) + n2, with T(1) = 1
Recursion Tree Method( Nothing Special… Very Useful ! ) Expanding the terms, we get: T(n) = n2 + 2T(n/2) = n2 + 2n2/4 + 4T(n/4) = n2 + 2n2/4 + 4n2/16 + 8T(n/8) = . . . = = Q(n2) + Q(n) = Q(n2)
Recursion Tree Method( Recursion Tree View ) We can express the previous recurrence by:
This term is from T(n/2) Further expressing gives us:
Recursion Tree Method( New Challenge ) How to solve this? T(n) = T(n/3) + T(2n/3) + n, with T(1) = 1 What will be the recursion tree view?
The corresponding recursion tree view is: The depth of the tree is log3/2n. Why?
Master Method( Save our effort ) When the recurrence is in a special form, we can apply the Master Theorem to solve the recurrence immediately The Master Theorem has 3 cases …
Master Theorem LetT(n) = aT(n/b) + f(n) with a 1 and b 1 are constants, where we interpret n/b to mean either n/b or n/b. Theorem: (Case 1) If f(n) = O(nlogb a- e) for some constant e 0 then T(n) = Q(nlogb a)
Theorem: (Case 2) If f(n) = Q(nlogb a), then T(n) = Q(nlogb a lg n) Theorem: (Case 3) If f(n) = W(nlogb a + e) for some constant e 0, and if af(n/b) c f(n) for some constant c 1, andall sufficiently large n, then T(n) = Q(f(n))
Master Theorem • Solve T(n) = 9T(n/3) + n (case 1) • Solve T(n) = T(2n/3) + 1 (case 2) • Solve T(n) = 3T(n/4) + nlgn (case 3) • How about this? • T(n) = 2T(n/2) + n lg n ? • 5. T(n) = 8T(n/2) + n2 , T(n) = 8T(n/2) + n • 6. T(n) = 7T(n/2) + n2 ,T(n) = 7T(n/2) + 1
Homework • Exercise: 4.1-3 (Programming) (due: Oct. 17) • Practice at home: 4.1-5, 4.2-1, 4.2-7 • Exercise: 4.3-6, 4.4-6, 4.5-1 (due: Oct. 19) • Problem: 4.1 (a, g) (due: Oct.19) • Practice at home: 4.3-7, 4.4-8, 4.4-7, 4.5-4