 Download Presentation Divide and Conquer Divide and Conquer - PowerPoint PPT Presentation

Download Presentation Divide and Conquer
An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

1. Divide and Conquer Chapter 2

2. Objectives • Describe the divide-and-conquer approach to solving problems • Apply the divide-and-conquer approach to solve a problem • Determine when the divide-and-conquer approach is an appropriate solution approach • Determine complexity analysis of divide and conquer algorithms • Contrast worst-case and average-case complexity analysis

3. Battle of Austerlitz – December 2, 1805 • Napoleon split the Austro-Russian Army and was able to conquer 2 weaker armies • Divide an instance of a problem into 2 or more smaller instances • Top-down approach

4. Divide and Conquer • In this approach, a problem is divided into sub-problems and the same algorithm is applied ( usually recursively) to every sub-problem • Examples • Binary Search • Mergesort • Quicksort

5. Binary Search • Locate key x in an array of size n sorted in non-decreasing order • Compare x with the middle element : If equal, done – quit. else • Divide the Array into two sub-arrays approximately half as large • If x is smaller than the middle item, select left sub-array • If x is larger than the middle item, select right sub-array

6. Binary Search • Conquer(solve) the sub-array: Determine if x in the sub-array using recursion until the sub-array is sufficiently small ? • Obtain the solution to the array from the solution to the sub-array

7. Algorithms with Pseudocode pass by reference • Algorithm 1.5:Binary Search void binSearch (int n, constkeytype S[ ], keytype x, index& loc){ index low, high, mid; low = 1; high = n; loc = 0; while (low <= high && loc == 0) { mid = floor((low+high) / 2); if (x == S[mid]) loc = mid; else if (x <= S[mid]) high = mid -1; else low = mid +1; } } loc = 0 : false (not found) > 0 : true (found); the location of x in S Requirement: S is a sorted array in nondecreasing order. Suppose n=32 and x is larger than all the elements in the array. The order of comparisons is: S S S S S S 1 2 3 4 5 6

8. Worst Case Complexity Analysis • Assume that n is a power of 2 and x > S[n] • W(n) = W(n/2) + 1 for n>1 and n power of 2 • W(1) = 1 • W(n/2) = the number of comparisons in the recursive call • 1 comparison at the top level • From Example B1 in Appendix B: • W(n) = lg n + 1 (see next slide) • If n is not a power of 2 • W(n) = lg n +1 ε θ(lg n)

9. W(n) = W(n/2) + 1, W(1) = 1 • Assume that n is a power of 2 • W( 1) = 1 = 0 + 1 • W( 2) = W( 2/2) + 1 = W(1) + 1 = 1+1 = 2 • W( 4) = W( 4/2) + 1 = W(2) + 1 = 2+1 = 3 • W( 8) = W( 8/2) + 1 = W(4) + 1 = 3+1 = 4 • W(16) = W(16/2) + 1 = W(8) + 1 = 4+1 = 5 • W(n) = lg n + 1 • Can be proved by mathematical induction

10. Mergesort • Sort an array S of size n (for simplicity, let n be a power of 2) • Divide S into 2 sub-arrays of size n/2 • Conquer (solve) recursivelysort each sub-array until array is sufficiently small (size 1) • Combinemerge the solutions to the sub-arrays into a single sorted array

11. Figure 2.2

12. Mergesort (Alg. 2.2) void mergesort (intn, keytype S[]) { if (n>1) { const int h = floor(n/2), m = n - h; keytype U[1:h], V[1:m]; copy S[1:h] to U[1:h]; // First part copy S[h+1:n] to V[1:m]; // Second part mergesort(h, U); // Recursive call mergesort(m, V); // Recursive call merge(h, m, U, V, S); // Merging U and V to get S } } Worst-case time complexity: W(n) = W(h) + W(m) + W(merge)

13. Merge • Merges the two sorted sub-arrays U and V created by the recursive calls to mergesort • Input size (n = h + m) • h: the number of items in U • m: the number of items in V • Merging: • Initialize three indices i (for U), j (for V), and k (for S) to 1 • Comparison of U[i] to V[j], move smaller to S[k], increment its index (either i or j), increment k • Keep doing 2) while 1≤ i ≤ h && 1≤ j ≤ m • Copy the remaining elements to S[k] through S[h+m]

14. Example of Merging k U V S(Result) 1 10 12 20 27 13 15 22 25 10 2 10 12 20 27 13 15 22 25 10 12 3 10 12 20 27 13 15 22 25 10 12 13 4 10 12 20 27 13 15 22 25 10 12 13 15 5 10 12 20 27 13 15 22 25 10 12 13 15 20 6 10 12 20 27 13 15 22 25 10 12 13 15 20 22 7 10 12 20 27 13 15 22 25 10 12 13 15 20 22 25 _ 10 12 20 27 13 15 22 25 10 12 13 15 20 22 25 27 i  j  k 

15. Merge (Alg. 2.3) Worst case: Loop exited with one index at exit point and the other one less than the exit point void merge (int h, int m, const keytype U[], const keytype V[], keytype S[]) { index i, j, k; i=1, j=1, k=1; while (i <= h && j <= m) { if (U[i] < V[j]) { S[k] = U[i]; i++; } else { S[k] = V[j]; j++; } k++; } if (i > h) copy V[j:m] to S[k:h+m]; else copy U[i:h] to S[k:h+m]; } Basic operation: Comparison Worst-case time complexity: W(merge) = W(h, m) = h+m-1 comparisons

16. Worst-Case Time Complexity Analysis • W(n) = time to sort U + time to sort V + time to merge • W(n) = W(h) + W(m) + (h+m-1) • First , assume n is a power of 2 • h = n/2 = n / 2 • m = n – h = n – n/2 = n / 2 • h + m -1 = n - 1 • W(n) = W(n/2) + W(n/2) + n–1 = 2W(n/2) + n-1 for n > 1 and n a power of 2 • W(1) = 0 • From B19 in Appendix B (pp. 624) • W(n) = n lgn - (n-1) ε θ(nlgn) 2K-1

17. Worst-Case Time Complexity Analysis • W(1) = 0 • W(n) = 2 W(n/2) + n-1for n > 1 and n a power of 2 • W(2) = 2W(1) + 1 = 1 = 0 + 1 • W(4) = 2W(2) + 3 = 5 = 4 + 1 • W(8) = 2W(4) + 7 = 17 = 16 + 1 • W(16) = 2W(8) + 15 = 49 = 48 + 1 • W(32) = 2W(16) + 31 = 129 = 128 + 1 …  W(n) = n lg(n/2) + 1 (same as n lg n – (n-1) in the textbook)

18. W(n) = 2W(n/2) + n-1, W(1) = 0 W(n) = 2 W(n/2) + n-1 = 2 [ 2w(n/22) + n/2 -1] + n-1 = 22 W(n/22) + 2n-3 = 22 [2W(n/23) + n/22 -1] + 2n-(1+2) = 23 W(n/23) + 3n-(1+2+4) … = 2k W(n/2k) + kn - (1+2+4+…+2k-1) For n = 2k , we have W(n) = n W(1) + (lg2n) n – (2k -1) = 0 + n lg2n – (n-1) ϵ O(n lg2n)

19. Worst-Case Analysis of Mergesort • For n not a power of 2 W(n) = W(n/2) + W(n/2) + n-1 W(1) = 0 • From Theorem B4 (pp. 631): W(n) ε Θ(n lgn)

20. Worst-Case Time Complexity Analysis • W(1) = 0 • W(n) = 2W(n/2) + n - 1 for n > 1 and n a power of 2 • Let n = 2K W(2K) = 2 W(2K-1) + (2K – 1) = 2 [2W(2K-2) + 2K-1 – 1] + (2K – 1) = 22W(2K-2) + (2K – 1) + (2K – 2) = 22 [2W(2K-3) + 2K-2 – 1] + (2K – 1) + (2K – 2) = 23W(2K-3) + (2K – 1) + (2K – 2) + (2K – 4) … = 2kW(20) + (2K – 1) + (2K – 2) + (2K – 4) + … (2K – 2K-1) = 2K k - (1+2+4+…+ 2K-1) = n lgn – (n–1)

21. An in-place sort is a sorting alg. that does not use any unnecessary extra memory space Mergesort 2 (an in-place sort) voidmergesort2 (index low, index high) { index mid; if (low < high) { mid = floor( (low + high)/2) ); mergesort2 (low, mid); mergesort2 (mid+1, high); merge2 (low, mid, high); } }

22. Merge 2

23. Quicksort • Array recursively divided into two partitions and recursively sorted • Division based on a pivot • pivot divides the two sub-arrays • All items < pivot placed in sub-array before pivot • All items ≥ pivot placed in sub-array after pivot

24. Figure 2.3

25. Example of Procedure partition(Table 2.2 ) j i S S S S S S S S - - 15 22 13 27 12 10 20 25 1 2 152213 27 12 10 20 25 2 3 15 22 13 27 12 10 20 25 2 4 15 13 22 27 12 10 20 25 3 5 15 13 22 27 12 10 20 25 4 6 15 13 12 27 22 10 20 25 4 7 15 13 12 10 22 27 20 25 4 8 15 13 12 10 22 27 20 25 4 - 10 13 12 15 22 27 20 25 j  i  Green numbers: items compared Circled numbers: items exchanged

26. Partition void partition (index low, index high, index& pivotpoint) { index i, j; keytype pivotitem; pivotitem = S[low]; // select the leftmost item to be the pivotitem j=low; for (i = low+1; i <= high; i++) if (S[i] < pivotitem){ j++; swap S[i] and S[j]; } pivotpoint = j; swap S[low] and S[pivotpoint]; // put pivotitem atpivotposition }

27. Analysis of the Partition Alg. • Basic Operation: Comparison of S[i] with pivotitem • Input size: n = high – low + 1 Note: n here is the # of items in the subarray, • Since every item except the first is compared, we have T(n) = n-1 (every case, partition part only)

28. Worst-Case Complexity Analysis of Quicksort • Array is sorted in non-decreasing order • Array is repeatedly sorted into 1. an empty sub-array which is less than the pivot and 2. a sub-array of n-1 containing items greater than pivot (This is the worst-case scenario.) • If there are k keys in the current sub-array, k-1 key comparisons are executed

29. Worst-Case Complexity Analysis of Quicksort • If the array is already sorted in non-decreasing order, the left sub-array will always be empty. • T(n) = time to sort left sub-array + time to sort right sub-array + time to partition T(n) = T(0) + T(n-1) + n – 1 T(0) = 0 T(n) = T(n–1) + n – 1 for n > 0 • From B.16 (pp. 620), we have T(n) = n(n-1) / 2 • This is the every-case complexity for the special case when the array is already sorted  The real worst-case will be at least n(n-1) / 2

30. Worst Case • We have already showed that the worst case is at least n(n-1)/2. But, is it possible that the worst-case complexity W(n) > n(n-1) / 2 ? • We now use induction to show n(n-1)/2 is really the worst case, i.e., W(n) ≤ n(n-1) / 2

31. Worst Case • Step 1: For n = 1, W(1) = 0 ≤ 1(1-1)/2 • Step 2: Assume that, for 1 ≤ k < n, W(k) ≤ k(k-1)/2 • Step 3: We need to show that W(n) ≤ n(n-1) / 2 Proof: Let p be the value of pivotpoint returned by partition at the top level when the worst case is encountered. W(n) ≤ W(p-1) + W(n-p) + (n-1) ≤ (p-1)(p-2) / 2 + (n-p)(n-p-1) / 2 + (n-1) 2W(n) ≤ (p2-3p+2) + (n2-2np+p2-n+p) + 2(n-1) = (n2–n) + 2(p2-p-np+n) = (n2–n) + 2(p-1)(p-n) ≤ (n2–n) W(n) ≤ n(n-1) / 2 ≤ 0 for 1 ≤ p ≤ n

32. Average-Case Time Complexity of Quicksort • In the average case, we need to consider all of the possible places where the pivot point (p) winds up. It may end up at position 1, 2, …, or n. • Assume the value of pivotpoint is equally likely to be any of the numbers from 1 to n  The probability that the pivotpoint is p is (1/n) • Average obtained is the average sorting time when every possible ordering is sorted the same number of times A(n)= ∑ (1/n) [A(p-1) + A(n-p)] + (n-1) • Solving equation and using B.22 (pp. 627) A(n) ε θ(nlgn) n p = 1

33. Average Case Analysis 1st part 2nd part 1 p n (p-1) elements (n-p) elements p=1  A(0) & A(n-1) p=2  A(1) & A(n-2) …

34. Average Case Analysis (A) replace n with (n-1) (B) (A) - (B)

35. Average Case Analysis (see pp.627)

36. Strassen’s Matrix Mutiplication Algorithm • Uses a set of seven formulas to multiply two 2 x 2 matrices • Does not rely on elements being commutative under multiplication • It can be applied recursively, in other words, two 4 x 4 matrices can be multiplied by treating each as a 2 x 2 matrix of 2 x 2 submatrices

37. Strassen’s Formulas • To compute C = A B: • Strassen’s Algorithm: 7 multiplications 18 additions/substraction

38. Strassen’s Formulas • To compute C = A B of dimension n x n, assuming n is a power of 2:

39. Strassen’s Algorithm

40. General Divide and Conquer recurrence: Design and Analysis of Algorithms - Chapter 4 T(n) = aT(n/b) + f (n) where f (n) ∈Θ(nd) • a < bd T(n) ∈Θ(nd) • a = bd T(n) ∈Θ(nd lg n ) • a > bd T(n) ∈Θ(nlogb a) Note: the same results hold with big-O too.

41. Analysis a =7, b=2, d=0 a >bd T(n) ϵ Ɵ(n 2.81 ) • Number of Multiplication. Let T(n)be the # of multiplications for matrices of size n Recurrence Eq.: T(n) = 7 T(n/2) , n > 1 , n a power of 2 Initial Condition:T(1) = 1 T(n) =nlg 7≈n2.81ϵ Ɵ(n2.81 ) • Number of Additions / Substractions. Let T(n)be the # of additions for matrices of size n Recurrence Eq.: T(n) = 7 T(n/2) + 18(n/2)2 , n > 1, n a power of 2 Initial Condition:T(1) = 0 T(n) = 6nlg 7 – 6n2 ≈ 6n2.81 – 6n2 ϵ Ɵ(n2.81 ) • This is the first algorithm that is faster than O(N3)

42. Avoid divide-and-Conquer if • An problem of size n is divided into two or more problems each almost of size n. • Ex: F(n) = F(n-1) + F(n-2) • An problem of size n is divided into almost n problems of size n / c, where c is a constant. • The first partitioning leads to an exponential-time algorithm, where the second leads to a nΘ(lg n)algorithm. • On the other hand, a problem may require exponentiality, and in such a case there is no reason to avoid the simple divide-and-conquer solution. • Ex. the Tower of Hanoi problem When Not To Use Divide-andConquer