 Download Download Presentation Analysis & Design of Algorithms (CSCE 321)

# Analysis & Design of Algorithms (CSCE 321)

Download Presentation ## Analysis & Design of Algorithms (CSCE 321)

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
##### Presentation Transcript

1. Analysis & Design of Algorithms(CSCE 321) Prof. Amr Goneid Department of Computer Science, AUC Part 7. Divide & Conquer Algorithms Prof. Amr Goneid, AUC

2. Divide & Conquer Algorithms Prof. Amr Goneid, AUC

3. Divide & Conquer Algorithms • The General Method • Time Complexity • Common Recurrence relations • A General Divide & Conquer Recurrence • Merge Sort Algorithm • Quicksort Algorithm • An Application of Quicksort: Convex Hull • Selection by Partitioning • Closest Pair Problem Prof. Amr Goneid, AUC

4. 1. The General Method To solve a problem using Divide and Conquer: • Divide it into smaller problems • Solve the smaller problems recursively • Combine their solutions into a solution for the big problem Prof. Amr Goneid, AUC

5. The General Method • The base case for recursion is a sub-problem of constant (and small) size. • Analysis of D&Q algorithms is done by constructing a recurrence relation • We can obtain T(n) in a closed form by solving the recurrence relation Prof. Amr Goneid, AUC

6. The General Method • For problem p, small(p) = true if p is simple. In this case S(p) is the solution. Otherwise, Divide, Conquer, then Combine. • Divide & Conquer Algorithm DQ(p) { if small(p) return S(p); else { 1. Divide p into smaller instances p1,p2,..,pk 2. Apply DQ to each pi 3. Return Combine(DQ(p1),DQ(p2),…,DQ(pk))} } Prof. Amr Goneid, AUC

7. 2. Time Complexity Let g(n) = time for S(p) when p is small f(n) = time to divide p and combine solutions then for p small T(n) = g(n) otherwise Prof. Amr Goneid, AUC

8. 3. Common Recurrence Relations • A recursive algorithm that halves the input in one step then processes the two halves: T(n) = 2T(n/2) + 1 for n >1 with T(1) given • A recursive algorithm that halves the input in one step then processes one half and ignores the other: T(n) = T(n/2) + 1 for n >1 with T(1) given Prof. Amr Goneid, AUC

9. Common Recurrence Relations • A recursive algorithm that examines every element before dividing and then ignores one half: T(n) = T(n/2) + n for n >1 with T(1) given • A recursive algorithm that makes a linear pass through the input, before, during or after it is split into two halves then processes the two halves: T(n) = 2 T(n/2) + n for n >1 with T(1) given Prof. Amr Goneid, AUC

10. Example: Recursive Binary Search Algorithm BinSearch (a , s , e , x) { if (s == e) // if small (p), i.e. one element { if (x ==as) return s ; else return -1 ;} // return S(p) else { // two or more elements mid = (s + e)/2 ; // Divide if (x ==amid) return mid ; else if (x <amid) // Conquer return BinSearch (a , s , mid-1 , x) ; // n/2 or else return BinSearch (a , mid+1 , e , x) ; } // n/2 } // No Combine Prof. Amr Goneid, AUC

11. Binary Search Demo http://www.cosc.canterbury.ac.nz/people/mukundan/dsal/BSearch.html Prof. Amr Goneid, AUC

12. Worst Case of Binary Search Let T(n) be the worst case number of comparisons used on an array of n items. Hence Now we seek a closed form solution for T(n) Prof. Amr Goneid, AUC

13. Worst Case of Binary Search Prof. Amr Goneid, AUC

14. 4. A General D&Q Recurrence Prof. Amr Goneid, AUC

15. A General Solution The secondary recurrence is solved as a first order linear recurrence. For T(1) i.e. F(0) given, we have as shown before: Prof. Amr Goneid, AUC

16. Very Common: f(n) = b nx Prof. Amr Goneid, AUC

17. Case(a): f(n) = b nx, a = cx Prof. Amr Goneid, AUC

18. Summary Prof. Amr Goneid, AUC

19. Case(a): f(n) = b nx, a = cx Prof. Amr Goneid, AUC

20. Other examples for a = cx Recursive Power Function The following function computes xn Pow (x , n) { if (n == 0) return 1.0; else if (n == 1) return x; else if (n mod 2) return Pow (x*x , n / 2) * x; else return Pow (x*x , n / 2); } Show that the number of arithmetic operations performed by this algorithm in the best case is T(n) = log n Prof. Amr Goneid, AUC

21. Solution In the best case n = 2m , so that n is always even and we obtain the recurrence relation: T(n) = T(n/2) + 1 , for n > 1 with T(1) = 0 Here, a = 1, b = 1, c = 2, x = 0 with the solution : T(n) = bnx log n = log n Prof. Amr Goneid, AUC

22. Other examples for a = cx Recursive log Function A recursive function to compute the largest integer less than or equal to log2 n. (Hint: for n > = 2 , the value of this function for n is one greater than for n/2 ). int Log2(int n) { if (n == 1) return 0; else return (1 + Log2(n/2)); } The number of integer divisions has the recurrence relation: T(n) = T(n/2) + 1 , for n > 1 with T(1) = b = 0 with the solution : T(n) = log n = O(log n) Prof. Amr Goneid, AUC

23. Other examples for a = cx Find the Number of Binary Digits in the Binary Representation of a Positive Decimal Integer using a D&Q recursive algorithm. int BinDig (int n) { if (n == 1) return 1; else return (1 + BinDig(n/2)); } T(n) = number of arithmetic operations. T(n) = T(n/2) + 2 for n > 1 with T(1) = 0 a = 1, b = 2, c = 2, x = 0 T(n) = 2 log n = O(log n) Prof. Amr Goneid, AUC

24. Other examples for a = cx Other Recurrences with a = cx • T(n) = 4 T(n/2) + n2 for n > 1 with T(2) = 1 Hence, T(n) = n2(log n – ¾) = O(n2 log n) • T(n) = 9T(n/3) + n2 for n > 1 with T(1) = 0 Hence, T(n) = n2 log3 n • T(n) = 16 T(n/4) + n2 for n > 1 with T(1) = 0 Hence, T(n) = n2 log4 n Prof. Amr Goneid, AUC

25. Case(b): f(n) = b nx, a cx Example: a Fractal Algorithm The following function draws recursive squares (called a fractal star). The drawing primitive is Box (x , y , n) which draws a square of side (n) pixels centered at (x,y) : void STAR( int x, int y, int n) { if (n > 1) { STAR(x-n , y+n , n/2); STAR(x+n , y+n , n/2); STAR(x-n , y-n , n/2); STAR(x+n , y-n , n/2); Box(x , y , n); } } Prof. Amr Goneid, AUC

26. Fractal Algorithm The recurrence relation is: T(n) = 4 T(n/2) + 4 n for n > 1 with T(1) = 0 Here, a = 4, b = 4, c = 2, x =1 so that a  cx The general solution is: Prof. Amr Goneid, AUC

27. Example: MaxMinStraightforward Algorithm // Return max and min values in locations s to e in array a[ ] MaxMin (a, s , e , mx, mn) { mx = mn = as; for i = s+1 to e { if (ai > mx) mx = ai ; if (ai < mn) mn = ai ; } } Invoke as MaxMin(a , 1 , n , mx , mn) ; Let T(n) be the number of comparisons of array elements. Hence, T(n) = 2 ( n – 1) independent of the data in a[ ]. Prof. Amr Goneid, AUC

28. MaxMin D&Q Algorithm MaxMin (a, s , e , mx , mn) { if (s  e-1) // one or two elements if (as > ae) {mx = as; mn = ae;} else {mx = ae; mn = as;} else { m = (s+e)/2; MaxMin (a,s,m,mx,mn); Maxmin (a,m+1,e,mx1,mn1); mx = max(mx,mx1); mn = min(mn,mn1); } } // The functions max/min(a,b) find the larger/smaller of (a,b). Each will cost one comparison. Prof. Amr Goneid, AUC

29. MaxMin Analysis The recurrence relation is: T(n) = 2 T(n/2) + 2 for n > 2 with T(2) = 1 Here, a = 2, b = 2, c = 2, x =0 so that a  cx The general solution is: This is only 75% of the cost of the straightforward algorithm. Prof. Amr Goneid, AUC

30. Other Examples with a  cx Copy Binary Tree Algorithm The following function receives a pointer (t) to a binary tree and returns a pointer to an exact copy of the tree : treeNode *CopyTree ( treeNode *t ) { treeNode *p; if (t) { p = new treeNode ; p->left = CopyTree(t->left) ; p->right = CopyTree(t->right); p->info = t->info ; return p ; }; else return NULL ; } Prof. Amr Goneid, AUC

31. Other Examples with a  cx Copy Binary Tree Algorithm In the worst case, the tree is full and the number of visits is: T(n) = 2 T(n/2) + 1 for n > 1 with T(1) = 1 Hence T(n) = 2n – 1 = O(n) Prof. Amr Goneid, AUC

32. Other Examples with a  cx Pre-Order Traversal of a Binary Tree void preorder (tree t) { if (t) { visit (t); preorder (t -> left); preorder (t -> right); } } In the worst case, the tree is full and the number of visits is: T(n) = 2 T(n/2) + 1 for n > 1 with T(1) = 1 Hence T(n) = 2n – 1 = O(n) Prof. Amr Goneid, AUC

33. Other Examples with a  cx Other Algorithms • T(n) = T(n/2) + n2 for n > 1 with T(1) = 0 Hence T(n) = (4/3) (n2 – 1) • T(n) = 2 T(n/2) + n2 for n > 1 with T(1) = 0 Hence T(n) = 2n(n – 1) • T(n) = 8 T(n/2) + n2 for n > 1 with T(1) = 0 Hence T(n) = n2 (n – 1) Prof. Amr Goneid, AUC

34. 5. MergeSort(a) Merging • Definition: Combine two or more sorted sequences of data into a single sorted sequence. • Formal Definition: The input is two sorted sequences, A={a1, ..., an} and B={b1, ..., bm} The output is a single sequence, merge(A,B), which is a sorted permutation of {a1, ..., an, b1, ..., bm}. Prof. Amr Goneid, AUC

35. Merging merge(A, B) is A, if B is empty, B, if A is empty, {a1}.merge({a2, ..., an}, B) if a1 <= b1, and {b1}.merge(A, {b2, ..., bm}) otherwise. The symbol "." stands for concatenation, for example, {a1, ..., an}.{b1, ..., bm} = {a1, ..., an, b1, ..., bm}. Prof. Amr Goneid, AUC

36. Merge Algorithm array B Merge(A,p,q,r) { copy Ap..r to Bp..r and set Ap..r to empty while (neither L1 nor L2 empty) { compare first items of L1 & L2 remove smaller of the two from its list add to end of A } concatenate remaining list to end of A return A } L1 q p q+1 L2 r Prof. Amr Goneid, AUC

37. Example i j B q r p A z Prof. Amr Goneid, AUC

38. Example i j B q r p A z Prof. Amr Goneid, AUC

39. Example i j B q r p A z Prof. Amr Goneid, AUC

40. Example i j B q r p A z Prof. Amr Goneid, AUC

41. Example i j B q r p A z Prof. Amr Goneid, AUC

42. Example i j B q r p A z Prof. Amr Goneid, AUC

43. Example i j B q r p A z Prof. Amr Goneid, AUC

44. Example i j B q r p A z Prof. Amr Goneid, AUC

45. Worst Case Analysis • |L1| = size of L1, |L2| = size of L2 • In the worst case |L1| = |L2| = n/2 • Both lists empty at about same time, so everything has to be compared. • Each comparison adds one item to A so the worst case is T(n) = |A|-1 = |L1|+|L2|-1 = n-1 = O(n) comparisons. Prof. Amr Goneid, AUC

46. (b) MergeSort Methodology • Invented by Von Neumann in 1945 • Recursive Divide-And-Conquer Prof. Amr Goneid, AUC

47. MergeSort Methodology • Divides the sequence into two subsequences of equal size, sorts the subsequences and then merges them into one sorted sequence • Fast, but uses an extra space Prof. Amr Goneid, AUC

48. Methodology (continued) • Divide: Divide n element sequence into two subsequences each of n/2 elements • Conquer: Sort the two sub-arrays recursively • Combine: Merge the two sorted subsequences to produce a sorted sequence of n elements Prof. Amr Goneid, AUC

49. Algorithm MergeSort (A, p, r) // Mergesort array A[ ] locations p..r { if (p < r) // if there are 2 or more elements { q = (p+r)/2; // Divide in the middle // Conquer both MergeSort (A , p , q); MergeSort(A , q+1 , r); Merge(A , p , q , r); // Combine solutions } } Invoke as MergeSort(a , 1 , n) Prof. Amr Goneid, AUC

50. Merge Sort Example Prof. Amr Goneid, AUC