Create Presentation
Download Presentation

Download Presentation
## Analysis & Design of Algorithms (CSCE 321)

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Analysis & Design of Algorithms(CSCE 321)**Prof. Amr Goneid Department of Computer Science, AUC Part 7. Divide & Conquer Algorithms Prof. Amr Goneid, AUC**Divide & Conquer Algorithms**Prof. Amr Goneid, AUC**Divide & Conquer Algorithms**• The General Method • Time Complexity • Common Recurrence relations • A General Divide & Conquer Recurrence • Merge Sort Algorithm • Quicksort Algorithm • An Application of Quicksort: Convex Hull • Selection by Partitioning • Closest Pair Problem Prof. Amr Goneid, AUC**1. The General Method**To solve a problem using Divide and Conquer: • Divide it into smaller problems • Solve the smaller problems recursively • Combine their solutions into a solution for the big problem Prof. Amr Goneid, AUC**The General Method**• The base case for recursion is a sub-problem of constant (and small) size. • Analysis of D&Q algorithms is done by constructing a recurrence relation • We can obtain T(n) in a closed form by solving the recurrence relation Prof. Amr Goneid, AUC**The General Method**• For problem p, small(p) = true if p is simple. In this case S(p) is the solution. Otherwise, Divide, Conquer, then Combine. • Divide & Conquer Algorithm DQ(p) { if small(p) return S(p); else { 1. Divide p into smaller instances p1,p2,..,pk 2. Apply DQ to each pi 3. Return Combine(DQ(p1),DQ(p2),…,DQ(pk))} } Prof. Amr Goneid, AUC**2. Time Complexity**Let g(n) = time for S(p) when p is small f(n) = time to divide p and combine solutions then for p small T(n) = g(n) otherwise Prof. Amr Goneid, AUC**3. Common Recurrence Relations**• A recursive algorithm that halves the input in one step then processes the two halves: T(n) = 2T(n/2) + 1 for n >1 with T(1) given • A recursive algorithm that halves the input in one step then processes one half and ignores the other: T(n) = T(n/2) + 1 for n >1 with T(1) given Prof. Amr Goneid, AUC**Common Recurrence Relations**• A recursive algorithm that examines every element before dividing and then ignores one half: T(n) = T(n/2) + n for n >1 with T(1) given • A recursive algorithm that makes a linear pass through the input, before, during or after it is split into two halves then processes the two halves: T(n) = 2 T(n/2) + n for n >1 with T(1) given Prof. Amr Goneid, AUC**Example: Recursive Binary Search Algorithm**BinSearch (a , s , e , x) { if (s == e) // if small (p), i.e. one element { if (x ==as) return s ; else return -1 ;} // return S(p) else { // two or more elements mid = (s + e)/2 ; // Divide if (x ==amid) return mid ; else if (x <amid) // Conquer return BinSearch (a , s , mid-1 , x) ; // n/2 or else return BinSearch (a , mid+1 , e , x) ; } // n/2 } // No Combine Prof. Amr Goneid, AUC**Binary Search Demo**http://www.cosc.canterbury.ac.nz/people/mukundan/dsal/BSearch.html Prof. Amr Goneid, AUC**Worst Case of Binary Search**Let T(n) be the worst case number of comparisons used on an array of n items. Hence Now we seek a closed form solution for T(n) Prof. Amr Goneid, AUC**Worst Case of Binary Search**Prof. Amr Goneid, AUC**4. A General D&Q Recurrence**Prof. Amr Goneid, AUC**A General Solution**The secondary recurrence is solved as a first order linear recurrence. For T(1) i.e. F(0) given, we have as shown before: Prof. Amr Goneid, AUC**Very Common: f(n) = b nx**Prof. Amr Goneid, AUC**Case(a): f(n) = b nx, a = cx**Prof. Amr Goneid, AUC**Summary**Prof. Amr Goneid, AUC**Case(a): f(n) = b nx, a = cx**Prof. Amr Goneid, AUC**Other examples for a = cx**Recursive Power Function The following function computes xn Pow (x , n) { if (n == 0) return 1.0; else if (n == 1) return x; else if (n mod 2) return Pow (x*x , n / 2) * x; else return Pow (x*x , n / 2); } Show that the number of arithmetic operations performed by this algorithm in the best case is T(n) = log n Prof. Amr Goneid, AUC**Solution**In the best case n = 2m , so that n is always even and we obtain the recurrence relation: T(n) = T(n/2) + 1 , for n > 1 with T(1) = 0 Here, a = 1, b = 1, c = 2, x = 0 with the solution : T(n) = bnx log n = log n Prof. Amr Goneid, AUC**Other examples for a = cx**Recursive log Function A recursive function to compute the largest integer less than or equal to log2 n. (Hint: for n > = 2 , the value of this function for n is one greater than for n/2 ). int Log2(int n) { if (n == 1) return 0; else return (1 + Log2(n/2)); } The number of integer divisions has the recurrence relation: T(n) = T(n/2) + 1 , for n > 1 with T(1) = b = 0 with the solution : T(n) = log n = O(log n) Prof. Amr Goneid, AUC**Other examples for a = cx**Find the Number of Binary Digits in the Binary Representation of a Positive Decimal Integer using a D&Q recursive algorithm. int BinDig (int n) { if (n == 1) return 1; else return (1 + BinDig(n/2)); } T(n) = number of arithmetic operations. T(n) = T(n/2) + 2 for n > 1 with T(1) = 0 a = 1, b = 2, c = 2, x = 0 T(n) = 2 log n = O(log n) Prof. Amr Goneid, AUC**Other examples for a = cx**Other Recurrences with a = cx • T(n) = 4 T(n/2) + n2 for n > 1 with T(2) = 1 Hence, T(n) = n2(log n – ¾) = O(n2 log n) • T(n) = 9T(n/3) + n2 for n > 1 with T(1) = 0 Hence, T(n) = n2 log3 n • T(n) = 16 T(n/4) + n2 for n > 1 with T(1) = 0 Hence, T(n) = n2 log4 n Prof. Amr Goneid, AUC**Case(b): f(n) = b nx, a cx**Example: a Fractal Algorithm The following function draws recursive squares (called a fractal star). The drawing primitive is Box (x , y , n) which draws a square of side (n) pixels centered at (x,y) : void STAR( int x, int y, int n) { if (n > 1) { STAR(x-n , y+n , n/2); STAR(x+n , y+n , n/2); STAR(x-n , y-n , n/2); STAR(x+n , y-n , n/2); Box(x , y , n); } } Prof. Amr Goneid, AUC**Fractal Algorithm**The recurrence relation is: T(n) = 4 T(n/2) + 4 n for n > 1 with T(1) = 0 Here, a = 4, b = 4, c = 2, x =1 so that a cx The general solution is: Prof. Amr Goneid, AUC**Example: MaxMinStraightforward Algorithm**// Return max and min values in locations s to e in array a[ ] MaxMin (a, s , e , mx, mn) { mx = mn = as; for i = s+1 to e { if (ai > mx) mx = ai ; if (ai < mn) mn = ai ; } } Invoke as MaxMin(a , 1 , n , mx , mn) ; Let T(n) be the number of comparisons of array elements. Hence, T(n) = 2 ( n – 1) independent of the data in a[ ]. Prof. Amr Goneid, AUC**MaxMin D&Q Algorithm**MaxMin (a, s , e , mx , mn) { if (s e-1) // one or two elements if (as > ae) {mx = as; mn = ae;} else {mx = ae; mn = as;} else { m = (s+e)/2; MaxMin (a,s,m,mx,mn); Maxmin (a,m+1,e,mx1,mn1); mx = max(mx,mx1); mn = min(mn,mn1); } } // The functions max/min(a,b) find the larger/smaller of (a,b). Each will cost one comparison. Prof. Amr Goneid, AUC**MaxMin Analysis**The recurrence relation is: T(n) = 2 T(n/2) + 2 for n > 2 with T(2) = 1 Here, a = 2, b = 2, c = 2, x =0 so that a cx The general solution is: This is only 75% of the cost of the straightforward algorithm. Prof. Amr Goneid, AUC**Other Examples with a cx**Copy Binary Tree Algorithm The following function receives a pointer (t) to a binary tree and returns a pointer to an exact copy of the tree : treeNode *CopyTree ( treeNode *t ) { treeNode *p; if (t) { p = new treeNode ; p->left = CopyTree(t->left) ; p->right = CopyTree(t->right); p->info = t->info ; return p ; }; else return NULL ; } Prof. Amr Goneid, AUC**Other Examples with a cx**Copy Binary Tree Algorithm In the worst case, the tree is full and the number of visits is: T(n) = 2 T(n/2) + 1 for n > 1 with T(1) = 1 Hence T(n) = 2n – 1 = O(n) Prof. Amr Goneid, AUC**Other Examples with a cx**Pre-Order Traversal of a Binary Tree void preorder (tree t) { if (t) { visit (t); preorder (t -> left); preorder (t -> right); } } In the worst case, the tree is full and the number of visits is: T(n) = 2 T(n/2) + 1 for n > 1 with T(1) = 1 Hence T(n) = 2n – 1 = O(n) Prof. Amr Goneid, AUC**Other Examples with a cx**Other Algorithms • T(n) = T(n/2) + n2 for n > 1 with T(1) = 0 Hence T(n) = (4/3) (n2 – 1) • T(n) = 2 T(n/2) + n2 for n > 1 with T(1) = 0 Hence T(n) = 2n(n – 1) • T(n) = 8 T(n/2) + n2 for n > 1 with T(1) = 0 Hence T(n) = n2 (n – 1) Prof. Amr Goneid, AUC**5. MergeSort(a) Merging**• Definition: Combine two or more sorted sequences of data into a single sorted sequence. • Formal Definition: The input is two sorted sequences, A={a1, ..., an} and B={b1, ..., bm} The output is a single sequence, merge(A,B), which is a sorted permutation of {a1, ..., an, b1, ..., bm}. Prof. Amr Goneid, AUC**Merging**merge(A, B) is A, if B is empty, B, if A is empty, {a1}.merge({a2, ..., an}, B) if a1 <= b1, and {b1}.merge(A, {b2, ..., bm}) otherwise. The symbol "." stands for concatenation, for example, {a1, ..., an}.{b1, ..., bm} = {a1, ..., an, b1, ..., bm}. Prof. Amr Goneid, AUC**Merge Algorithm**array B Merge(A,p,q,r) { copy Ap..r to Bp..r and set Ap..r to empty while (neither L1 nor L2 empty) { compare first items of L1 & L2 remove smaller of the two from its list add to end of A } concatenate remaining list to end of A return A } L1 q p q+1 L2 r Prof. Amr Goneid, AUC**Example**i j B q r p A z Prof. Amr Goneid, AUC**Example**i j B q r p A z Prof. Amr Goneid, AUC**Example**i j B q r p A z Prof. Amr Goneid, AUC**Example**i j B q r p A z Prof. Amr Goneid, AUC**Example**i j B q r p A z Prof. Amr Goneid, AUC**Example**i j B q r p A z Prof. Amr Goneid, AUC**Example**i j B q r p A z Prof. Amr Goneid, AUC**Example**i j B q r p A z Prof. Amr Goneid, AUC**Worst Case Analysis**• |L1| = size of L1, |L2| = size of L2 • In the worst case |L1| = |L2| = n/2 • Both lists empty at about same time, so everything has to be compared. • Each comparison adds one item to A so the worst case is T(n) = |A|-1 = |L1|+|L2|-1 = n-1 = O(n) comparisons. Prof. Amr Goneid, AUC**(b) MergeSort Methodology**• Invented by Von Neumann in 1945 • Recursive Divide-And-Conquer Prof. Amr Goneid, AUC**MergeSort Methodology**• Divides the sequence into two subsequences of equal size, sorts the subsequences and then merges them into one sorted sequence • Fast, but uses an extra space Prof. Amr Goneid, AUC**Methodology (continued)**• Divide: Divide n element sequence into two subsequences each of n/2 elements • Conquer: Sort the two sub-arrays recursively • Combine: Merge the two sorted subsequences to produce a sorted sequence of n elements Prof. Amr Goneid, AUC**Algorithm**MergeSort (A, p, r) // Mergesort array A[ ] locations p..r { if (p < r) // if there are 2 or more elements { q = (p+r)/2; // Divide in the middle // Conquer both MergeSort (A , p , q); MergeSort(A , q+1 , r); Merge(A , p , q , r); // Combine solutions } } Invoke as MergeSort(a , 1 , n) Prof. Amr Goneid, AUC**Merge Sort Example**Prof. Amr Goneid, AUC