1 / 50

Contents of Chapter 3

Contents of Chapter 3. Chapter 3 Divide-and Conquer 3.1 General method 3.2 Binary search 3.3 Finding the maximum and minimum 3.4 Merge sort 3.5 Quicksort 3.6 Selection 3.7 Strassen ’ s matrix multiplication 3.8 Convex hull 3.9 References and readings 3.10 Additional exercises.

kylene
Download Presentation

Contents of Chapter 3

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Contents of Chapter 3 • Chapter 3 Divide-and Conquer • 3.1 General method • 3.2 Binary search • 3.3 Finding the maximum and minimum • 3.4 Merge sort • 3.5 Quicksort • 3.6 Selection • 3.7 Strassen’s matrix multiplication • 3.8 Convex hull • 3.9 References and readings • 3.10 Additional exercises

  2. 3.1 General Method • Control abstraction for divide-and-conquer (Program 3.1) • Subproblem Pi is the same type as original problem Type DAndC(P) { if Small(P) return S(P); else { divide P into smaller instances P1, P2, …, PK, k≥1; Apply DAndC to each of these subproblems; return Combine(DAndC(P1), DAndC(P2), …, DAndC(Pk)); } }

  3. n small otherwise 3.1 General Method • Computing time of DAndC (Formula 3.1) • g(n): time to compute the answer for small inputs • f(n): time for dividing P and combining the solutions to subproblems

  4. n = 1 3.1 General Method • Solving recurrence relations • General form (Formula 3.2) • Example 3.1: a=2, b=2, T(1)=2, f(n)=n • T(n) = 2T(n/2) + n = 4T(n/4) + 2n (= 2[2T(n/4)+n/2] + n) = 8T(n/8) + 3n (= 4[2T(n/8)+n/4] + 2n) = … = nT(1)+ n log2 n = n log2 n + 2n

  5. 3.2 Binary Search • An instance of binary search problem • P = (n, ai,ai+1,…,al, x) • n: number of elements • ai,ai+1,…,al: list of elements in nondecreasing order • x: element to be searched for • Divide-and-conquer • Small(P) • True if n=1 • S(P) takes the value i if x=ai, otherwise 0 • So, g(1)=(1) • Divide (if Small(P) is not true) • Pick an index q (in the range [i,l]) and compare x with aq • Three possibilities • x=aq: P is solved • x<aq: P reduces to (q-i, ai,ai+1,…,aq-1, x) • x>aq: P reduces to (l-q, aq+1,aq+2,…,al, x) • Division takes only (1) • If q is chosen to be q=floor((n+1)/2), the algorithm is binary search • No need for combining (because the answer to new subproblem is also the answer to the original problem)

  6. 3.2 Binary Search • Recursive binary search (Program 3.2) int BinSrch(Type a[], int i, int l, Type x) // Given an array a[i:l] of elements in nondecreasing // order, 1<=i<=l, determine whether x is present, and // if so, return j such that x == a[j]; else return 0. { if (l==i) { // If Small(P) if (x==a[i]) return i; else return 0; } else { // Reduce P into a smaller subproblem. int mid = (i+l)/2; if (x == a[mid]) return mid; else if (x < a[mid]) return BinSrch(a,i,mid-1,x); else return BinSrch(a,mid+1,l,x); } }

  7. 3.2 Binary Search • Nonrecursive binary search (Program 3.3) int BinSearch(Type a[], int n, Type x) // Given an array a[1:n] of elements in nondecreasing // order, n>=0, determine whether x is present, and // if so, return j such that x == a[j]; else return 0. { int low = 1, high = n; while (low <= high){ int mid = (low + high)/2; if (x < a[mid]) high = mid - 1; else if (x > a[mid]) low = mid + 1; else return(mid); } return(0); }

  8. 3.2 Binary Search • Example 3.6 • 14 entries in a[] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] -15 -6 0 7 9 23 54 82 101 112 125 131 142 151 • Three examples of binary search (Table 3.2) • Theorem 3.1: Function BinSearch(a,n,x) works correctly

  9. 3.2 Binary Search • Space complexity • n+4: a[1:n], low, high, mid, x • (n) • Time complexity • Basic operation: element comparison • Other operations (arithmetic) are of the same order • Binary decision tree (Figure 3.1)

  10. 3.2 Binary Search • Successful search a: [1] [2] [3] [4] [5] [6] [7] [8] [9][10] [11] [12] [13] [14] Elements: -15 -6 0 7 9 23 54 82 101 112 125 131 142 151 Comparisons: 3 4 2 4 3 4 1 4 3 4 2 4 3 4 On the average: 45/14  3.21 comparisons • Unsuccessful search: (3+14*4)/15  3.93 • Theorem 3.2: • If n is in the range [2k-1, 2k), then BinSearch makes at most k element comparisons for a successful search and either k-1 or k comparisons for an unsuccessful search. (In other words, the time for a successful search is O(log n) and for an unsuccessful search is (log n)). • Computing time of BinSearch • Successful searches • Best (1), Average (log n), Worst (log n) • Unsuccessful searches • Best, average, worst: (log n)

  11. 3.2 Binary Search • Can expect another algorithm to be significantly better than binary search in the worst case? • No! (refer to Chapter 10) • Binary search using one comparison per cycle (Program 3.4) BinSearch1(Type a[], int n, Type x) // Same specifications as BinSearch except n > 0 { int low=1, high=n+1; // high is one more than possible. while (low < (high-1)) { int mid = (low + high)/2; if (x < a[mid]) high = mid; // Only one comparison // in the loop else low = mid; // x >= a[mid] } if (x == a[low]) return(low); // x is present. else return(0); // x is not present. }

  12. 3.2 Binary Search • BinSearch1 is better than BinSearch in average case (Table 3.3) • BinSearch1 is worse than BinSearch in the best case • best case of BinSearch: (1) • best case of BinSearch1: (log n)

  13. 3.3 Finding the Maximum and Minimum • A straightforward algorithm (Program 3.5) • Analysis • basic operation: element comparisons • 2(n-1) in best,average,and worst cases void StraightMaxMin(Type a[], int n, Type& max, Type& min) // Set max to the maximum and min to the minimum of a[1:n]. { max = min = a[1]; for (int i=2; i<=n; i++) { if (a[i] > max) max = a[i]; if (a[i] < min) min = a[i]; } }

  14. 3.3 Finding the Maximum and Minimum • An improvement if(a[i]>max) max = a[i]; else if(a[i]<min) min = a[i]; • best case: n-1 element comparisons (when in increasing order) • worst case: 2(n-1) element comparisons (when in decreasing order) • Divide-and-conquer algorithm (Program 3.6) void MaxMin(int i, int j, Type& max, Type& min) // a[1:n] is a global array. Parameters i and j are // integers, 1 <= i <= j <= n. The effect is to set // max and min to the largest and smallest values in // a[i:j], respectively. { if (i == j) max = min = a[i]; // Small(P) else if (i == j-1) { // Another case of Small(P) if (a[i] < a[j]) { max = a[j]; min = a[i]; } else { max = a[i]; min = a[j]; } }

  15. else { // If P is not small // divide P into subproblems. // Find where to split the set. int mid=(i+j)/2; Type max1, min1; // Solve the subproblems. MaxMin(i, mid, max, min); MaxMin(mid+1, j, max1, min1); // Combine the solutions. if (max < max1) max = max1; if (min > min1) min = min1; } } 3.3 Finding the Maximum and Minimum

  16. 9 1,9,60,-8 8 5 6,9,60,17 1,5,22,-8 3 4 6 7 1,3,22,-5 8,9,47,31 4,5,15,-8 6,7,60,17 1 2 1,2,22,13 3,3,-5,-5 3.3 Finding the Maximum and Minimum • Example a: [1] [2] [3] [4] [5] [6] [7] [8] [9] Elements: 22 13 -5 -8 15 60 17 31 47 • Trees of recursive calls (Figure 3.2)

  17. n > 2 n = 2 n = 1 • 1 0 3.3 Finding the Maximum and Minimum • Time complexity • Recurrence relations • When n=2k

  18. 3.3 Finding the Maximum and Minimum • Time complexity • 3n/2-2: best, average,and worst cases • 25% savings compared with 2n-2 for StraightMaxMin • no algorithm can do better, so MaxMin is optimal (See Chapter 10) • Does this mean that MaxMin is better in practice? • Not necessarily due to recursion (log2 n+1 levels of recursion and stacking seven values for each recursive call)

  19. void MergeSort(int low, int high) // a[low : high] is a global array to be sorted. // Small(P) is true if there is only one element to // sort. In this case the list is already sorted. { if (low < high) { // If there are more than one element // Divide P into subproblems. // Find where to split the set. int mid = (low + high)/2; // Solve the subproblems. MergeSort(low, mid); MergeSort(mid + 1, high); // Combine the solutions. Merge(low, mid, high); } } 3.4 Merge Sort • General idea • Given n elements a[1],…, a[n],split into two sets a[1],…,a[n/2] and a[n/2+1],…,a[n]. • Each set is individually sorted, and the resulting sorted sequences are merged to a single sorted sequence of n elements. • Merge sort algorithm (Programs 3.7)

  20. 3.4 Merge Sort • Merge sort algorithm (Programs 3.8) void Merge(int low, int mid, int high) // a[low:high] is a global array containing two sorted // subsets in a[low:mid] and in a[mid+1:high]. The goal // is to merge these two sets into a single set residing // in a[low:high]. b[] is an auxiliary global array. { int h = low, i = low, j = mid+1, k; while ((h <= mid) && (j <= high)) { if (a[h] <= a[j]) { b[i] = a[h]; h++; } else { b[i] = a[j]; j++; } i++; } if (h > mid) for (k=j; k<=high; k++) { b[i] = a[k]; i++; } else for (k=h; k<=mid; k++) { b[i] = a[k]; i++; } for (k=low; k<=high; k++) a[k] = b[k]; }

  21. 3.4 Merge Sort • Example 3.7 a[1:10]=(310, 285, 179, 652, 351, 423, 861, 254, 450, 520) • Hand simulation (310 285 179 652 351 | 423 861 254 450 520) split (310 285 179 | 652 351 | 423 861 254 450 520) split (310 285 | 179 | 652 351 | 423 861 254 450 520) split (310 | 285 | 179 | 652 351 | 423 861 254 450 520) split (285 310 | 179 | 652 351 | 423 861 254 450 520) merge (179 285 310 | 652 351 | 423 861 254 450 520) merge (179 285 310 | 652 | 351 | 423 861 254 450 520) split (179 285 310 | 351 652 | 423 861 254 450 520) merge (179 285 310 351 652 | 423 861 254 450 520) merge (179 285 310 351 652 | 423 861 254 | 450 520) split ……

  22. 3.4 Merge Sort • Example 3.7 (Continued) • Tree of calls of MergeSort(1,10) (Figure 3.3)

  23. 3.4 Merge Sort • Tree of calls of Merge (Figure 3.4)

  24. a n = 1, a a constant n > 1, c a constant – 3.4 Merge Sort • Time complexity of merge sort • When n=2k • T(n)= O(n log n)

  25. 3.4 Merge Sort • Two inefficiencies of MergeSort • Not in place (It uses another array b[].) • Copy between a[] and b[] needed • Space and time for stack due to recursion • For small set sizes, most of time consumed by recursion instead of sorting • Improvements • Using link[1:n] • Containing integers in the range [0,n], interpreted as pointers (indices) to elements of a[] • Example link: [1] [2] [3] [4] [5] [6] [7] [8] 6 4 7 1 3 0 8 0 • Q=2 denoting Q=(2,4,1,6) and sorted sublist (i.e., a[2]a[4] a[1]a[6]) • R=5 denoting R=(5,3,7,8) and sorted sublist (i.e., a[5]a[3] a[7]a[8])

  26. 3.4 Merge Sort • Improvements (Continued) • Changing the condition Small(P) • True if n16 • And use insertion sort for the small problem (since insertion sort works exceedingly fast on arrays of less than, say 16 elements) • Improved algorithm (Programs 3.10) int MergeSort1(int low, int high) // The global array a[low : high] is sorted in // nondecreasing order using the auxiliary array // link[low:high]. The values in link will // represent a list of the indices low through // high giving a[] in sorted order. A pointer // to the beginning of the list is returned. { if ((high-low+1)<16) return InsertionSort1(a, link, low, high); else { int mid = (low + high)/2; int q = MergeSort1(low, mid); int r = MergeSort1(mid+1, high); return(Merge1(q,r)); } }

  27. 3.4 Merge Sort • Improved algorithm (Programs 3.11) int Merge1(int q, int r) // q and r are pointers to lists contained in the global // array link[0:n]. link[0] is introduced only for // convenience and need not be initialized. The lists // pointed at by q and r are merged and a pointer to the // beginning of the merged list is returned. { int i=q, j=r, k=0; // The new list starts at link[0]. while (i && j) { // While both lists // are nonempty do if (a[i] <= a[j]) { // Find the smaller key. link[k] = i; k = i; i = link[i]; // Add a new key // to the list. } else { link[k] = j; k = j; j = link[j]; } } if (!i) link[k] = j; else link[k] = i; return(link[0]); }

  28. 3.4 Merge Sort • Example 3.8 (Table 3.4) • Write InsertionSort1(a,link,low,high) for yourself !

  29. 3.5 Quick Sort • Division • Partitioning a[1:n] into two subarrays a[1:m] and a[m+1:n] such that a[i]a[j] for all 1im and m+1jn • No need for merging • Each of two subarrays can be sorted independently

  30. 3.5 Quick Sort • Partitioning (due to C.A.R Hoare) (Program 3.12) int Partition(Type a[], int m, int p) // Within a[m], a[m+1],..., a[p-1] the elements // are rearranged in such a manner that if // initially t==a[m], then after completion // a[q]==t for some q between m and p-1, a[k]<=t // for m<=k<q, and a[k]>=t for q<k<p. q is returned. { Type v=a[m]; int i=m, j=p; do { do i++; while (a[i] < v); do j--; while (a[j] > v); if (i < j) Interchange(a, i, j); } while (i < j); a[m] = a[j]; a[j] = v; return(j); } inline void Interchange(Type a[], int i, int j) // Exchange a[i] with a[j]. { Type p = a[i]; a[i] = a[j]; a[j] = p; }

  31. 3.5 Quick Sort • Example 3.9 • Hand simulation (i.e., partitioned into [60 45 50 55] 65 [85 80 75 70]) • Do yourself for another example ! [15 22 13 27 12 10 20 25] • Why + at a[n+1]? • Explain using the example 1 2 3 4 5 6 7 8 9 10 [65 12 15 11 16 20 23 17 21 + ]

  32. 3.5 Quick Sort • Quick sort (Program 3.13) void QuickSort(int p, int q) // Sorts the elements a[p],..., a[q] which reside in // the global array a[1:n] into ascending order; a[n+1] // is considered to be defined and must be >= all the // elements in a[1:n]. { if (p < q) { // If there are more than one element // divide P into two subproblems. int j = Partition(a, p, q+1); // j is the position of the // partitioning element. // Solve the subproblems. QuickSort(p, j-1); QuickSort(j+1,q); // There is no need for combining solutions. } }

  33. 3.5 Quick Sort • Time complexity • Basic operation: element comparison • Assumption • n elements are distinct • Partitioning element v=a[m] has an equal probability of being the i-th smallest element, 1ip-m • Worst case: O(n2) • Exercise 7 (on p.163) • On what input data does QuickSort exhibit its worst-case behavior? • Answer (a) for the case in which the partitioning element is selected according to the median of three rule. • Average case: O(n log n)

  34. 3.5 Quick Sort • Iterative version of quick sort (Program 3.14) • Smaller of two subarrays always sorted first by the iterative version • Maximum stack depth • QuickSort (recursive version): n-1 • QucikSort1 (iterative version): O(log n) void QuickSort2(int p, int q) // Sorts the elements in a[p:q]. { Stack<int> stack(SIZE); // SIZE is 2*log(n). do { while (p < q) { int j = Partition(a, p, q+1); if ((j-p) < (q-j)) { stack.Add(j+1); stack.Add(q); q = j-1; } else { stack.Add(p); stack.Add(j-1); p = j+1; } }; // Sort the smaller subfile. if (stack.StackEmpty()) return; stack.Delete(q); stack.Delete(p); } while (1); }

  35. 3.5 Quick Sort • Performance measurement • Experiments • Comparison between QuickSort and MergeSort on Sun 10/30 • Data made by random integer generator in the range [0,1000] • Average-case on 50 random inputs (Table 3.5)

  36. 3.5 Quick Sort • Performance measurement • Worst-case on 50 random inputs (Table 3.6) • QuickSort always faster than MergeSort in Tables 3.5 and 3.6 • QuickSort usually perform better in practice (though both algorithm have O(n log n))

  37. 3.5 Quick Sort • Randomized sorting algorithms (Program 3.15) • With the aim of selecting a better partitioning element void RQuickSort(int p, int q) // Sorts the elements a[p],..., a[q] which reside in // the global array a[1:n] into ascending order. a[n+1] // is considered to be defined and must be >= all the // elements in a[1:n]. { if (p < q) { if ((q-p)>5) Interchange(a, random()%(q-p+1)+p, p); int j = Partition(p, q+1); // j is the position of // the partitioning element. RQuickSort(p,j-1); RQuickSort(j+1,q); } }

  38. 3.6 Selection • Problem • Select k-th smallest element from a[1:n] • Algorithm using partitioning (Program 3.17) • Complexity • Worst case: O(n2) • Average case: O(n) (Theorem 3.3) void Select1(Type a[], int n, int k) // Selects the kth-smallest element in a[1:n] and // places it in the kth position of a[]. The // remaining elements are rearranged such that // a[m] <= a[k] for 1 <= m < k, and // a[m] >= a[k] for k < m <= n. { int low=1, up=n+1; a[n+1] = INFTY; // a[n+1] is set to infinity. do { // Each time the loop is entered, // 1<=low<=k<=up<=n+1. int j = Partition(a, low, up); // j is such that a[j] is the // jth-smallest value in a[]. if (k == j) return; else if (k < j) up = j; // j is the new upper limit. else low = j+1; // j+1 is the new lower limit. } while (TRUE); }

  39. • • • • • • • • • • • • 3.8 Convex Hull • Definitions • Convex hull: the smallest convex polygon containing all the points of S • Extreme point: vertex of convex hull • Example (Figure 3.6)

  40. 3.8 Convex Hull • A simple algorithm • Algorithm for(each point p in S) Look at each triplet of points and see whether p lies in the triangle formed by these three points; if p lies in any such triangle, it is not extreme, otherwise it is; • Time • (n3) to check a point, since there are (n3) possible triangles • Total time is (n4), since there are n points

  41. q p2 p1 3.8 Convex Hull • Some geometric primitives • q is to the left (right) of <p1,p2> if the angle p1p2q is a left (right) turn • Signed area of p1(x1,y1), p2(x2,y2), and p3(x3,y3) • One-half of det(A) where A= • Positive if p3 is at the left of <p1,p2>, negative otherwise

  42. 3.8 Convex Hull • The QuickHull algorithm • Similar to QuickSort • Algorithm QuickHull • Identify two points p1 and p2 with the smallest and largest x-coordinate values (both p1 and p2 are extreme points) • Divide the point set X into X1 and X2 such that X1 (X2) has all the points to the left (right) of <p1,p2> • Compute the upper hull for X1 and lower hull for X2 using the recursive algorithm Hull • Merge two hulls into one

  43. 3.8 Convex Hull • The QuickHull algorithm (Continued) • Algorithm Hull • Divide • Pick the point p with the largest area formed by p1, p, and p2 (call such point p3) // partitioning element (Figure 3.7) • Partition the points so that one set contains the points to the left of <p1,p3> and the other set contains the ones to the left of <p3,p2> • Eliminate the remaining points // they are interior points • Apply Hull recursively to each of two subsets • Merge • Place one convex hull next to other in the right order

  44. • • • • • • • • • • • • • • 3.8 Convex Hull • The QuickHull algorithm (Continued) • Identifying the partitioning element (Figure 3.7)

  45. 3.8 Convex Hull • The QuickHull algorithm (Continued) • Time • Algorithm Hull • m=number of point in point set • Partitioning: O(m) • Merging: O(1) • Recurrence relation • T(m) = T(m1) + T(m2) + O(m) // similar to QuickSort • Worst case: O(m2) • Average case: O(m log m) • Algorithm QuickHull • Worst case: O(n2) • Average case: O(n log n)

  46. 3.8 Convex Hull • Graham’s scan algorithm • Algorithm • Identify the point p with the lowest y-coordinate value • Sort the points of S according to the angle subtended by the points and p with the positive x-axis // Figure 3.8 • Considering three successive points p1, p2, and p3 at a time • If left turn, move to the next point • Otherwise, delete p2 and move one point back in the list by setting p1 to its predecessor • Repeat until the point p is reached • Example (Figure 3.8) 4 • 6 3 • • 5 • 7 • 2 1 • • 9 • 8 • • 10 • P

  47. 3.8 Convex Hull • Graham’s scan algorithm (Continued) • C++ programs (Program 3.21) # define NIL 0 # include<iostream.h> struct point { float x, y; struct point *prev, *next; }; typedef struct point Type; class PointSet { private: Type* ptslist; float Area(float, float, float, float, float, float); void PrintList(Type *); void Scan(Type *); void Sort(Type *); public: PointSet() {ptslist = NIL;} void Insert(float a, float b); void ConvexHull(); };

  48. void PointSet::Scan(Type* list) { Type *p=list, *p1=list, *p2, *p3; float temp; do { p2 = p1->next; if (p2->next) p3 = p2->next; else p3 = p; temp = Area(p1->x, p1->y, p2->x, p2->y, p3->x, p3->y); if (temp>=0.0) p1 = p1->next; // If p1,p2,p3 form a left turn // move one point ahead; // if not delete p2 and move back. else { p1->next=p3; p3->prev=p1; delete p2; p1 = p1->prev; } } while (!((p3 == p) && (temp>=0.0))); } void PointSet::ConvexHull() { // Find the point p in ptslist of lowest // x-coordinate. Sort the points according // to the angle made with p and x-axis. Sort(ptslist); Scan(ptslist); PrintList(ptslist); }

  49. 3.8 Convex Hull • Graham’s scan algorithm (Continued) • Time • Scan: O(n) • Testing left or right turn for each triplet: O(1) • Sorting: O(n log n) • So, total time: O(n log n)

  50. 3.8 Convex Hull • DCHull algorithm • O(n log n) divide-and-conquer algorithm • Left for your self-study !

More Related