Donald E. Knuth (1938---)

1. Donald E. Knuth (1938---) The father of algorithm analysis popularizing asymptotic notation. The author of TAOCP -- The Art of Computer Programming • Volume 1: Fundamental Algorithms • Volume 2: Seminumerical Algorithms • Volume 3: Sorting and Searching (among tons of others) “ I have been a happy man ever since January 1, 1990, when I no longer had an email address.” ITK 168

2. Donald E. Knuth TAOCP – The Art of Computer Programming • Volume 1: Fundamental Algorithms • Basic Concepts and Information Structures • Volume 2: Seminumerical Algorithms • Random numbers and Arithmetic • Volume 3: Sorting and Searching • Sorting and Searching • Volume 4: Combinatorial Algorithms • Combinatorial Searching and Recursion • Volume 5: Syntactical Algorithms • Lexical Scanning and Parsing Techniques • Volume 6: Theory of Languages • Mathematical Linguistics • Volume 7: Compiler • Programming Language Translation Highlighted in the preface of Vol. 1 ITK 168

3. Sorting: Arranging items into a certain order. A collection of items can be stored in a one dimensional array (or linked list, sets or trees, which are more advanced data structure you can learn from ITK179 and ITK279.) Sometimes, we prefer using one dimensional array due to its random access propertry in the internal memory ITK 168

4. Selection Sort: Select the right item into the right place. The right item The right place swap ITK 168

5. Selection Sort: Select the right item into the right place. The right place The right item swap ITK 168

6. Selection Sort: Select the right item into the right place. The right item The right place swap ITK 168

7. Selection Sort: Select the right item into the right place. The right item The right place swap ITK 168

8. Selection Sort: Select the right item into the place. ITK 168

9. Selection sort algorithm in Java publicstaticclass SelectionSort { publicvoid sort(int a[]) { for (int i = 0; i<a.length-1; i++) { // select one for a[i] int j = min(a,i); exchange(a,i,j); } } // select the minimum between a[s] to the end privateint min(int a[], int s) {int m = s; for (int i=s; i<a.length;i++) if (a[i] < a[m]) m=i; return m; } privatevoid exchange(int[] a, int i, int j) { int temp = a[i]; a[i] = a[j]; a[j] = temp; } } // end of SelectionSort ITK 168

10. Insertion Sort: Insert an item into the right place. sorted 7 ITK 168

11. Insertion Sort: Insert an item into the right place. sorted ITK 168

12. Insertion Sort: Insert an item into the right place. sorted 7 8 9 10 12 7 ITK 168

13. Insertion Sort: Insert an item into the right place. ITK 168

14. Insertion sort algorithm in Java publicstaticclass InsertionSort { publicvoid sort(int a[]) { for (int i = 1;i<a.length;i++) insert(a,i); // insert a[i] into a[0]..a[i-1] } privatevoid insert(int a[],int i){//inert a[i] into a[0...i-1] int v = a[i]; // v is the value to insert int j; for (j=i-1; j>=0; j--) { if (v < a[j]) // a[j] needs to shift down a[j+1] = a[j]; elsebreak; } a[j+1] = v; } } // end of InsertionSort ITK 168

15. Analysis of Insertion/Selection Sorts: T(n) = 1 +2 + 3 + ... + n = O(n2) Shell Sort in the worst case O(n3/2) O(n log n) Lower bound of sorting algorithms ITK 168

16. Sorting, Sorting, Sorting, and more Sorting ITK 168 students don’t have to “know” the details in the following slides but just “enjoy” the beatuty of algorithms ITK 168

17. Quick Sort < < < < < < < < < < < < < < < < < < < < < < < < < < ITK 168

18. Quick Sort 4 3 2 16 14 8 ITK 168

19. /* Quick Sort */ private void SwapTwo(int A[], int i, int j){ int temp = A[i]; A[i]=A[j]; A[j]=temp; } public void QuickSort(int A[], int h, int t){ // h: head // t: tail if (h == t) return; inti=h+1, j=t; if (i==j) { if (A[h] > A[i]) SwapTwo(A,h,i); return; } while (i < j) { while (A[h] >= A[i] && i < t) i++; while (A[h] <= A[j] && j > h) j--; if (i < j) SwapTwo(A,i++,j--); if (i==j && A[h]>A[i]) SwapTwo(A,h,i); } QuickSort(A,h,i-1); QuickSort(A,i,t); } Quick Sort ITK 168

20. Mergesort Merge two sorted lists 3 5 7 8 1 2 6 O(n) ITK 168

21. Merge Sort O(log n) O(n log n) ITK 168

22. Incremental insertion (selection) sort. Shell Sort: h = 4 ITK 168

23. Shell Sort: hk-sorted h = 4 4-sorted 2-sorted 1-sorted ITK 168

24. Shell Sort: Incremental insertion with Increment sequence, hk-sorted 1-sorted ht-sorted ht-1-sorted hk-1-sorted  (implies) hk-sorted hk-1-sorted 10 9 30 20 hk-sorted Shell suggested: Not a good suggestion, worst case: O(n2) ITK 168

25. publicstaticclass ShellSort { publicvoid sort(int a[]) { int h = a.length/2; while (h > 0) { // h-sort for (int i=h; i<a.length;i++) { int t=a[i], j=i; while (j >= h && a[j-h] > t) { a[j] = a[j-h]; j -= h; } a[j]=t; } h /= 2; } // end increment } } // end of ShellSort Shell sort algorithm in Java bad strategy h = 4 h = 4 h = 4 i h = 4 j j j j ITK 168

26. Shell Sort: using Shell’s suggestion Worst case analysis: O(n2) Idea: Right before the final sort, let the smallest n/2 be distributed in the even position. e.g. 2-sorted 1 2 3 4 5 6 7 8 final sort ITK 168

27. Shell Sort: using Shell’s suggestion What kind of initial array will result in such 2-sorted? Let Ah[s] = A[s], A[s+h], A[s+2h], .... Ah[1] Ah[0] even positions odd positions 5 13 1 11 3 Ah[2] 12 7 4 15 9 10 8 Ah[3] 6 2 14 16 Ah[5] Ah[6] Ah[4] Ah[7] elements will never cross the even-/odd-cell before the final sort 8-sort , 4-sort , 2-sort ITK 168

28. Shell Sort: hk-sorted 3-sorted A[p-6] A[p] 6 2-sorted 6 6 A[p-6] A[p] A[p-6] A[p] A[p-(x2+y3)] < A[p] ITK 168

29. Heapsort Percolating a non-heap Worst case: O(n) 34 2 2 15 5 30 7 7 6 15 2 23 5 9 30 21 7 25 6 23 9 21 15 6 32 23 5 40 11 31 30 21 25 7 15 32 34 40 11 31 30 25 ITK 168

30. Heapsort O(log n) 2 Percolating a non-heap O(log n-1) 5 Worst case: O(n) O(log n-2) 6 7 2 5 7 6 23 9 21 40 O(log 1) 15 32 34 40 11 31 30 25 O(n+n log n) = O(n log n) ITK 168