Sorting

1. Sorting Chapter 11

2. Sorting • Consider list x1, x2, x3, … xn • We seek to arrange the elements of the list in order • Ascending or descending • Some O(n2) schemes • easy to understand and implement • inefficient for large data sets

3. Categories of Sorting Algorithms • Selection sort • Make passes through a list • On each pass reposition correctly some element

4. Categories of Sorting Algorithms • Insertion sort • Repeatedly insert a new element into an already sorted list • Note this works well with a linked list implementation

5. Categories of Sorting Algorithms • Exchange sort • Systematically interchange pairs of elements which are out of order • Bubble sort does this All these have computing time O(n2)

6. Improved Schemes • We seek improved computing times for sorts of large data sets • Chapter presents schemes which can be proven to have worst case computing time O( n log2n ) • Heapsort • Quicksort

7. Heaps A heap is a binary tree with properties: • It is complete • Each level of tree completely filled • Except possibly bottom level (nodes in left most positions) • It satisfies heap-order property • Data in each node >= data in children

8. Heaps • Example 22 28 12 24 22 24 14 28 12 14 Heap Not a heap – Why?

9. Implementing a Heap • Use an array or vector • Number the nodes from top to bottom • Number nodes on each row from left to right • Store data in ith node in ith location of array (vector)

10. Implementing a Heap • If heapis the name of the array or vector used, the items in previous heap is stored as follows:heap[0]=78; heap[1]=56; heap[2]=32;heap;[3]=45; heap;[4]=8; heap[5]=23;heap[6]=19;

11. Implementing a Heap • In an array implementation children of ith node are at heap[2*i+1] and heap[2*(i+1)] • Parent of the ithnode is atheap[(i-1)/2]

12. Converting a Complete Binary Tree to a Heap • Percolate down the largest value

13. Tighter Analysis of Complexity of Build Heap Heap • Suppose we have n elements. • We want to build a heap of n elements. • We present an O(n) algorithm to do so. • Insert these elements in n positions of an array. • Is an almost complete binary tree. • But does not satisfy the heap order • So we need to interchange these elements to obtain heap order

14. Percolate Down Heap Parents are at position n/2 to 1. If we percolate these down suitably, (interchange with lesser of the two children, if it is less than either of the two children, and so on), then we will get heap order. For (j = n/2; j--; j >0) Percolate Down (j);

15. Percolate Down PercolateDown (j) { If (2j > n) break; If Heap[j]  min(Heap[2j],Heap(2j+1), break; If (Heap[2j] < Heap[2j + 1]) { interchange(Heap[j], Heap[2j]); Percolate Down (2j); } else { interchange(Heap[j], Heap[2j+1]); Percolate Down (2j+1); } }

16. Convert Complete Binary Tree to a Heap

17. Convert Complete Binary Tree to a Heap

18. Heapsort Consider array x as a complete binary tree and use the Heapify algorithm to convert this tree to a heap. 1. For i = n down to 2: Interchange x[1] and x[i], thus putting the largest element in the sublist x[1],...,x[i] at end of sublist. 2. Apply the PercolateDown algorithm to convert the binary tree corresponding to the sublist stored in positions 1 through i - 1 of x.

19. Heapsort • In PercolateDown, the number of items in the subtree considered at each stage is one-half the number of items in the subtree at the preceding stage. Thus, the worst-case computing time is O(log 2n). • Heapify algorithm executes PercolateDown n/2 times: worst-case computing time is O(nlog2n). • Heapsort executes Heapify one time and PercolateDown n - 1 times; consequently, its worst-case computing time is O(n log2n). • Internet Animation Applet

20. Heapsort

21. Heapsort • Note the way thelarge values arepercolated down

22. Quicksort • A more efficient exchange sorting scheme than bubble sort • A typical exchange involves elements that are far apart • Fewer interchanges are required to correctly position an element. • Quicksort uses a divide-and-conquer strategy • A recursive approach • The original problem partitioned into simpler sub-problems, • Each sub problem considered independently. • Subdivision continues until sub problems obtained are simple enough to be solved directly

23. Quicksort • Choose some element called a pivot • Perform a sequence of exchanges so that • All elements that are less than this pivot are to its left and • All elements that are greater than the pivot are to its right. • Divides the (sub)list into two smaller sub lists, • Each of which may then be sorted independently in the same way.

24. Quicksort If the list has 0 or 1 elements, return. // the list is sorted Else do: Pick an element in the list to use as the pivot.   Split the remaining elements into two disjoint groups: SmallerThanPivot = {all elements < pivot} LargerThanPivot = {all elements > pivot}  Return the list rearranged as: Quicksort(SmallerThanPivot), pivot, Quicksort(LargerThanPivot).

25. Quicksort Example • Given to sort:75, 70, 65, , 98, 78, 100, 93, 55, 61, 81, • Select, arbitrarily, the first element, 75, as pivot. • Search from right for elements <= 75, stop at first element <75 • Search from left for elements > 75, stop at first element >=75 • Swap these two elements, and then repeat this process 68 84

26. Quicksort Example 75, 70, 65, 68, 61, 55, 100, 93, 78, 98, 81, 84 • When done, swap with pivot • This SPLIT operation placed pivot 75 so that all elements to the left were <= 75 and all elements to the right were >75. • See code page 602 • 75 is now placed appropriately • Need to sort sublists on either side of 75

27. Quicksort Example • Need to sort (independently): 55, 70, 65, 68, 61 and 100, 93, 78, 98, 81, 84 • Let pivot be 55, look from each end for values larger/smaller than 55, swap • Same for 2nd list, pivot is 100 • Sort the resulting sublists in the same manner until sublist is trivial (size 0 or 1)

28. Quicksort • Note thepartitionsand pivotpoints • Note codepgs 602-603 of text • InternetExample

29. Quicksort Performance • is the average case computing time • If the pivot results in sublists of approximately the same size. • O(n2) worst-case • List already ordered, elements in reverse • When Split() repetitively results, for example, in one empty sublist

30. Improvements to Quicksort • Quicksort is a recursive function • stack of activation records must be maintained by system to manage recursion. • The deeper the recursion is, the larger this stack will become. • The depth of the recursion and the corresponding overhead can be reduced • sort the smaller sublist at each stage first

31. Improvements to Quicksort • Another improvement aimed at reducing the overhead of recursion is to use an iterative version of Quicksort() • To do so, use a stack to store the first and last positions of the sublists sorted "recursively".

32. Improvements to Quicksort • An arbitrary pivot gives a poor partition for nearly sorted lists (or lists in reverse) • Virtually all the elements go into either SmallerThanPivot or LargerThanPivot • all through the recursive calls. • Quicksort takes quadratic time to do essentially nothing at all.

33. Improvements to Quicksort • Better method for selecting the pivot is the median-of-three rule, • Select the median of the first, middle, and last elements in each sublist as the pivot. • Often the list to be sorted is already partially ordered • Median-of-three rule will select a pivot closer to the middle of the sublist than will the “first-element” rule.

34. Improvements to Quicksort • For small files (n <= 20), quicksort is worse than insertion sort; • small files occur often because of recursion. • Use an efficient sort (e.g., insertion sort) for small files. • Better yet, use Quicksort() until sublists are of a small size and then apply an efficient sort like insertion sort.

35. Mergesort • Sorting schemes are either … • internal -- designed for data items stored in main memory • external --designed for data items stored in secondary memory. • Previous sorting schemes were all internal sorting algorithms: • required direct access to list elements • not possible for sequential files • made many passes through the list • not practical for files

36. Mergesort • Mergesort can be used both as an internal and an external sort. • Basic operation in mergesort is merging, • combining two lists that have previously been sorted • resulting list is also sorted.

37. Mergesort • The general algorithm for the merge sort is as follows: • if list is of size greater than 1 • Divide the list into two sublists. • Merge sort the first sublist. • Merge sort the second sublist. • Merge the first sublist and the second sublist.

38. Merge Flow Chart Another example was the file merge program done as last assignment in CS2 Open files, read 1st records Trans Key < OM key Trans key > OM key Compare keys Write OM record to NM file, Trans key yet to be matched Trans key = = OM key Type of Trans Type of Trans Other, Error Del, go on Other, Error Add OK Modify, Make changes