Tirgul 4

1. Tirgul 4 Subjects of this Tirgul: Counting Sort Radix Sort Bucket Sort

2. Linear-time Sorting • The lower-bound running time for comparison sorting algorithms is • The catch • cannot use anything but pair wise comparisons • No prior information on the elements compared • With more information we can do better

3. Counting Sort • The input –n integers in the range [0,k-1] (k is an integer) • When k = O(n) , the total running time of the sort is also O(n) • We use three arrays for the sort: in[0..n-1]-The input array out[0..n-1]-The output array C[0..k-1]-A temporary array (the counting array)

4. Counting Sort • The idea • For each element x, how many elements are less than x? • If there are 6 elements less than x, it belongs in the 7thposition (in the output array) • If there are several elements with the same value we locate them one after the other.

5. Counting Sort publicstaticvoid CountingSort(int[] in, int[] out, int k) { int[] c = new int[k]; // now c[i] = 0 for all i for (int i = 0; i<in.length ; i++) c[in[i]] ++; // now c[i-1] holds the number of elements in the input equal to i for (int i = 1 ; i<c.length ; i++ ) c[i] += c[i-1]; // now c[i-1] holds the number of elements in the input which are <= i for (int i=in.length-1 ; i>=0 ; i--) { out[ c[in[i]] -1 ] = in[i]; c[in[i]] --; //We make sure that equal elements will be located one after the other. } }

6. Counting Sort - example • First loop in: 3 6 4 1 3 4 1 4 C: 0 2 0 2 3 0 1 • Second loop C: 0 2 2 4 7 7 8 • Third loop: • first cycle: out: 0 0 0 0 0 0 4 0 C: 0 2 2 4 6 7 8 • final cycle: out: 1 1 3 3 4 4 4 6 C: 0 0 2 2 4 7 7

7. Counting Sort • Running time • The first for loop runs inO(n) • The second for loop runs in O(k) • The third for loop runs in O(n) • The total run time O(n+ k) • If k = O(n) the total run time will be O(n). • We gain run time. Counting sort’s run time is shorter than any comparison sort. • Advantage: stable • Disadvantage: requires additional memory (not in place)

8. Radix Sort • Some history…Once upon a time in order to compile their programs the poor programmers had to punch cards. They had to punch numbers with 80 digits, each digit in base 12. The compilation process was operated by a miserable guy called the operator. This operator had to feed ordered cards to the computer. Unfortunately he received the cards in messy jumbles from the programmers. • One day, in his desperation, the poor operator invented Radix Sort.

9. Radix Sort • Input: d-digit numbers, each digit in the range 0..k-1. • First intuition: • sort from most significant to least significant digit. • Recursively sort every sub pile. • The problem: many sub piles to keep track of when the recursion done. • Radix sort idea: • sort from least significant to most significant digit. • each time by one digit only. Use stable sort each time.

10. Radix Sort RADIX-SORT(A,d) for i 1 tod do use a stable sort to sort array A on digit i • When every digit is in the range [0,k-1] and k is not too big we use Counting Sort. • Running time: • For every digit O(n+ k) • For d digits O(dn+ dk) • Total run time, when d is a constant and k = O(n), is O(n) .

11. 329 720 720 329 457 355 329 355 657 436 436 436 839 457 839 457 From least significant digit 436 657 355 657 720 329 457 720 355 839 657 839 Radix Sort • Why does it work? The trick is by using the stable sort. Example:

12. Radix Sort • Prove that the Radix Sort works properly. • Proof by induction. • Definition for the proof: • For any number x in the input, define xi as the number excepted after considering only the i list significant digits of the number x. • For example x =531 x2=31

13. Radix Sort • We claim that after i passages of the algorithm the group of numbers xi is sorted. • Induction base: i =1 If xi<yix<y (we have only one digit) • Assume that the Radix Sort works properly until iteration i-1 and prove the claim for the i’th iteration.

14. Radix Sort • Assume that xi<yi . There are two options. • The i’th digit of x < than the i’th digit of y. • The i’th digit of x and y are equal. (xi<yi do to the previews i-1 digits) • In the first case i’th iteration sort according to the i’th digit • Since we assume xi-1<yi-1 after the i-1’th iteration and the i’th iteration sort according to the i’th digit using a stable sort xi<yi

15. Questions • How to sort n integers in the range 1 to n2 in O(n) time? • Suppose you have an array of n data records to sort and that the key of each record has the value 0/1. Give a linear time algorithm for sorting these records with no extra space. • Can the algorithm from the previous question be applied to radix sort n records with b-bits keys in O(b*n) time?

16. Bucket Sort • The input –n real numbers in the interval [0,1) (i.e. ), distributed uniformly • The running time is O(n) on the average • The idea • Divide the interval into n‘buckets’. Put each element into its matching bucket. As the numbers are uniformly distributed, not too many elements will be placed in each bucket – so use insertion sort to sort each bucket, and then concatenate the contents into a single list

17. Bucket Sort • Running time: • The first loop and last step (concatenation) are O(n). • The insertion sorts: • Denote by nithe number of elements in bucket i. • The expected time to sort bucket i is therefore • The total • An element fits into each bucket with probability 1/n so P(ni=k) is binomial:

18. Bucket Sort • The total average running time of the sort is O(n)