Data Structures and Algorithms

1. Data Structures and Algorithms Sorting I Gal A. Kaminka Computer Science Department

2. What we’ve done so far • We’ve talked about complexity • O(), run-time and space requirements • We’ve talked about ADTs • Implementations for: • Stacks (2 implementations) • Queues (2 implementations) • Sets (4 implementations)

3. Sorting • Take a set of items, order unknown • Set: Linked list, array, file on disk, … • Return ordered set of the items • For instance: • Sorting names alphabetically • Sorting by height • Sorting by color

4. Sorting Algorithms Issues of interest: • Running time in worst case, other cases • Space requirements • In-place algorithms: require constant space • The importance of empirical testing Often Critical to Optimize Sorting

5. Short Example: Bubble Sort Key: “large unsorted elements bubble up” • Make several sequential passes over the set • Every pass, fix local pairs that are not in order Considered inefficient, but useful as first example

6. (Naïve)BubbleSort(array A, length n) • for in to 2// note: going down • for j2 to i // loop does swaps in [1..i] • if A[j-1]>A[j] • swap(A[j-1],A[j])

7. Pass 1: 25 57 48 37 12 92 86 33 25 48 57 37 12 92 86 33 25 48 37 57 12 92 86 33 25 48 37 12 57 92 86 33 25 48 37 12 57 86 92 33 25 48 37 12 57 86 33 92

8. Pass 2: 25 48 37 12 57 86 33 92 25 37 48 12 57 86 33 92 25 37 12 48 57 86 33 92 25 37 12 48 57 33 86 92 Pass 3: 25 37 12 48 57 33 86 92 25 12 37 48 57 33 86 92 25 12 37 48 33 57 86 92

9. Pass 4: 25 12 37 48 33 57 86 92 12 25 37 48 33 57 86 92 12 25 37 33 48 57 86 92 Pass 5: 12 25 37 33 48 57 86 92 12 25 33 37 48 57 86 92 Pass 6: 12 25 33 37 48 57 86 92 Pass 7: 12 25 33 37 48 57 86 92

10. Bubble Sort Features • Worst case: Inverse sorting • Passes: n-1 • Comparisons each pass: (n-k) where k pass number • Total number of comparisons: (n-1)+(n-2)+(n-3)+…+1 = n2/2-n/2 = O(n2) • In-place: No auxilary storage • Best case: already sorted • O(n2) Still: Many redundant passes with no swaps

11. BubbleSort(array A, length n) • in • quitfalse • while (i>1 AND NOT quit) // note: going down • quittrue • for j=2 to i // loop does swaps in [1..i] • if A[j-1]>A[j] • swap(A[j-1],A[j]) // put max in I • quitfalse • ii-1

12. Bubble Sort Features • Best case: Already sorted • O(n) – one pass over set, verifying sorting • Total number of exchanges • Best case: None • Worst case: O(n2) Lots of exchanges: A problem with large items

13. Selection Sort • Observation: Bubble-Sort uses lots of exchanges • These always float largest unsorted element up • We can save exchanges: • Move largest item up only after it is identified • More passes, but less total operations • Same number of comparisons • Many fewer exchanges

14. SelectSort(array A, length n) • for in to 2 // note we are going down • largest  A[1] • largest_index  1 • for j1 to i // loop finds max in [1..i] • if A[j]>A[largest_index] • largest_index  j • swap(A[i],A[largest_index]) // put max in i

15. Initial: 25 57 48 37 12 92 86 33 Pass 1: 25 57 48 37 12 33 86| 92 Pass 2: 25 57 48 37 12 33 I 86 92 Pass 3: 25 33 48 37 12 I 57 86 92 Pass 4: 25 33 12 37I 48 57 86 92 Pass 5: 25 33 12 I 37 48 57 86 92 Pass 6: 25 12 I 33 37 48 57 86 92 Pass 7: 12 I 25 33 37 48 57 86 92

16. Selection Sort Summary • Best case: Already sorted • Passes: n-1 • Comparisons each pass: (n-k) where k pass number • # of comparisons: (n-1)+(n-2)+…+1 = O(n2) • Worst case: Same. • In-place: No external storage • Very few exchanges: • Always n-1 (better than Bubble Sort)

17. Selection Sort vs. Bubble Sort • Selection sort: • more comparisons than bubble sort in best case O(n2) • But fewer exchanges O(n) • Good for small sets/cheap comparisons, large items • Bubble sort: • Many exchanges O(n2) in worst case • O(n) on sorted input

18. Insertion Sort • Improve on # of comparisons • Key idea: Keep part of array always sorted • As in selection sort, put items in final place • As in bubble sort, “bubble” them into place

19. InsertSort(array A, length n) • for i2 to n // A[1] is sorted • y=A[i] • j  i-1 • while (j>0 AND y<A[j]) • A[j+1]  A[j] // shift things up • jj-1 • A[j+1]  y // put A[i] in right place

20. Initial: 25 57 48 37 12 92 86 33 Pass 1: 25 | 57 48 37 12 92 86 33 Pass 2: 25 57 I 48 37 12 92 86 33 25 48 | 57 37 12 92 86 33 Pass 3: 25 48 57 | 37 12 92 86 33 25 48 57 | 57 12 92 86 33 25 48 48 | 57 12 92 86 33 25 37 48 | 57 12 92 86 33

21. Pass 4: 25 37 4857 | 12 92 86 33 25 37 48 57 | 57 92 86 33 25 37 48 48 | 57 92 86 33 25 37 37 48 | 57 92 86 33 25 25 37 48 | 57 92 86 33 12 25 37 48 | 57 92 86 33

22. Pass 5: 12 25 37 48 57 | 92 86 33 Pass 6: 12 25 37 48 57 92 | 86 33 12 25 37 48 57 86 | 92 33 Pass 7: 12 25 37 48 57 86 92 | 33 12 25 37 48 57 86 92 | 92 12 25 37 48 57 86 86 | 92 12 25 37 48 57 57 86 | 92 12 25 37 48 48 57 86 | 92

23. Pass 7: 12 25 37 48 48 57 86 | 92 12 25 37 37 48 57 86 | 92 12 25 33 37 48 57 86 | 92

24. Insertion Sort Summary • Best case: Already sorted  O(n) • Worst case: O(n2) comparisons • # of exchanges: O(n2) • In-place: No external storage • In practice, best for small sets (<30 items) • BubbleSort does more comparisons! • Very efficient on nearly-sorted inputs

25. Divide-and-Conquer An algorithm design technique: • Divide a problem of size N into sub-problems • Solve all sub-problems • Merge/Combine the sub-solutions This can result in VERY substantial improvements

26. Small Example: f(x) • if x = 0 OR x = 1 • return 1 • else • return f(x-1) + f(x-2) What is this function?

27. Small Example: f(x) • if x = 0 OR x = 1 • return 1 • else • return f(x-1) + f(x-2) What is this function? Fibbonacci!

28. Divide-and-Conquer in Sorting • Mergesort • O(n log n) always, but O(n) storage • Quick sort • O(n log n) average, O(n^2) worst • Good in practice (>30), O(log n) storage