Outline

1. Outline • Priority Queues • Binary Heaps • Randomized Mergeable Heaps

2. Priority Queues • A priority queue stores a collection of Comparable elements • Operations: • add(x) : Add x to the collection • findMin() : Return the smallest element in the collection • deleteMin() : Remove the smallest element in the collection • In the JCF Queue interface, these are: • add(x)/offer(x) : add(x) • peek() : findMind() • remove()/poll() : deleteMin()

3. Priority Queues: Applications • We can sort a set of elements using a priority queue. How? • We just need to insert them all into a priority queue and then removing them publicstatic <T> void sort (T[] a) { Queue <T> pq = newPriorityQueue <T >(); for (T x : a) pq. add(x); for (inti = 0; i < a. length ; i++) a[i] = pq. remove (); }

4. Priority Queues: Applications • In simulations, priority queues store events ordered by time of occurrence

5. Priority Queues: Applications • In simulations, priority queues store events ordered by time of occurrence

6. Priority Queues: Applications • In simulations, priority queues store events ordered by time of occurrence

7. Priority Queues: Applications • In simulations, priority queues store events ordered by time of occurrence

8. Priority Queues: Applications • In simulations, priority queues store events ordered by time of occurrence

9. Priority Queues • Which structure should we use to implement a priority queue? • Priority queues can be implemented as (binary) heaps • A binary heap is a binary tree where each node u stores a value u.prio • Heap property: • u.prio < u.left.prio and u.prio < u.right.prio

10. Priority Queues • A binary heap is a binary tree where each node u stores a value u.prio • Heap property: • u.prio < u.left.prio and u.prio < u.right.prio 0.1 0.3 0.5 0.8 0.4 0.9 1.1 1.9 2.1 1.3 9.2 5.1

11. Priority Queues • A complete heap uses a complete binary tree: • A complete binary tree of height d has up to 2d+1 - 1 nodes • To store n nodes, we require 2d+1 - 1 ≥ n 0.1 0.3 0.5 0.8 0.4 0.9 1.1 1.9 2.1 1.3 9.2 5.1 • 2d+1 ≥ n + 1 • d+1 ≥ log2(n + 1) • d ≥ log2(n + 1) – 1 = O(log n) • A complete heap of size n has height O(log n)

12. Priority Queues • How can we maps the nodes of a complete binary tree to the positions of an array? • Eytzinger method: • a[0] is the root • the left child of a[i] is a[2*i+1] • the right child of a[i] is a[2*i+2] • the parent of a[i] is a[(i-1)/2]

13. Eytzinger method • Eytzinger method: 0 2 1 5 6 3 4 11 7 8 9 10 12 13 14 5 6 7 8 9 10 11 12 13 14 0 1 2 3 4

14. Eytzinger method • Eytzinger method: 0 2 1 5 6 3 4 11 7 8 9 10 12 13 14 root Left child Right child 5 6 7 8 9 10 11 12 13 14 0 1 2 3 4 a[2*0+1] = a[1] i = 0 a[2*0+2]= a[2]

15. Eytzinger method • Eytzinger method: 0 2 1 5 6 3 4 11 7 8 9 10 12 13 14 Left child Right child f g h i j k l m n o a b c d e a[2*5+1] = a[11] i = 5 a[2*0+2]= a[12]

16. Implicit Binary Heap • An implicit binary heap represents a complete binary heap in an array a using the Eytzinger method 0.1 0.3 0.5 0.8 0.4 0.9 1.1 1.9 2.1 1.3 9.2 5.1 0.8 0.4 2.1 1.3 9.2 5.1 1.9 0.1 0.5 0.3 0.9 1.1 • No extra pointers, all the data lives in a

17. Implicit Binary Heap publicclassBinaryHeap <T extends Comparable <T>> extendsAbstractQueue <T> { protected T[] a; protectedintn; protectedint left (inti) { return2*i + 1; } protectedint right (inti) { return2*i + 2; } protectedint parent (inti) { return(i -1)/2; } ... }

18. Finding the minimum • How to find the minimum value in a heap? • Finding the minimum value in a heap is easy • It's stored at the root • This is a[0] public T peek () { returna [0]; } • Runs in O(1) time

19. Inserting elements • How to insert an element into a heap? • To insert x into a (implicitbinary) heap, we • add x as a leaf • while x is smaller than parent(x) • swap x with its parent • Runs in O(log n) time

20. Inserting elements • add x as a leaf 0.1 0.3 0.5 0.8 0.4 0.9 1.1 1.9 2.1 1.3 9.2 5.1 0.2 0.2 0.2 0.8 0.4 2.1 1.3 9.2 5.1 1.9 0.1 0.5 0.3 0.9 1.1

21. Inserting elements • add x as a leaf • while x is smaller than parent(x) • swap x with its parent 0.1 0.3 0.5 0.2 0.4 0.9 1.1 1.9 2.1 1.3 9.2 5.1 0.8 0.2 0.8 0.2 0.4 2.1 1.3 9.2 5.1 1.9 0.1 0.5 0.3 0.9 1.1

22. Inserting elements • add x as a leaf • while x is smaller than parent(x) • swap x with its parent 0.1 0.2 0.5 0.3 0.4 0.9 1.1 1.9 2.1 1.3 9.2 5.1 0.8 0.2 0.8 0.3 0.4 2.1 1.3 9.2 5.1 1.9 0.1 0.5 0.2 0.9 1.1

23. Inserting elements • publicboolean offer (T x) { • if (n + 1 > a. length ) • grow (); • a[n++] = x; • bubbleUp (n -1); • returntrue ; • } • protectedvoidbubbleUp (inti) { • intp = parent (i); • while (i > 0 && a[i]. compareTo (a[p]) < 0) { • swap (i,p); • i = p; • p = parent (i); • } • }

24. Removing the minimum • How can we remove the minimum value from a heap? • To delete the minimum from a (implicitbinary) heap, we • copy x=a[n-1] into a[0] • while x is larger than either of its children • swap x with the smaller of its children • Runs in O(log n) time

25. Removing the minimum • copy x=a[n-1] into a[0] 0.1 0.2 0.5 0.3 0.4 0.9 1.1 1.9 2.1 1.3 9.2 5.1 0.8 0.8 0.3 0.4 2.1 1.3 9.2 5.1 1.9 0.1 0.5 0.2 0.9 1.1

26. Removing the minimum • copy x=a[n-1] into a[0] • while x is larger than either of its children • swap x with the smaller of its children 0.2 0.8 0.8 0.2 0.5 0.3 0.4 0.9 1.1 1.9 2.1 1.3 9.2 5.1 0.8 0.3 0.4 2.1 1.3 9.2 5.1 1.9 0.2 0.8 0.8 0.5 0.2 0.9 1.1

27. Removing the minimum • copy x=a[n-1] into a[0] • while x is larger than either of its children • swap x with the smaller of its children 0.2 0.3 0.8 0.5 0.8 0.3 0.4 0.9 1.1 1.9 2.1 1.3 9.2 5.1 0.8 0.8 0.3 0.4 2.1 1.3 9.2 5.1 1.9 0.2 0.8 0.5 0.8 0.9 1.1 0.3

28. Removing the minimum • public T poll () { • T x = a [0]; • swap (0, --n); • trickleDown (0); • returnx; • }

29. Removing the minimum • protectedvoidtrickleDown (inti) { • do { • intj = -1; • intr = right (i); • if (r < n && a[r]. compareTo (a[i]) < 0) { • intl = left (i); • if (a[l]. compareTo (a[r]) < 0) { j = l; } • else { j = r; } • } else { • intl = left (i); • if (l < n && a[l]. compareTo (a[i]) < 0) { j = l; } • } • if (j >= 0) swap (i, j); • i = j; • } while (i >= 0); • }

30. Summary • Theorem: A (implicit) binary heap supports the operations add(x) and deleteMin() in O(log n) (amortized) time and supports the findMin() operation in O(1) time.

31. Building a heap • How can we build a heap from an unsorted array a? • How quickly can we make a into an implicit binary heap? • We can insert elements a[0],...,a[n-1] one at a time. • publicBinaryHeap (T[] ia) { • a = ia; • for (intn = 1; n < a. length ; n++) { add(a[n]); } • } • This takes O(1 + log(1)) + O(1 + log(2)) + O(1 + log n) • Runs in O(nlog n) time Can we do better?

32. Building a heap • We can do it in O(n) time. How? • By working bottom up. • First build n/2 heaps of size 1 • Next build n/4 heaps of size 3 • Next build n/8 heaps of size 7 … • build 1 heap of size n • publicBinaryHeap (T[] ia) { • a = ia; • n = a. length ; • for (inti = n/2; i >= 0; i --) { trickleDown (i); } • }

33. Building a heapin linear time (analysis) • We call trickleDown(i), n/2j times where j is the root of a heap of size 2j - 1 • trickleDown(i) then takes O(log(2j - 1)) = O(j) time • Total running time is • (n/4) * O(1) • (n/8) * O(2) • (n/16) * O(3) … • 1 * O(log n) • = O(n)

34. Heapsort • The heapsort algorithm for sorting an array a of length n: • Make a into a heap in (O(n) time) • Repeat n times • Delete the minimum • public T deleteMin () { • swap (0, --n); • trickleDown (0); • } • Each deletion takes O(log n) time • Runs in O(n log n) time • Doesn't require any extra space ( does all work inside of input array a)

35. Merging two heaps • We know how to add one element to a heap. • What can we do if we want to add another heap? • In other words, how can we merge 2 heaps? publicNode<T> merge(Node<T> h1, Node<T> h2){ while(h2.size()>0){ T u = h2.peek(); h2.deleteMin(); h1.add(u); } } • The cost for deleteMin() and add() is O(log n) • We perform those operations n times • Total cost O(n log n)

36. Merging two heaps • Can we do better? • Actually we can do it in O(log n). How? • merge(h1,h2) can be defined recursively • If either of h1 or h2 is nil, we just return h2 or h1. • We take the one with the biggestminimumvalue and merge it either with the right or left child of the other. • We continue the process recursively.

37. Merging two heaps • What is the complexity of this operation? • in an implicit binary heap it would be O(n) • we need to copy all the elements in a new array • How can we reduce it? • Using a heap ordered binary tree structure we can reduce the cost to O(log n).Why?

38. Merging two heaps Node<T> merge(Node<T> h1, Node<T> h2) { if(h1 == nil) returnh2; if(h2 == nil) returnh1; if(compare(h2.x, h1.x) < 0) return merge(h2, h1); // now we know h1.x <= h2.x if(rand.nextBoolean()) { h1.left = merge(h1.left, h2); h1.left.parent = h1; } else { h1.right = merge(h1.right, h2); h1.right.parent = h1; } returnh1; }

39. Merging two heaps • Why we need the random function to merge the heaps? • How can we define the add() and remove() operations in a heap ordered binary tree ? public booleanadd(T x) { Node<T> u = newNode(); u.x= x; r= merge(u, r); r.parent= nil; n++; returntrue; } public T remove() { T x = r.x; r= merge(r.left, r.right); if(r != nil) r.parent = nil; n--; returnx; }

40. Analysis of merge(h1,h2) • Lemma: The expected length of a random walk in a binary tree with n nodes is at most log(n+1) • proof by induction: For n=0 is true log(0+1)=0 • Suppose it is true for every n0 < n • n1the size of the left sub-tree and n2 = n - n1 -1 • E[W]= 1 + log(n1+1)/2 + log(n2+1)/2 • E[W]≤ 1 + log( (n-1)/2 + 1) • = 1 + log( (n+ 1)/2 ) • = log(n+ 1)

41. Summary • Lemma: The expected running time of merge(h1,h2) is at most O(log n), where n = h1.size() + h2.size() • Theorem: A MeldableHeap implements the (priority) Queue interface and supports the operations add(x), remove() and merge(h1,h2) in O(log n), expected time per operation.

42. quicksort(S): Recall Quicksort • Pick a random element p from S • Compare every element to p to get 3 sets • S< = {x ϵ S : x < p} • S= = {x ϵ S : x = p} • S> = {x ϵ S : x > p} • quicksort(S<) • quicksort(S>) 5 5 7 6 7 6 9 8 9 8 p=5 0 0 4 1 2 2 1 3 3 4 S= S> S<

43. Quicksort j i 5 3 7 6 9 0 p=5 5 4 2 1 8 i j < 3 7 6 9 5 5 0 4 2 1 8 > i j i = j 3 7 6 9 8 0 4 2 1 8 5 i j 5 3 7 6 9 8 0 4 2 1 3 5 3 7 6 9 8 0 4 2 1 3

44. Quicksort 3 7 6 9 0 5 4 2 1 8 5 7 6 9 8 0 4 2 1 3 5 6 7 9 8 0 4 2 1 3 5 6 7 8 9 0 3 2 1 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4

45. Recall Heapsort • The heapsort algorithm for sorting an array a of length n: • Make a into a heap in (O(n) time) • Repeat n times • Delete the minimum 3 7 6 9 0 5 4 2 1 8 3 7 6 9 8 0 5 2 1 4 5 6 7 8 9 0 1 2 3 4

46. Recall Heapsort 3 7 6 9 0 5 4 2 1 8 3 7 6 9 8 0 1 2 5 4 3 7 6 9 1 4 2 5 8 9 7 6 2 4 3 5 8 9 7 3 4 6 5 8 9 4 5 6 7 8 5 7 6 9 8 6 7 8 9 5 6 0 1 2 3 4

47. Recall Heapsort 9 7 3 4 6 5 8 9 4 5 6 7 8 5 7 6 9 8 6 7 8 9 7 9 8 8 9 9 5 6 7 8 9 0 1 2 3 4

48. Mergesort • The mergesort algorithm for sorting an array a of length n: • Divide a in 2 parts a1 and a2 • Recursively, sort a1 and a2 with mergesort • Merge a1 and a2 3 7 6 9 0 5 4 2 1 8 3 7 6 9 0 5 4 2 1 8 1 2 4 5 8 0 3 6 7 9 5 6 7 8 9 0 1 2 3 4

49. Merging process i0++ i1++ i0 i0 < i1 1 2 4 5 8 i1 < i0 1 2 4 5 8 9 0 3 6 7 0 3 6 7 9 i1

50. Analysis of Mergesort • The mergesort algorithm for sorting an array a of length n: • Divide a in 2 parts a1 and a2 • Recursively, sort a1 and a2 with mergesort • Merge a1 and a2 • The merging process ( step 3 ) costs (O(n) time) • It is performed log n times. • Total costs (O(n log n) time) Theorem: The Mergesort algorithm can sort an array of n items in O(n log n) time