Prelim 2 Review

1. Prelim 2 Review CS 2110 Fall 2009

2. Overview • Complexity and Big-O notation • ADTs and data structures • Linked lists, arrays, queues, stacks, priority queues, hash maps • Searching and sorting • Graphs and algorithms • Searching • Topological sort • Minimum spanning trees • Shortest paths • Miscellaneous Java topics • Dynamic vs static types, casting, garbage collectors, Java docs

3. Big-O notation • Big-O is an asymptotic upper bound on a function • “f(x) is O(g(x))” Meaning: There exists some constant k such that f(x) ≤ k g(x) …as x goes to infinity • Often used to describe upper bounds for both worst-case and average-case algorithm runtimes • Runtime is a function: The number of operations performed, usually as a function of input size

4. Big-O notation • For the prelim, you should know… • Worst case Big-O complexity for the algorithms we’ve covered and for common implementations of ADT operations • Examples • Mergesort is worst-case O(n log n) • PriorityQueue insert using a heap is O(log n) • Average case time complexity for some algorithms and ADT operations, if it has been noted in class • Examples • Quicksort is average case O(n log n) • HashMap insert is average case O(1)

5. Big-O notation • For the prelim, you should know… • How to estimate the Big-O worst case runtimes of basic algorithms (written in Java or pseudocode) • Count the operations • Loops tend to multiply the loop body operations by the loop counter • Trees and divide-and-conquer algorithms tend to introduce log(n) as a factor in the complexity • Basic recursive algorithms, i.e., binary search or mergesort

6. Abstract Data Types • What do we mean by “abstract”? • Defined in terms of operations that can be performed, not as a concrete structure • Example: Priority Queue is an ADT, Heap is a concrete data structure • For ADTs, we should know: • Operations offered, and when to use them • Big-O complexity of these operations for standard implementations

7. ADTs:The Bag Interface interface Bag<E> { void insert(E obj); E extract(); //extract some element booleanisEmpty(); E peek(); // optional: return next element without removing } Examples: Queue, Stack, PriorityQueue

8. Queues • First-In-First-Out (FIFO) • Objects come out of a queue in the same order they were inserted • Linked List implementation • insert(obj): O(1) • Add object to tail of list • Also called enqueue, add (Java) • extract(): O(1) • Remove object from head of list • Also called dequeue, poll (Java)

9. Stacks • Last-In-First-Out (LIFO) • Objects come out of a queue in the opposite order they were inserted • Linked List implementation • insert(obj): O(1) • Add object to tail of list • Also called push(Java) • extract(): O(1) • Remove object from head of list • Also called pop(Java)

10. Priority Queues • Objects come out of a Priority Queue according to their priority • Generalized • By using different priorities, can implement Stacks or Queues • Heap implementation (as seen in lecture) • insert(obj, priority): O(log n) • insert object into heap with given priority • Also called add (Java) • extract(): O(log n) • Remove and return top of heap (minimum priority element) • Also called poll (Java)

11. Heaps • Concrete Data Structure • Balanced binary tree • Obeys heap order invariant: Priority(child) ≥ Priority(parent) • Operations • insert(value, priority) • extract()

12. Heap insert() • Put the new element at the end of the array • If this violates heap order because it is smaller than its parent, swap it with its parent • Continue swapping it up until it finds its rightful place • The heap invariant is maintained!

13. 21 22 Heap insert() 4 6 14 8 19 35 38 55 10 20

14. 21 22 Heap insert() 4 6 14 8 19 35 38 55 10 20 5

15. 21 22 Heap insert() 4 6 14 8 35 5 19 38 55 10 20

16. 21 22 Heap insert() 4 6 5 8 35 14 19 38 55 10 20

17. 21 22 Heap insert() 4 6 5 8 35 14 19 38 55 10 20

18. insert() • Time is O(log n), since the tree is balanced • size of tree is exponential as a function of depth • depth of tree is logarithmic as a function of size

19. extract() • Remove the least element – it is at the root • This leaves a hole at the root – fill it in with the last element of the array • If this violates heap order because the root element is too big, swap it down with the smaller of its children • Continue swapping it down until it finds its rightful place • The heap invariant is maintained!

20. 21 22 extract() 4 6 5 8 14 35 19 38 55 10 20

21. 21 22 extract() 4 6 5 8 14 35 19 38 55 10 20

22. 21 22 extract() 4 6 5 8 14 35 19 38 55 10 20

23. 21 22 extract() 4 19 6 5 8 14 35 38 55 10 20

24. 21 22 extract() 5 4 6 19 8 14 35 38 55 10 20

25. 21 22 extract() 5 4 6 14 8 35 19 38 55 10 20

26. 21 22 extract() 5 4 6 14 8 35 19 38 55 10 20

27. 21 22 extract() 4 5 6 14 8 35 19 38 55 10 20

28. 21 22 extract() 4 5 6 14 8 35 19 38 55 10 20

29. 21 22 extract() 20 4 5 6 14 8 35 19 38 55 10

30. 21 22 extract() 6 4 5 14 20 8 35 19 38 55 10

31. 21 22 extract() 6 4 5 8 14 35 20 19 38 55 10

32. 21 22 extract() 6 4 5 8 14 35 10 19 38 55 20

33. 21 22 extract() 6 4 5 8 14 35 10 19 38 55 20

34. extract() • Time is O(log n), since the tree is balanced

35. Store in an ArrayList or Vector • Elements of the heap are stored in the array in order, going across each level from left to right, top to bottom • The children of the node at array index n are found at 2n + 1 and 2n + 2 • The parent of node n is found at (n – 1)/2

36. Sets A set makes no promises about ordering, but you can still iterate over it. • ADT Set • Operations: • void insert(Object element); • boolean contains(Object element); • void remove(Object element); • int size(); • iteration • No duplicates allowed • Hash table implementation: O(1) insert and contains • SortedSet tree implementation: O(log n) insert and contains

37. Dictionaries A HashMap is a particular implementation of the Map interface • ADT Dictionary (aka Map) • Operations: • void insert(Object key, Object value); • void update(Object key, Object value); • Object find(Object key); • void remove(Object key); • booleanisEmpty(); • void clear(); • Think of: key = word; value = definition • Where used: • Symbol tables • Wide use within other algorithms

38. Dictionaries A HashMap is a particular implementation of the Map interface • Hash table implementation: • Use a hash function to compute hashes of keys • Store values in an array, indexed by key hash • A collision occurs when two keys have the same hash • How to handle collisions? • Store another data structure, such as a linked list, in the array location for each key • Put (key, value) pairs into that data structure • insert and find are O(1) when there are no collisions • Expected complexity • Worst case, every hash is a collision • Complexity for insert and find comes from the tertiary data structure’s complexity, e.g., O(n) for a linked list

39. For the prelim… • Don’t spend your time memorizing Java APIs! • If you want to use an ADT, it’s acceptable to write code that looks reasonable, even if it’s not the exact Java API. For example, Queue<Integer> myQueue = new Queue<Integer>(); myQueue.enqueue(5); … int x = myQueue.dequeue(); • This is not correct Java (Queue is an interface! And Java calls enqueueand dequeue“add” and “poll”) • But it’s fine for the exam.

40. Searching • Find a specific element in a data structure • Linear search, O(n) • Linked lists • Unsorted arrays • Binary search, O(log n) • Sorted arrays • Binary Search Tree search • O(log n) if the tree is balanced • O(n) worst case

41. Sorting • Comparison sorting algorithms • Sort based on a ≤ relation on objects • Examples • QuickSort • HeapSort • MergeSort • InsertionSort • BubbleSort

42. QuickSort • Given an array A to sort, choose a pivot value p • Partition A into two sub-arrays, AX and AY • AX contains only elements ≤ p • AY contains only elements ≥ p • Recursively sort sub-arrays separately • Return AX + p + AY • O(n log n) average case runtime • O(n2) worst case runtime • But in the average case, very fast! So people often prefer QuickSort over other sorts.

43. QuickSort QuickSort 20 31 24 19 45 56 4 65 5 72 14 99 concatenate partition pivot 20 4 5 14 19 24 31 45 56 65 72 99 4 5 14 19 20 24 31 45 56 65 72 99 31 5 19 72 14 65 45 99 56 4 24

44. QuickSort Questions • Choosing a pivot • Ideal pivot is the median, since this splits array in half • Computing the median of an unsorted array is O(n), but algorithm is quite complicated • Popular heuristics: • Use first value in array (usually not a good choice) • Use middle value in array • Use median of first, last, and middle values in array • Choose a random element • Key problems • How should we choose a pivot? • How do we partition an array in place? • Partitioning in place • Can be done in O(n)

45. Heap Sort • Insert all elements into a heap • Extract all elements from the heap • Each extraction returns the next largest element; they come out in sorted order! • Heap insert and extract are O(log n) • Repeated for n elements, we have O(n log n) worst-case runtime

46. Graphs • Set of vertices (or nodes) V, set of edges E • Number of vertices n = |V| • Number of edges m = |E| • Upper bound O(n2) on number of edges • A complete graph has m = n(n-1)/2 • Directed or undirected • Directed edges have distinct head and tail • Weighted edges • Cycles and paths • Connected components • DAGs • Degree of a node (in- and out- degree for directed graphs)

47. Graph Representation • You should be able to write a Vertex class in Java and implement standard graph algorithms using this class • However, understanding the algorithms is much more important than memorizing their code

48. Topological Sort • A topological sort is a partial order on the vertices of a DAG • Definition: If vertex A comes before B in topological sort order, then there is no path from B to A • A DAG can have many topological sort orderings; they aren’t necessarily unique (except, say, for a singly linked list) • No topological sort possible for a graph that contains cycles • One algorithm for topological sorting: • Iteratively remove nodes with no incoming edges until all nodes removed (see next slide) • Can be used to find cycles in a directed graph • If no remaining node has no incoming edges, there must be a cycle

49. Topological Sorting B A B C E D A E C D

50. Graph Searching • Works on directed and undirected graphs • You have a start vertex which you visit first • You want to visit all vertices reachable from the start vertex • For directed graphs, depending on your start vertex, some vertices may not be reachable • You can traverse an edge from an already visited vertex to another vertex