RAIK 283: Data Structures & Algorithms

RAIK 283: Data Structures & Algorithms Decrease and Conquer II Dr. Ying Lu ylu@cse.unl.edu Design and Analysis of Algorithms – Chapter 4

RAIK 283: Data Structures & Algorithms • Giving credit where credit is due: • Most of the lecture notes are based on the slides from the Textbook’s companion website • http://www.aw-bc.com/info/levitin • Some examples and slides are based on lecture notes created by Dr. Ben Choi, Louisiana Technical University and Dr. Chuck Cusack, Hope College • I have modified many of their slides and added new slides. Design and Analysis of Algorithms – Chapter 4

Examples of Decrease and Conquer • Decrease by one: • Insertion sort • Graph search algorithms: • DFS • BFS • Topological sorting • Algorithms for generating permutations, subsets • Decrease by a constant factor • Binary search • Fake-coin problems • multiplication à la russe • Josephus problem • Variable-size decrease • Euclid’s algorithm • Selection by partition • BST Design and Analysis of Algorithms – Chapter 4

Fake coin problem • You have 8 coins and a two-pan balance. All the coins weigh the same, except for one, which is lighter than all the others. The coins are otherwise indistinguishable. You may make no assumption about how much lighter the fake coin is. What is the minimum number of weighings needed to be certain of identifying the fake coin? • How about identifying one light fake coin out of n coins? Design and Analysis of Algorithms – Chapter 4

Examples of Decrease and Conquer • Decrease by one: • Insertion sort • Graph search algorithms: • DFS • BFS • Topological sorting • Algorithms for generating permutations, subsets • Decrease by a constant factor • Binary search • Fake-coin problems • multiplication à la russe (read textbook yourself) • Josephus problem (read textbook yourself) • Variable-size decrease • Euclid’s algorithm • Selection by partition • BST (slides provided for reviewing yourself) Design and Analysis of Algorithms – Chapter 4

The selection problem • Input: A set S of n elements • Output: The kth smallest element of S Design and Analysis of Algorithms – Chapter 4

K = The selection problem • Input: A set S of n elements • Output: The kth smallest element of S • To find the median (the middle value) • To find the smallest element • To find the largest element K = 1 K = n Design and Analysis of Algorithms – Chapter 4

The selection problem • Input: A set S of n elements • Output: The kth smallest element of S • The straightforward algorithm: • step 1: Sort the n elements • step 2: Locate the kth element in the sorted list. • Time complexity: O(nlogn) • This algorithm is an overkill! Design and Analysis of Algorithms – Chapter 4

p A[i]≤p A[i]p Partition problem • Select a pivot (partitioning element), say the left most element • Rearrange the list so that all the elements in the positions before the pivot are smaller than or equal to the pivot and those after the pivot are larger or equal to the pivot Design and Analysis of Algorithms – Chapter 4

p A[i]≤p A[i]p Partition problem • Select a pivot (partitioning element), say the left most element • Rearrange the list so that all the elements in the positions before the pivot are smaller than or equal to the pivot and those after the pivot are larger or equal to the pivot • Design a linear O(n) partition algorithm • Work on problem instance: 7, 8, 10, 2, 9, 6, 5, 4, 11, 13, 0 to help form the idea Design and Analysis of Algorithms – Chapter 4

Partition Example 7, 8, 10, 2, 9, 6, 5, 4, 11, 13, 0

Partition algorithm (I) Design and Analysis of Algorithms – Chapter 4

Partition algorithm (II)  Design and Analysis of Algorithms – Chapter 4

Partition algorithm (II)  A “sentinel” at A[n] to prevent i advances beyond position n. Design and Analysis of Algorithms – Chapter 4

p A[i]≤p A[i]p Partition algorithm • Select a pivot (partitioning element) • Rearrange the list so that all the elements in the positions before the pivot are smaller than or equal to the pivot and those after the pivot are larger or equal to the pivot • How to solve the selection problem based on the partition algorithm? Design and Analysis of Algorithms – Chapter 4

p A[i]≤p A[i]>p Selection by partition • Select a pivot (partitioning element) • Partition the list using the pivot • If pivot is in the kth position, the pivot is the element, done! • If pivot’s position is > k, repeat the above process with the first sublist • If pivot’s position is < k, repeat the above process with the second sublist Design and Analysis of Algorithms – Chapter 4

Example: • Find the median of the following list of nine elements: 4, 1, 10, 9, 7, 12, 8, 2, 15. Design and Analysis of Algorithms – Chapter 4

p A[i]≤p A[i]>p Selection by partition • Select a pivot (partitioning element) • Partition the list using the pivot • If pivot is in the kth position, the pivot is the element, done! • If pivot’s position is > k, repeat the above process with the first sublist • If pivot’s position is < k, repeat the above process with the second sublist • Time efficiency Design and Analysis of Algorithms – Chapter 4

Binary Search Trees (BSTs) Design and Analysis of Algorithms – Chapter 4

Binary Search Trees (BSTs) • Binary Search Tree property • A binary tree in which the key of an internal node is greater than the keys in its left subtree and less than or equal to the keys in its right subtree. • An inorder traversal of a binary search tree produces a sorted list of keys. Design and Analysis of Algorithms – Chapter 4

k <k k Variable-size-decrease: Binary search trees • Keys are arranged in a binary tree with the binary search tree property: Design and Analysis of Algorithms – Chapter 4

BST Examples • Binary Search trees with different degrees of balance • Black dots denote empty trees • Size of a search tree Design and Analysis of Algorithms – Chapter 4

k <k k Variable-size-decrease: Binary search trees • Arrange keys in a binary tree with the binary search tree property: Example 1: 5, 10, 3, 1, 7, 12, 9 Example 2: 4, 5, 7, 2, 1, 3, 6 Design and Analysis of Algorithms – Chapter 4

BST Operations • Find the min/max element () • Search for an element • Find the successor/predecessor of an element • Delete an element Design and Analysis of Algorithms – Chapter 4

BST: Min/Max • The minimum element is the left-most node x is a non-empty BST • The maximum element is the right-most node Design and Analysis of Algorithms – Chapter 4

BST: Min/Max • The minimum element is the left-most node x is a non-empty BST • The maximum element is the right-most node (Time efficiency ) Design and Analysis of Algorithms – Chapter 4

BST Operations • Find the min/max element • Search for an element () • Find the successor/predecessor of an element • Delete an element Design and Analysis of Algorithms – Chapter 4

BST Search (Retrieval) Element bstSearch(BinTree bst, Key K) • Element found • if (bst == nil) • found = null; • else • Element root = root(bst); • if (K == root.key) • found = root; • else if (K < root.key) • found = bstSearch (leftSubtree(bst), K); • else • found = bstSearch(rightSubtree(bst), K); • return found; (Time efficiency ) Design and Analysis of Algorithms – Chapter 4

BST Operations • Find the min/max element • Search for an element • Find the successor/predecessor of an element () • Delete an element Design and Analysis of Algorithms – Chapter 4

BST: Successor/Predecessor • Finding the successor of a node x (if it exists): • If x has a nonempty right subtree, then successor(x) is the smallest element in the tree root at rightSubtree(x) Design and Analysis of Algorithms – Chapter 4

BST: Successor/Predecessor • Finding the successor of a node x (if it exists): • If rightSubtree(x) is empty, then successor(x) is the lowest ancestor of x whose left child is also an ancestor of x. Design and Analysis of Algorithms – Chapter 4

BST: Successor/Predecessor • The predecessor operation is symmetric to successor. Design and Analysis of Algorithms – Chapter 4

Why binary search tree? Array: 1 3 4 5 7 8 9 10 13 17 21 23 BST: Design and Analysis of Algorithms – Chapter 4

BST: advantage • The advantage to the binary search tree approach is that it combines the advantage of an array--the ability to do a binary search with the advantage of a linked list--its dynamic size. Design and Analysis of Algorithms – Chapter 4

BST Operations • Find the min/max element • Search for an element • Find the successor/predecessor of an element • Delete an element () Design and Analysis of Algorithms – Chapter 4

BST: Delete • Deleting a node z is by far the most difficult BST operation. • There are three cases to consider • If z has no children, just delete it. • If z has one child, splice out z, That is, link z’s parent and child • If z has two children, splice out z’s successor y, and replace the contents of z with the contents of y • The last case works because if z has two children, then its successor has no left child Design and Analysis of Algorithms – Chapter 4

BST: splice out examples Design and Analysis of Algorithms – Chapter 4

BST: splice out algorithm Only works when a node has at most one child Design and Analysis of Algorithms – Chapter 4

BST: Deletion Algorithm • Delete is now simple! Design and Analysis of Algorithms – Chapter 4

BST: one more delete example Design and Analysis of Algorithms – Chapter 4

Analysis of BST Operations • All of the BST operations have time complexity (h), where h is the height of the tree • However, in the worst-case, the height may be (n) where n is the number of internal nodes • For example, a long chain tree structure • For a tree as balanced as possible, (log2n) • The objective is to make the tree as balanced as possible • Technique: Binary Tree Rotations • Balanced search tree: AVL tree and 2-3 tree Design and Analysis of Algorithms – Chapter 4

RAIK 283: Data Structures & Algorithms