Data S tructures and Algorithms

1. Data StructuresandAlgorithms Course’s slides: Hierarchical data structures www.mif.vu.lt/~algis

2. Trees • Linear access time of linked lists is prohibitive • Does there exist any simple data structure for which the running time of most operations (search, insert, delete) is O(log N)?

3. Trees • A tree is a collection of nodes • The collection can be empty • (recursive definition) If not empty, a tree consists of a distinguished node r (the root), and zero or more nonempty subtrees T1, T2, ...., Tk, each of whose roots are connected by a directed edge from r

4. Some Terminologies • Child and parent • Every node except the root has one parent  • A node can have an arbitrary number of children • Leaves • Nodes with no children • Sibling • nodes with same parent

5. Some Terminologies • Path • Length • number of edges on the path • Depth of a node • length of the unique path from the root to that node • The depth of a tree is equal to the depth of the deepest leaf • Height of a node • length of the longest path from that node to a leaf • all leaves are at height 0 • The height of a tree is equal to the height of the root • Ancestor and descendant • Proper ancestor and proper descendant

6. Example: UNIX Directory

7. Binary Trees • A tree in which no node can have more than two children • The depth of an “average” binary tree is considerably smaller than N, eventhough in the worst case, the depth can be as large as N – 1.

8. Example: Expression Trees • Leaves are operands (constants or variables) • The other nodes (internal nodes) contain operators • Will not be a binary tree if some operators are not binary

9. Tree traversal • Used to print out the data in a tree in a certain order • Pre-order traversal • Print the data at the root • Recursively print out all data in the left subtree • Recursively print out all data in the right subtree

10. Preorder, Postorder and Inorder • Preorder traversal • node, left, right • prefix expression • ++a*bc*+*defg

11. Preorder, Postorder and Inorder • Postorder traversal • left, right, node • postfix expression • abc*+de*f+g*+ • Inorder traversal • left, node, right. • infix expression • a+b*c+d*e+f*g

12. Preorder, Postorder and Inorder

13. Binary Trees Possible operations on the Binary Tree ADT parent left_child, right_child sibling root, etc Implementation Because a binary tree has at most two children, we can keep direct pointers to them

14. Compare: Implementation of a general tree

15. Binary Search Trees Stores keys in the nodes in a way so that searching, insertion and deletion can be done efficiently. Binary search tree property • For every node X, all the keys in its left subtree are smaller than the key value in X, and all the keys in its right subtree are larger than the key value in X

16. Binary Search Trees A binary search tree Not a binary search tree

17. Binary search trees Two binary search trees representing the same set: • Average depth of a node is O(log N); maximum depth of a node is O(N)

18. Searching BST • If we are searching for 15, then we are done. • If we are searching for a key < 15, then we should search in the left subtree. • If we are searching for a key > 15, then we should search in the right subtree.

19. Inorder traversal of BST • Print out all the keys in sorted order Inorder: 2, 3, 4, 6, 7, 9, 13, 15, 17, 18, 20

20. findMin/findMax Return the node containing the smallest element in the tree Start at the root and go left as long as there is a left child. The stopping point is the smallest element Similarly for findMax Time complexity = O(height of the tree)

21. Insert Proceed down the tree as you would with a find If X is found, do nothing (or update something) Otherwise, insert X at the last spot on the path traversed Time complexity = O(height of the tree)

22. Delete When we delete a node, we need to consider how we take care of the children of the deleted node. This has to be done such that the property of the search tree is maintained.

23. Delete Three cases: (1) the node is a leaf • Delete it immediately (2) the node has one child • Adjust a pointer from the parent to bypass that node

24. Delete (3) the node has 2 children • replace the key of that node with the minimum element at the right subtree • delete the minimum element • Has either no child or only right child because if it has a left child, that left child would be smaller and would have been chosen. So invoke case 1 or 2 Time complexity = O(height of the tree)

25. Binary search tree – best time • All BST operations are O(d), where d is tree depth • minimum d is for a binary tree with N nodes • What is the best case tree? • What is the worst case tree? • So, best case running time of BST operations is O(log N) AVL Trees - Lecture 8

26. Binary Search Tree - Worst Time • Worst case running time is O(N) • What happens when you insert elements in ascending order? • Insert: 2, 4, 6, 8, 10, 12 into an empty BST • Problem: Lack of “balance”: • compare depths of left and right subtree • Unbalanced degenerate tree AVL Trees - Lecture 8

27. Balanced and unbalanced BST 1 4 2 2 5 3 1 3 4 4 Is this “balanced”? 5 2 6 6 1 3 5 7 7 AVL Trees - Lecture 8

28. Approaches to balancing trees • Don't balance • May end up with some nodes very deep • Strict balance • The tree must always be balanced perfectly • Pretty good balance • Only allow a little out of balance • Adjust on access • Self-adjusting AVL Trees - Lecture 8

29. Balancing binary search trees • Many algorithms exist for keeping binary search trees balanced • Adelson-Velskii and Landis (AVL) trees (height-balanced trees) • Splay trees and other self-adjusting trees • B-trees and other multiway search trees AVL Trees - Lecture 8

30. Perfect balance • Want a complete tree after every operation • tree is full except possibly in the lower right • This is expensive • For example, insert 2 in the tree on the left and then rebuild as a complete tree 6 5 Insert 2 & complete tree 4 9 2 8 1 5 8 1 4 6 9 AVL Trees - Lecture 8

31. AVL - good but not perfect balance • AVL trees are height-balanced binary search trees • Balance factor of a node • height(left subtree) - height(right subtree) • An AVL tree has balance factor calculated at every node • For every node, heights of left and right subtree can differ by no more than 1 • Store current heights in each node AVL Trees - Lecture 8

32. Height of an AVL tree • N(h) = minimum number of nodes in an AVL tree of height h. • Basis • N(0) = 1, N(1) = 2 • Induction • N(h) = N(h-1) + N(h-2) + 1 • Solution (recall Fibonacci analysis) • N(h) >h (  1.62) h h-2 h-1 AVL Trees - Lecture 8

33. Height of an AVL Tree • N(h) >h (  1.62) • Suppose we have n nodes in an AVL tree of height h. • n >N(h) (because N(h) was the minimum) • n >h hence log n > h (relatively well balanced tree!!) • h < 1.44 log2n (i.e., Find takes O(logn)) AVL Trees - Lecture 8

34. Node Heights Tree A (AVL) Tree B (AVL) height=2 BF=1-0=1 2 6 6 1 0 1 1 4 9 4 9 0 0 0 0 0 1 5 1 5 8 height of node = h balance factor = hleft-hright empty height = -1 AVL Trees - Lecture 8

35. Node heights after insert 7 Tree A (AVL) Tree B (not AVL) balance factor 1-(-1) = 2 2 3 6 6 1 1 1 2 4 9 4 9 -1 0 0 0 0 0 1 7 1 5 1 5 8 0 7 height of node = h balance factor = hleft-hright empty height = -1 AVL Trees - Lecture 8

36. Insert and rotation in AVL trees • Insert operation may cause balance factor to become 2 or –2 for some node • only nodes on the path from insertion point to root node have possibly changed in height • So after the Insert, go back up to the root node by node, updating heights • If a new balance factor (the difference hleft-hright) is 2 or –2, adjust tree by rotation around the node AVL Trees - Lecture 8

37. Single Rotation in an AVL Tree 2 2 6 6 1 2 1 1 4 9 4 8 0 0 0 0 1 0 0 7 9 1 5 8 1 5 0 7 AVL Trees - Lecture 8

38. Insertions in AVL trees Let the node that needs rebalancing be . There are 4 cases: Outside Cases (require single rotation) : 1. Insertion into left subtree of left child of . 2. Insertion into right subtree of right child of . Inside Cases (require double rotation) : 3. Insertion into right subtree of left child of . 4. Insertion into left subtree of right child of . The rebalancing is performed through four separate rotation algorithms. AVL Trees - Lecture 8

39. AVL insertion: outside case j Consider a valid AVL subtree k h Z h h X Y AVL Trees - Lecture 8

40. AVL Insertion: Outside Case j Inserting into X destroys the AVL property at node j k h Z h+1 h Y X AVL Trees - Lecture 8

41. AVL Insertion: Outside Case j Do a “right rotation” k h Z h+1 h Y X AVL Trees - Lecture 8

42. Single right rotation j Do a “right rotation” k h Z h+1 h Y X AVL Trees - Lecture 8

43. Outside Case Completed “Right rotation” done! (“Left rotation” is mirror symmetric) k j h+1 h h X Z Y AVL property has been restored! AVL Trees - Lecture 8

44. AVL Insertion: Inside Case j Consider a valid AVL subtree k h Z h h X Y AVL Trees - Lecture 8

45. AVL Insertion: Inside Case j Inserting into Y destroys the AVL property at node j Does “right rotation” restore balance? k h Z h h+1 X Y AVL Trees - Lecture 8

46. AVL Insertion: Inside Case k “Right rotation” does not restore balance… now k is out of balance j h X h h+1 Z Y AVL Trees - Lecture 8

47. AVL Insertion: Inside Case j Consider the structure of subtree Y… k h Z h h+1 X Y AVL Trees - Lecture 8

48. AVL Insertion: Inside Case j Y = node i and subtrees V and W k h Z i h h+1 X h or h-1 W V AVL Trees - Lecture 8

49. AVL Insertion: Inside Case j We will do a left-right “double rotation” . . . k Z i X W V AVL Trees - Lecture 8

50. Double rotation : first rotation j left rotation complete i Z k W V X AVL Trees - Lecture 8