Data Structures Lecture 11

1. Fall 2010 Data StructuresLecture 11 Fang Yu Department of Management Information Systems National Chengchi University

2. A Review of Dynamic Programming • Main concepts: • The global optimum value can be defined in terms of optimal subproblems • The subproblems are not independent, but instead they overlap (hence, should be constructed bottom-up). • LCS problem: • Given two strings X and Y, the longest common subsequence (LCS) problem is to find a longest subsequence common to both X and Y

3. LCS • Define L[i,j] to be the longest common subsequence of X[0..i] and Y[0..j]. • Allow for -1 as an index, so L[-1,k] = “” and L[k,-1]=“”, to indicate that the null part of X or Y has no match with the other. • Then we can define L[i,j] in the general case as follows: • If xi=yj, then L[i,j] = L[i-1,j-1] + xi (we can add this match) • If xi≠yj, then L[i,j] = max{L[i-1,j], L[i,j-1]} (we have no match here)

4. Search Trees Binary Search Trees, AVL trees, and Splay Trees

5. A binary search tree is a binary tree storing keys (or key-value entries) at its internal nodes and satisfying the following property: Let u, v, and w be three nodes such that u is in the left subtree of v and w is in the right subtree of v. We have key(u)key(v) key(w) External nodes do not store items An inorder traversal of a binary search trees visits the keys in increasing order 6 2 9 1 4 8 Binary Search Trees

6. Search AlgorithmTreeSearch(k, v) ifT.isExternal (v) returnv if k<key(v) returnTreeSearch(k, T.left(v)) else if k=key(v) returnv else{ k>key(v) } returnTreeSearch(k, T.right(v)) • To search for a key k, we trace a downward path starting at the root • The next node visited depends on the comparison of k with the key of the current node • If we reach a leaf, the key is not found • Example: get(4): • Call TreeSearch(4,root) • The algorithms for floorEntry and ceilingEntryare similar 6 < 2 9 > = 8 1 4

7. Insertion 6 < 2 9 > • To perform operation put(k, o), we search for key k (using TreeSearch) • Assume k is not already in the tree, and let w be the leaf reached by the search • We insert k at node w and expand w into an internal node • Example: insert 5 1 4 8 > w 6 2 9 1 4 8 w 5

8. Deletion 6 < 2 9 > • To perform operation remove(k), we search for key k • Assume key k is in the tree, and let let v be the node storing k • If node v has a leaf child w, we remove v and w from the tree with operation removeExternal(w), which removes w and its parent • Example: remove 4 v 1 4 8 w 5 6 2 9 1 5 8

9. Deletion (cont.) 1 v 3 • We consider the case where the key k to be removed is stored at a node v whose children are both internal • we find the internal node w that follows v in an inorder traversal • we copy key(w) into node v • we remove node w and its left child z (which must be a leaf) by means of operation removeExternal(z) • Example: remove 3 2 8 6 9 w 5 z 1 v 5 2 8 6 9

10. Performance • Considernordered set items implemented by means of a binary search tree of height h • the space used is O(n) • methods get,put and remove take O(h) time • The height h is O(n) in the worst case and O(logn) in the best case We want a balanced binary tree!

11. AVL Tree Definition • AVL trees are balanced • An AVL Tree is a binary search tree such that for every internal node v of T, the heights of the children of v can differ by at most 1 An example of an AVL tree where the heights are shown next to the nodes:

12. n(2) 3 n(1) 4 Height of an AVL Tree • Fact: The height of an AVL tree storing n keys is O(log n). • Proof: Let us bound n(h): the minimum number of internal nodes of an AVL tree of height h. • We easily see that n(1) = 1 and n(2) = 2 • For n > 2, an AVL tree of height h contains the root node, one AVL subtree of height n-1 and another of height n-2. • That is, n(h) = 1 + n(h-1) + n(h-2) • Knowing n(h-1) > n(h-2), we get n(h) > 2n(h-2). So n(h) > 2n(h-2), n(h) > 4n(h-4), n(h) > 8n(n-6), … (by induction), n(h) > 2in(h-2i) • Solving the base case we get: n(h) > 2 h/2-1 • Taking logarithms: h < 2log n(h) +2 • Thus the height of an AVL tree is O(log n) AVL Trees

13. 44 17 78 44 32 50 88 17 78 48 62 32 50 88 54 48 62 Insertion • Insertion is as in a binary search tree • Always done by expanding an external node. • Example: c=z a=y b=x w before insertion after insertion

14. 44 17 78 44 32 50 88 17 78 48 62 32 50 88 54 48 62 After Insertion 5 4 4 3 3 2 2 1 1 All nodes along the path increase their height by 1 It may violate the AVL property

15. Search and repair Let z be the first violation node from the bottom along the path Let y be z’child with the higher height (y is 2 greater than its sibling) Let x be y’s child with the higher height We rebalance z by calling trinode restructuring method

16. a=z a=z T0 c=y T0 b=y b=x T3 c=x T1 b=y b=x T1 T2 T2 T3 a=z c=x a=z c=y T2 T3 T1 T3 T0 T0 T2 T1 Trinode Restructuring • let (a,b,c) be an inorder listing of x, y, z • perform the rotations needed to make b the topmost node of the three (other two cases are symmetrical) case 2: double rotation (a right rotation about c, then a left rotation about a) case 1: single rotation (a left rotation about a) AVL Trees

17. Restructuring (as Single Rotations) • Single Rotations: AVL Trees

18. Restructuring (as Double Rotations) • double rotations: AVL Trees

19. T T 1 1 Insertion Example, continued unbalanced... 4 44 x 3 2 17 62 z y 2 1 2 78 32 50 1 1 1 ...balanced 54 88 48 T 2 T T 0 3 AVL Trees

20. 44 17 62 32 50 78 88 48 54 Removal • Removal begins as in a binary search tree, which means the node removed will become an empty external node. Its parent, w, may cause an unbalance. • Example: 44 17 62 50 78 88 48 54 before deletion of 32 after deletion AVL Trees

21. Rebalancing after a Removal • Let z be the first unbalanced node encountered while travelling up the tree from w. Also, let y be the child of z with the larger height, and let x be the child of y with the larger height • As this restructuring may upset the balance of another node higher in the tree, we must continue checking for balance until the root of T is reached 62 a=z 44 44 78 w b=y 17 62 17 50 88 c=x 50 78 48 54 88 48 54 AVL Trees

22. AVL Tree Performance • a single restructure takes O(1) time • using a linked-structure binary tree • get takes O(log n) time • height of tree is O(log n), no restructures needed • put takes O(log n) time • initial find is O(log n) • Restructuring up the tree, maintaining heights is O(log n) • remove takes O(log n) time • initial find is O(log n) • Restructuring up the tree, maintaining heights is O(log n)

23. Splay Tree • a splay tree is a binary search tree where a node is splayed after it is accessed (for a search or update) • deepest internal node accessed is splayed • splay: move the node to the root • splaying costs O(h), where h is height of the tree – which is still O(n) worst-case • O(h) rotations, each of which is O(1)

24. Splay Tree • which nodes are splayed after each operation? method splay node if key found, use that node if key not found, use parent of ending external node get(k) put(k,v) use the new node containing the entry inserted use the parent of the internal node that was actually removed from the tree (the parent of the node that the removed item was swapped with) remove(k)

25. (20,Z) (10,A) (35,R) (14,J) (7,T) (21,O) (37,P) (1,Q) (8,N) (36,L) (40,X) (1,C) (10,U) (5,H) (7,P) (2,R) (5,G) (6,Y) (5,I) Searching in a Splay Tree: Starts the Same as in a BST • Search proceeds down the tree to found item or an external node. • Example: Search for time with key 11. Splay Trees

26. (20,Z) (10,A) (35,R) (14,J) (7,T) (21,O) (37,P) (1,Q) (8,N) (36,L) (40,X) (1,C) (10,U) (5,H) (7,P) (2,R) (5,G) (6,Y) (5,I) Example Searching in a BST, continued • search for key 8, ends at an internal node. Splay Trees

27. Splay Trees do Rotations after Every Operation (Even Search) • right rotation • makes the left child x of a node y into y’s parent; y becomes the right child of x • left rotation • makes the right child y of a node x into x’s parent; x becomes the left child of y • new operation: splay • splaying moves a node to the root using rotations y x a right rotation about y a left rotation about x y T1 x T3 x y T3 T1 T2 T2 y T1 x T3 T3 T2 T1 T2 (structure of tree above x is not modified) (structure of tree above y is not modified) Splay Trees

28. “x is aleft-left grandchild” means x is a left child of its parent, which is itself a left child of its parent • p is x’s parent; g is p’s parent start with node x Splaying: is x a left-left grandchild? is x the root? zig-zig yes stop right-rotate about g, right-rotate about p yes no is x a right-right grandchild? zig-zig is x a child of the root? no left-rotate about g, left-rotate about p yes is x a right-left grandchild? yes zig-zag is x the left child of the root? left-rotate about p, right-rotate about g no yes is x a left-right grandchild? zig zig zig-zag yes right-rotate about p, left-rotate about g right-rotate about the root left-rotate about the root yes Splay Trees

29. Visualizing the Splaying Cases zig-zag x z z z y y T4 T1 y T1 T2 T3 T4 T4 x T3 x T2 T3 T1 T2 zig-zig y zig x T4 x T1 x y w T2 y z T3 w T1 T2 T3 T4 T3 T4 T1 T2 Splay Trees

30. (20,Z) (20,Z) (10,A) (35,R) (35,R) (8,N) x g (14,J) (7,T) (10,A) (21,O) (37,P) p (21,O) (37,P) (20,Z) (10,A) g (36,L) (40,X) (1,Q) (8,N) (36,L) (40,X) (35,R) (8,N) x (14,J) (14,J) (7,T) (7,T) (1,C) (21,O) (37,P) (10,U) (5,H) p (7,P) (1,Q) (1,Q) (36,L) (40,X) (2,R) (5,G) (1,C) (1,C) (10,U) (10,U) (5,H) (5,H) (7,P) (7,P) (6,Y) (5,I) (2,R) (2,R) (5,G) (5,G) (6,Y) (6,Y) (5,I) (5,I) g Splaying Example 1. (before rotating) p x • let x = (8,N) • x is the right child of its parent, which is the left child of the grandparent • left-rotate around p, then right-rotate around g 2. (after first rotation) 3. (after second rotation) x is not yet the root, so we splay again Splay Trees

31. (20,Z) (35,R) (8,N) x (8,N) x (10,A) (21,O) (37,P) (20,Z) (36,L) (40,X) (35,R) (10,A) (21,O) (37,P) (14,J) (14,J) (7,T) (7,T) (36,L) (40,X) (1,Q) (1,Q) (1,C) (1,C) (10,U) (10,U) (5,H) (5,H) (7,P) (7,P) (2,R) (2,R) (5,G) (5,G) (6,Y) (6,Y) (5,I) (5,I) Splaying Example, Continued • now x is the left child of the root • right-rotate around root 2. (after rotation) 1. (before applying rotation) x is the root, so stop Splay Trees

32. (20,Z) (20,Z) (40,X) (10,A) (35,R) (10,A) (14,J) (37,P) (7,T) (14,J) (7,T) (21,O) (37,P) (35,R) (1,Q) (8,N) (40,X) (20,Z) (1,Q) (8,N) (36,L) (40,X) (21,O) (36,L) (1,C) (10,U) (5,H) (7,P) (10,A) (37,P) (2,R) (5,G) (1,C) (10,U) (5,H) (7,P) (14,J) (35,R) (7,T) (6,Y) (5,I) (1,Q) (8,N) (21,O) (36,L) (2,R) (5,G) (1,C) (10,U) (5,H) (7,P) (6,Y) (5,I) (2,R) (5,G) (6,Y) (5,I) before Example Result of Splaying • tree might not be more balanced • e.g. splay (40,X) • before, the depth of the shallowest leaf is 3 and the deepest is 7 • after, the depth of shallowest leaf is 1 and deepest is 8 after first splay after second splay Splay Trees

33. Performance of Splay Trees • Amortized cost of any splay operation is O(logn) • This implies that splay trees can actually adapt to perform searches on frequently-requested items much faster than O(logn) in some cases Splay Trees

34. Project Hints How to call google? How to find the reference links? How to encode Chinese?

35. HW11 (Due on Dec 1) Use Google and get the links! Get a keyword from user Return the urls listed in the search result After this HW, you can step to the third stage of the project You can apply the same technique to other search engines