Balanced Trees (AVL and RedBlack)

1. Balanced Trees (AVL and RedBlack)

2. Binary Search Trees • Optimal Behavior • O(log2N) – perfectly balanced tree (e.g. complete tree with all levels filled) • Degenerate Behavior • O(N) – tree becomes list • How to avoid degenerate trees? • Keep them perfectly balanced • Impossible • Keep them balanced within some criteria • Maintain balance over insertions and deletions

3. Balanced Trees • Balanced binary trees • AVL Tree • Red-Black Tree • Balanced N-ary trees • B-Trees • B+-Trees

4. Height of BST • A BST is not balanced • E.g., 5 12 23 35 65 85

5. AVL Trees Discovered by two Russian mathematicians, Adelson-Velskii and Landis, in 1962. Mathematical Definition: If T is a a nonempty binary search tree, then T is an AVL tree if and only if its left and right subtrees are AVL trees and |left height – right height| < 1. “In English” Definition: A binary search tree that is somewhat balanced, but not necessarily complete or full.

6. 45 10 20 70 7 15 10 30 60 3 9 13 30 50 11 AVL Trees?

7. AVL Trees Representation: Use the linked representation, with an addition to the node class. The addition is a balance factor (bf), which is defined as the following for a node x. bf(x) = height of x’s left subtree – height of x’s right subtree The possible balance factors are –1, 0, and 1.

8. AVL Trees Insertion: Using the standard insertion logic for a binary search tree could cause problems with an AVL tree because it’s possible that some of the properties could be violated. In other words, it’s possible that the tree could become unbalanced.

9. JFK +1 DCA -1 ORD 0 JFK +2 GCM 0 DCA -2 ORD 0 GCM +1 DUS 0 AVL Trees Adding the node “DUS” results in the tree below:

10. AVL Trees • When an insertion is performed, the balancing information for ALL of the nodes on the path back to the root must be updated. • If the AVL properties were destroyed, they can be fixed by a simple modification to the tree, which is known as a rotation.

11. AVL Trees To fix the tree, start at the inserted node and trace its path back to the root, stopping at the first node that violates the AVL property (call this node A). There are four possible violations that could occur: 1. Inserted into the left subtree of A’s left child 2. Inserted into the right subtree of A’s right child 3. Inserted into the right subtree of A’s left child 4. Inserted into the left subtree of A’s right child

12. A 10 B 7 B 7 10 A 3 3 AVL Trees 1. Inserted into the left subtree of A’s left child (lets call this B). This problem is fixed by applying a single right rotation. - Change the parent of A to point to B - Make A’s left link equal to B’s right link - Make B’s right link point to A

13. A 45 B B 55 55 A 65 45 65 AVL Trees 2. Inserted into the right subtree of A’s right child (lets call this B). This problem is fixed by applying a single left rotation. - Change the parent of A to point to B - Make A’s right link equal to B’s left link - Make B’s left link point to A

14. 3 2 2 2 1 3 1 1 3 4 AVL Trees Let’s build a tree using the nodes 3, 2, 1, 4, 5, 6, 7.

15. 2 2 1 4 1 3 3 5 4 5 AVL Trees Let’s build a tree using the nodes 3, 2, 1, 4, 5, 6, 7.

16. After step 1 of a single left rotation 2 2 4 1 4 1 3 5 3 5 6 6 AVL Trees Let’s build a tree using the nodes 3, 2, 1, 4, 5, 6, 7.

17. After step 2 of a single left rotation 2 2 4 1 1 3 5 4 6 5 3 6 AVL Trees Let’s build a tree using the nodes 3, 2, 1, 4, 5, 6, 7.

18. After step 3 of a single left rotation 2 1 2 4 4 1 3 5 5 3 6 6 AVL Trees Let’s build a tree using the nodes 3, 2, 1, 4, 5, 6, 7.

19. 4 4 5 2 2 6 6 1 1 5 3 3 7 7 AVL Trees Let’s build a tree using the nodes 3, 2, 1, 4, 5, 6, 7.

20. A A 15 15 B C C C B B 12 7 12 7 15 A 7 12 AVL Trees 3. Inserted into the right subtree of A’s left child (lets call this B). This problem is fixed by applying a left-right rotation. - Make A’s left link equal to B’s right link (call this C) - Make B’s right link equal to C’s left link - Make C’s left link equal to B - Change the parent of A to point to C - Make A’s left link equal to C’s right link - Make C’s right link point to A

21. A A 15 15 21 C C B C A B 32 21 15 32 B 21 32 AVL Trees 4. Inserted into the left subtree of A’s right child (lets call this B). This problem is fixed by applying a right-left rotation. - Make A’s right link equal to B’s left link (call this C) - Make B’s left link equal to C’s right link - Make C’s right link equal to B - Change the parent of A to point to C - Make A’s right link equal to C’s left link - Make C’s left link point to A

22. 4 4 6 6 2 2 5 7 5 7 1 1 3 3 16 17 17 16 AVL Trees Let’s build onto the previous example by adding the nodes: 17, 16, 15, 13, 14. Right-left rotation

23. 4 4 6 6 2 2 5 5 7 7 1 1 3 3 16 16 17 17 AVL Trees Let’s build onto the previous example by adding the nodes: 17, 16, 15, 13, 14. Right-left rotation

24. 4 4 2 6 6 2 1 3 5 5 7 16 1 3 16 17 7 17 AVL Trees Let’s build onto the previous example by adding the nodes: 17, 16, 15, 13, 14. Right-left rotation

25. 4 4 2 2 6 6 1 1 3 3 5 5 16 16 7 17 17 7 AVL Trees Let’s build onto the previous example by adding the nodes: 17, 16, 15, 13, 14. Right-left rotation

26. 4 2 6 1 3 5 16 16 7 17 17 15 15 AVL Trees Let’s build onto the previous example by adding the nodes: 17, 16, 15, 13, 14. 4 Right-left rotation 2 6 1 3 5 7

27. 4 4 2 2 6 7 6 1 1 3 3 5 5 7 16 16 17 17 15 15 AVL Trees Let’s build onto the previous example by adding the nodes: 17, 16, 15, 13, 14. Right-left rotation

28. 4 7 2 7 6 6 1 3 5 5 4 16 16 2 15 17 13 17 1 15 3 13 AVL Trees Let’s build onto the previous example by adding the nodes: 17, 16, 15, 13, 14. single left rotation

29. 7 7 6 6 5 5 4 4 16 16 2 2 15 14 17 17 1 1 3 3 13 13 AVL Trees Let’s build onto the previous example by adding the nodes: 17, 16, 15, 13, 14. Left - right rotation 14 15

30. Red-Black Trees • Similar idea to AVL Trees • Similar worse case time complexity • Keep tree somewhat balanced • not as rigidly balanced as AVL • A Red-Black tree with n nodes has height at most 2log(n+1), i.e., most of the common operations are performed in O(log n). • Longest path is no more than twice as long as the shortest branch

31. Red-Black Trees • A red-black tree is an extended binary tree (every node has two children or is a leaf) which satisfies the following properties: • The root is black (if non-empty) • Root property • If a node is red, it’s parent must be black • Red property • Every path from the root to a leaf/single-child node contains the same number of black nodes. This number is called the black-height of the tree. • Path property

32. Red-Black?

33. Red-Black? yes no yes

34. Red-Black?

35. Red-Black? no

36. Properties Revisited • The root is black (if non-empty) • Root property • If a node is red, it’s parent must be black • Red property • Every path from the root to a leaf/single-child node contains the same number of black nodes. This number is called the black-height of the tree. • Path property

37. Adding null leaf nodes • Every node has 2 children • Don’t include null nodes in the black-height of the tree

38. Properties Revisited • Objective: • Restore red property violation while preserving other properties (i.e. path property). • Techniques: re-color and/or rotate.

39. Property Restoration • Invariant: • There is exactly one red node xin the tree whose parent may be red. • When a new node is added, it is always added as a red node • Strategy: • Fix the violation of red property at x. • This may violate the condition at some ancestor. • Continue fixing the property at the ancestor (ancestor becomes x)

40. Property Restoration • Terminology: • x is the current node in violation of the red property • It is red, and its parent is also red • parent(x) is its parent • parent(parent(x) ) is its grandparent (i.e. grandparent(x)) • The other sibling of parent(x) is its uncle (i.e. uncle(x))

41. Property Restoration • There are several cases: • Case 1: • X is the root • Case 2: • Parent(x) is black • Case 3: • Parent(x) and Uncle(x) are both red • Case 4: • Parent(x) is red, but Uncle(x) is black • Multiple parts of this case…

42. Property Restoration Case 1: X is the root Change its color to black. x x

43. Property Restoration Case 2: Parent(x) is black Don’t do anything p p x x

44. Property Restoration Case 3: Parent(x) and Uncle(x) are both red Change the color of the parent and its uncle to black. Change the color of the grandparent node to red, and repeat with x = grandparent(x) g g p u p u x x

45. Property Restoration Case 4: Parent(x) is red, but Uncle(x) is black There are 4 different cases, much like the 4 different AVL cases left-left left-right right-left right-right g g g g p u p u u p u p x x x x

46. Property Restoration Case 4 cont: left-left and right-right are symmetric left-right and right-left are symmetric left-left left-right right-left right-right g g g g p u p u u p u p x x x x

47. Property Restoration Case 4 left-left: First rotate the parent RIGHT Then recolor by switching colors of P and G Case 4 right-right is symmetric – simply rotate LEFT instead g p p u p x g x g x 3 4 5 1 2 3 u 1 2 3 u 1 2 4 5 4 5

48. Property Restoration Case 4 left-right: First rotate x LEFT (shown below) then rotate the parent RIGHT and recolor by switching colors of P and G Case 4 right-left is symmetric – simply rotate RIGHT then LEFT g g u p x u 1 x 4 5 p 3 4 5 1 2

49. Property Restoration Case 4 left-right: First rotate x LEFT (shown previously) then rotate the parent RIGHT and recolor by switching colors of X and G (just like left-left case but where p and x switch rolls) Case 4 right-left is symmetric – simply rotate RIGHT then LEFT g x x u x p g p g p 3 4 5 1 2 3 u 1 2 3 u 1 2 4 5 4 5

50. Insertion Complexity • Observations: • Every one of the three cases take constant time. • Case 3 brings x two steps closer to the root and only re-colors nodes without any rotations. • For the multiple cases in 4, one or two rotations plus a re-coloring are required and we are done. • Lemma: • An insertion into a red-black tree with n nodes takes O(log2 n) time and performs at most two rotations.