COSC 2P03

1. COSC 2P03 Lecture #5 – Trees Part III, Heaps

2. Today • Take up the quiz • Assignment Questions • Red-Black Trees • Binary Heaps • Heap sort • D-Heaps, Leftist Heaps • Brief: Skew, Binomial, Fibonacci

3. Review From Last Class • Why balance trees? • How to DSW balanced tree? • What is the idea behind AVL balancing? • What is the idea behind splay trees? • When should we use AVL? Splay? • What is amortized analysis?

4. Red Black Trees • Is really a 2-4 tree, that has been represented as a binary tree • Comparing with 2-4 tree • Still has O(lgN) performance • Simpler to implement! 5 2 5 8 2 8

5. Red-Black Trees • Is a BST where • Every node is either colored “Red” or “Black” • Every null pointer is Black • Meaning that every “real” node has 2 children • If a node is Red, then its children are Black • So, can’t have 2 consecutive reds on a path • Every path from root to null contains the same # of Black nodes • Meaning that all leaves have the same “Black Height” • The root is always black • The BLACK HEIGHT of a node N is the number of Black Nodes on any path to null, not including N – but including null as black

6. The Black-Height • Black Height of the root? • 3 • Of “X” • 2 X

7. 16 14 10 8 7 9 3 2 4 1 Coloring an RBT • Every node is either red or black • Every leaf (null) is black • If a node is red, then both its children are black • All paths from node to descendants contain the same number of black nodes • The root is black

8. 16 14 10 8 7 9 3 2 4 1 Coloring an RBT • Every node is either red or black • Every leaf (null) is black • If a node is red, then both its children are black • All paths from node to descendants contain the same number of black nodes • The root is black

9. 16 14 10 8 7 9 3 2 4 1 Coloring an RBT • Every node is either red or black • Every leaf (null) is black • If a node is red, then both its children are black • All paths from node to descendants contain the same number of black nodes • The root is black

10. 16 14 10 8 7 9 3 2 4 1 Coloring an RBT • Every node is either red or black • Every leaf (null) is black • If a node is red, then both its children are black • All paths from node to descendants contain the same number of black nodes • The root is black

11. 16 14 10 8 7 9 3 2 4 1 Coloring an RBT • Every node is either red or black • Every leaf (null) is black • If a node is red, then both its children are black • All paths from node to descendants contain the same number of black nodes • The root is black

12. 16 14 10 8 7 9 3 2 4 1 Coloring an RBT H = 4 B = 2 H = 3 B = 2 H = 2 B = 2 H = 1 B = 1 H = 2 B = 1 H = 2 B = 1 H = 1 B = 1 H = 1 B = 1 • Every node is either red or black • Every leaf (null) is black • If a node is red, then both its children are black • All paths from node to descendants contain the same number of black nodes • The root is black

13. Theorem 1 • Any RBT (Red-Black Tree) rooted at X has n ≥ 2bh(X) – 1 nodes • bh(X) is black height of node X • Proof: (idea behind inductive proof) • Consider a node with 2 children • Each child has a bh(X) or bh(X)-1, depending on if it is red or black • Since height of child < height of X, can apply 2bh(X) – 1 for the children node: 2bh(X)-1 – 1 • So, subtree rooted at X has at least: (2bh(X)-1 – 1)+(2bh(X)-1 – 1) + 1= 2bh(X) – 1 internal nodes

14. Theorem 2 • In an RBT at least ½ the nodes on any path from the root to null must be Black • Proof: • If there is a red node, by definition of RBT there must exist a black node • Meaning that bh(X) ≥h/2

15. Theorem 3 • There does not exist a path from any X to null that is more than twice as long as any other path from X to any another null • Proof: • Any path from X to any null contains the same number of black nodes • From Theorem 2, we know that at least ½ of the nodes on any path are black • This means that the length of any path from X is no more than twice as long as any other

16. Theorem 4 • A RBT with N nodes has a height of h ≤ 2 lg(N + 1) • Proof: • Let h be the height of an RBT rooted at X • From Theorem 2: bh(X) ≥ h/2 • From Theorem 1: N≥ 2bh(X) – 1 • Thus, N ≥ 2 h/2 – 1 N + 1 ≥ 2h/2lg(N + 1) ≥ h/22lg(N + 1) ≥ h • Implying that the height of an RBT is O(lgN)

17. Insertion • Insert a node Z uses the same insertion algorithm as BST, where Z is given the color red (unless it is the root) • Preserve root, external and depth properties • If parent of Z is black, also preserves internal property • If parent of Z is red, it causes a violation of internal property and the tree must be reorganized • How to fix this?

18. Fixing the Double Red • Let Z be the newly inserted node, P its parent and S its sibling (uncle/aunt of Z), then there are 2 cases that may arise • Case 1: S is red • Double red corresponds to an overflow • Recolor the nodes • In 2-4 tree same as a split • Case 2: S is black • Double red is an incorrect placement of a 4-node • Restructure the 4-node

19. G 5 P S 2 9 7 Re-coloring • Fixes the case where the parent node P and its sibling S are both red • P and S become black and parent G of P and S becomes red, unless it is the root • It is possible that this can propagate back to the grandparent of G G 5 P S 2 9 7

20. Restructuring • Fixes the case where the Parent (P) is red and its sibling (S) is black • Same as restoring the correct location of a 4-node • Restores internal property • There are 4 cases to consider, depending on structure of subtree • Same as the ones used in AVL trees, but slightly modified to keep the color properties

21. Ex. Restructuring • 1 2 3 4 • => 8 3 3 8 6 8 6 3 3 6 8 6 6 3 8

22. Insertion Algorithm • Search for location to insert node Z Add Z to the tree and color it red while doubleRed(Z) if sibling of Parent(Z) is black Z = restructure(Z) else Z = recolor(Z) • Searching is O(lgN) • Adding Z is O(1) • At most 1 restructure: O(lgN), O(lgN) recolorings each is O(1) • So, insertion is O(lgN)

23. Insertion Example 10 Insert 10, 3, 15, 20, 1 5 15 1 20

24. Insertion Example 10 10 Insert 5 3 15 3 15 1 1 5 20 20 .

25. 10 Red  red parent has red child 3 15 1 5 20 7 7 10 Recolor parent, uncle to black. Recolor grandparent to red. Recolor Color red, then fix by recoloring. 3 15 1 5 20 7 Insertion Example 10 Insert 7 3 15 1 5 20

26. 10 Previous recoloring doesn’t work. Uncle (a leaf) already black. 3 15 1 5 20 7 10 8 3 15 Recolor parent black. Recolor grandparent red. Rotate grandparent. Recolor & Rotate 1 7 20 5 8 Insertion Example 10 Insert 8 3 15 1 5 20 7

27. 10 Uncle is so… Recolor parent, uncle to black. Recolor grandparent to red. 3 15 1 7 20 10 5 8 3 15 9 Recolor 1 7 20 5 8 9 Insertion Example 10 Insert 9 3 15 1 7 20 5 8 Grandparent & its parent might now both be red. Continue.

28. 10 15 7 Rotate 3 8 20 1 5 9 Insertion Example 10 Uncle of current node is black, so… Need to recolor parent & grandparent. 3 15 But current node is a right child while parent is a left child. Want both to be same side. 1 7 20 5 8 9

29. Insertion Example 10 7 15 7 3 10 Recolor & Rotate 3 8 20 1 5 8 15 1 5 9 9 20

30. Deletion • Basically the same as insertion • Delete as you would in a BST, then check to maintain properties • Read about this in Weiss (12.2) and/or Cormen (13)

31. Conclusion • Red Black vs Splay Trees • Fewer Rotations • 1 extra color bit • Where are they used? • Generally for look-up tables (key, value) data • In Java: The TreeMap and TreeSet classes in the java.util package • When would you use? • AVL? • B-Tree? • Red-Black? • Splay?

33. Priority Queues • Unordered Linked List • Insert O(1), delete O(n) • Ordered Linked List • Insert O(n), delete O(1) • Balanced BST • Insert, delete, remove O(log n) • Merge O(n) • A better way exists…Heaps!!!

34. Binary Heaps • Binary Heaps Are Complete and Partially ordered trees • Complete: Filled on all levels, except the last one where it is filled left to right (so between 2h and 2h+1-1 nodes) • Partially ordered: Key of each node is <= parent (for min-heap) or >= parent (max-heap) • Note: left child is not necessarily less than right child • Since they are complete, can be easily stored as an array

35. Binary Heaps • Min/Max element is in the root • Heap with N elements has height 6 14 45 78 18 47 53 91 77 83 84 81 99 64

36. Insertion • Idea: • Put into next available leaf node • Recursively, if the new node is < parent then swap (if min-heap) • This may “bubble up” to the root, also called “sifting up” • Complexity? • O(lgN), but on average O(1)

37. next free slot 42 Insertion • Insert element X into heap • Insert into next available slot • Bubble up until obeys heap ordering rule (min/max heap) 6 14 45 78 18 47 53 91 77 83 84 81 99 64

38. 42 42 Insertion • Insert element X into heap • Insert into next available slot • Bubble up until obeys heap ordering rule (min/max heap) 6 14 45 swap with parent 78 18 47 53 91 77 83 84 81 99 64

39. 42 Insertion • Insert element X into heap • Insert into next available slot • Bubble up until obeys heap ordering rule (min/max heap) 6 swap with parent 14 45 78 18 47 42 91 77 83 84 81 99 64 53

40. Insertion • Insert element X into heap • Insert into next available slot • Bubble up until obeys heap ordering rule (min/max heap) 6 stop: heap ordered 14 42 78 18 47 45 91 77 83 84 81 99 64 53

41. Deletion • Idea: • Remove the root from the tree • Remove the last (rightmost) leaf and place it at the root • Perform the “sift down” operation • If the key is larger then swap with the smallest child, repeat. • Complexity? • O(lgN), on average O(lgN)

42. Binary Heap: Delete Min • Delete minimum element from heap. • Exchange root with rightmost leaf. • Bubble root down until it's heap ordered. 6 14 42 78 18 47 45 91 77 83 84 81 99 64 53

43. Binary Heap: Delete Min • Delete minimum element from heap. • Exchange root with rightmost leaf. • Bubble root down until it's heap ordered. 53 14 42 78 18 47 45 91 77 83 84 81 99 64 6

44. Binary Heap: Delete Min • Delete minimum element from heap. • Exchange root with rightmost leaf. • Bubble root down until it's heap ordered. 53 exchange with left child 14 42 78 18 47 45 91 77 83 84 81 99 64

45. Binary Heap: Delete Min • Delete minimum element from heap. • Exchange root with rightmost leaf. • Bubble root down until it's heap ordered. 14 exchange with right child 53 42 78 18 47 45 91 77 83 84 81 99 64

46. Binary Heap: Delete Min • Delete minimum element from heap. • Exchange root with rightmost leaf. • Bubble root down until it's heap ordered. 14 stop: heap ordered 18 42 78 53 47 45 91 77 83 84 81 99 64

47. Building a Heap • Start at the root • Recursively travel to each of the leaves. • From each leaf, perform the “sift up” operations • Pseudo-code: • for i= downto 1 minHeapify(A,i) //or maxHeapify • Where A is an array representation of the heap, and i is the current node being inspected for the heap property

48. Fix the bottom level Fix the next to bottom level Fix the top level

49. Building a Heap

50. Complexity of Heap Building • Theorem: Given a perfect binary tree of height h, containing 2h+1-1 nodes the sum of the heights of the nodes is 2h+1-1-(h+1) • What is the complexity? • O(n) • Why? • Sum of the heights of the nodes is N • Proof: