CSC 332 – Algorithms and Data Structures

1. CSC 332 – Algorithms and Data Structures Binary Search Trees – Unbalanced and Balanced Dr. Paige H. Meeker Computer Science Presbyterian College, Clinton, SC (Portions of these notes from Weiss book and University of Washington Website)

2. Binary Search Trees • Goal: Create a simple data structure that will allow insertions and deletions to be O(log N) on average. • (Unfortunately, without “tweaking”, these operations also take O(N) worst case, though they are O(log N) on average.

3. Binary Search Tree (BST) • For any node in the BST, all smaller nodes are in the left subtree and all larger nodes are in the right subtree. Duplicates are not allowed.

4. BST Operations • insert • find • findMin • findMax • remove • removeMin • removeMax How would we implement these?

5. BST Operations find • Start at the root and repeatedly branch to either the left or right, depending on a comparison with the element in the node and the element we are searching for. • Can now either return true (found), false (not found), or insert the value at this point.

6. BST Operations find - insert • To insert a node, use the “find” method and if null is found, insert at that point.

7. Example Growing a Binary Search Tree 7 1. insert 7 3 9 2. insert 3 3. insert 9 5 4. insert 5 5. insert 6 6

8. BST Operations findMin • Start at the root and repeatedly branch left until we run out of left children. The stopping point is the smallest element

9. BST Operations findMax • Start at the root and repeatedly branch right until we run out of right children. The stopping point is the largest element

10. BST Operations remove • Most difficult operation • Non-leaf nodes hold the tree together and we don’t want to disconnect the tree • We need to keep the tree connected and maintain the BST property, while avoiding making the tree unnecessarily deep.

11. BST Operations remove • Consider: • Node is a leaf • Node has one child • Node has two children

12. BST Operations remove • Node is a leaf – easy! Just remove it. It’s removal does not disconnect the tree.

13. BST Operations remove • Node has one child – not too difficult. • Node can be removed after adjusting its parent’s child link to bypass the node.

14. BST Operations remove • Node has two children – most difficult case • Node is replaced by using the smallest item in the right subtree and then removing that item • The smallest item in the right subtree is either a leaf (simple remove) or has a right subtree of its own, (a one child removal) • (See Example)

15. BST Remove Node with 2 Children

16. BST Remove Node with 2 Children

17. BST Implementation In order to implement a BST, we need to understand the concept of a “node” of the tree – a data structure that will contain references to another object as well as to two additional nodes. Your author created a small, simple class to serve this purpose. The same purpose could be served if the BinaryNode class was an inner class.

18. Class “BinaryNode” class BinaryNode<AnyType> { BinaryNode( AnyType theElement ) { element = theElement; left = right = null; } // Data; accessible by other package routines AnyType element; // The data in the node BinaryNode<AnyType> left; // Left child BinaryNode<AnyType> right; // Right child }

19. Class “BinarySearchTree” // CONSTRUCTION: with no initializer // // *********PUBLIC OPERATIONS**************** // void insert( x ) --> Insert x // void remove( x ) --> Remove x // void removeMin( ) --> Remove minimum item // Comparable find( x ) --> Return item that matches x // Comparable findMin( ) --> Return smallest item // Comparable findMax( ) --> Return largest item // boolean isEmpty( ) --> Return true if empty; else false // void makeEmpty( ) --> Remove all items // **************ERRORS*************************** // Exceptions are thrown by insert, remove, and removeMin if warranted /** * Implements an unbalanced binary search tree. * Note that all "matching" is based on the compareTo method. * @author Mark Allen Weiss */

20. Class “BinarySearchTree” Class contains both public and private methods. The public methods have implementations to call the hidden ones. The only data member is the reference to the root of the tree root. If the tree is empty, the root is null.

21. Class “BinarySearchTree” Public Methods public BinarySearchTree( ) // Construct the tree { root = null; } public void insert( AnyType x ) // insert x into the tree { root = insert( x, root ); } public void remove( AnyType x ) // remove x from the tree { root = remove( x, root ); } public void removeMin( ) // remove minimum item from the tree { root = removeMin( root ); } public AnyType findMin( ) // find smallest item in the tree { return elementAt( findMin( root ) ); } public AnyType findMax( ) // find the largest item in the tree { return elementAt( findMax( root ) ); } public AnyType find( AnyType x ) // find x in the tree { return elementAt( find( x, root ) ); } public void makeEmpty( ) // Make the tree logically empty { root = null; } public boolean isEmpty( ) // Test if the tree is logically empty { return root == null; }

22. Class “BinarySearchTree” Next, there are several methods that operate on a node passed as a parameter. The idea is that the publicly visible routines call these hidden routines and pass root as a parameter. These hidden routines do all the work.

23. Class “BinarySearchTree” Private and Protected Methods /** * Internal method to get element field. * @param t the node. * @return the element field or null if t is null. */ private AnyType elementAt( BinaryNode<AnyType> t ) { return t == null ? null : t.element; }

24. Class “BinarySearchTree” Private and Protected Methods /** * Internal method to insert into a subtree. * @param x the item to insert. * @param t the node that roots the tree. * @return the new root. * @throws DuplicateItemException if x is already present. */ protected BinaryNode<AnyType> insert( AnyType x, BinaryNode<AnyType> t ) { if( t == null ) t = new BinaryNode<AnyType>( x ); else if( x.compareTo( t.element ) < 0 ) t.left = insert( x, t.left ); else if( x.compareTo( t.element ) > 0 ) t.right = insert( x, t.right ); else throw new DuplicateItemException( x.toString( ) ); // Duplicate return t; }

25. Class “BinarySearchTree” Private and Protected Methods /** * Internal method to remove from a subtree. * @param x the item to remove. * @param t the node that roots the tree. * @return the new root. * @throws ItemNotFoundException if x is not found. */ protected BinaryNode<AnyType> remove( AnyType x, BinaryNode<AnyType> t ) { if( t == null ) throw new ItemNotFoundException( x.toString( ) ); if( x.compareTo( t.element ) < 0 ) t.left = remove( x, t.left ); else if( x.compareTo( t.element ) > 0 ) t.right = remove( x, t.right ); else if( t.left != null && t.right != null ) // Two children { t.element = findMin( t.right ).element; t.right = removeMin( t.right ); } else t = ( t.left != null ) ? t.left : t.right; return t; }

26. Class “BinarySearchTree” Private and Protected Methods /** * Internal method to remove minimum item from a subtree. * @param t the node that roots the tree. * @return the new root. * @throws ItemNotFoundException if t is empty. */ protected BinaryNode<AnyType> removeMin( BinaryNode<AnyType> t ) { if( t == null ) throw new ItemNotFoundException( ); else if( t.left != null ) { t.left = removeMin( t.left ); return t; } else return t.right; }

27. Class “BinarySearchTree” Private and Protected Methods /** * Internal method to find the smallest item in a subtree. * @param t the node that roots the tree. * @return node containing the smallest item. */ protected BinaryNode<AnyType> findMin( BinaryNode<AnyType> t ) { if( t != null ) while( t.left != null ) t = t.left; return t; }

28. Class “BinarySearchTree” Private and Protected Methods /** * Internal method to find the largest item in a subtree. * @param t the node that roots the tree. * @return node containing the largest item. */ private BinaryNode<AnyType> findMax( BinaryNode<AnyType> t ) { if( t != null ) while( t.right != null ) t = t.right; return t; }

29. Class “BinarySearchTree” Private and Protected Methods /** * Internal method to find an item in a subtree. * @param x is item to search for. * @param t the node that roots the tree. * @return node containing the matched item. */ private BinaryNode<AnyType> find( AnyType x, BinaryNode<AnyType> t ) { while( t != null ) { if( x.compareTo( t.element ) < 0 ) t = t.left; else if( x.compareTo( t.element ) > 0 ) t = t.right; else return t; // Match } return null; // Not found }

30. BST Analysis of Operations • Cost of each BST operation is proportional to the number of nodes accessed during the operation. • Cost of access of any node in the BST is 1+depth of the node. • This cost is logarithmic for a well balanced tree, but can be as bad as linear. (How? – What would the tree look like?)

31. BST Analysis of Operations • The internal path length of a binary tree is the sum of the depths of its nodes • The internal path length is used to measure the cost of a successful search.

32. BST Analysis of Operations • The external path length of a binary tree is the sum of the depths of the N+1 null links. The terminating null node is considered a node for these purposes. • The external path length is used to measure the cost of an unsuccessful search.

33. BST Analysis of Operations • What type of input would cause our worst case tree?

34. BST Analysis of Operations • How do we solve the problem? • We must insist on balance – no node is allowed to get too deep. • Several algorithms can be used to implement a balanced binary search tree, which guarantees logarithmic depth in the worst case. • Plus’? Faster access time, Protection against poor performance, Searching averages 25% faster. • Minus’? More complicated to implement, longer insertions and deletions

35. Balanced BST’s • AVL Trees • Red-Black Trees • AA Trees

36. AVL Trees • First balanced BST • Named after its discoverers, Adelson-Velskii and Landis • Balance condition ensures that depth of the tree is O(logN).

37. AVL Trees • An AVL Tree is a binary search tree with the additional balance property that, for any node in the tree, the height of the left and right subtrees can differ by at most 1. • Updates in AVL trees could destroy the balance – therefore, before the operation is complete, the tree must be rebalanced if necessary.

38. AVL Trees • So, we must modify insert and remove to rebalance the tree if necessary • We must also keep track of the height balance factor of the subtrees. • Only nodes on the path from the root to the insertion point can have their balances altered, because only those nodes have their subtrees altered.

39. AVL Trees – Balance Factor • balance factor, for a tree node n : • height of n's right subtree minus height of n's left subtree • BFn = Heightn.right - Heightn.left • a binary search tree is an AVL tree if: • balance factor of each node is 0, 1 or -1(if no node's two child subtrees differ in height by more than 1)

40. AVL Rebalancing • As we follow the path up to the root from the insertion/deletion of a node, and update the balances, we may find a node that violates the AVL condition. Rebalancing the tree at this first node (i.e. deepest) will guarantee that the tree is an AVL tree once more. • Do we believe this?? (We should!)

41. AVL Rebalancing There are four cases to consider that causes the AVL tree to become unbalanced at node “X” • An insertion into the left subtree of the left child of X • An insertion into the right subtree of the left child of X • An insertion in the left subtree of the right child of X • An insertion in the right subtree of the right child of X Cases 1 and 4 are mirror images, as are 2 and 3.

42. AVL Rebalancing Balance is restored by tree rotations. There are two types: • Single rotation – switches the roles of the parent and child while maintaining the search order. • Double rotation – involves 3 nodes and 4 subtrees; equivalent to two single rotations.

43. AVL Rebalancing • Single rotations occur when the insertion occurs on the outside (left,left or right,right) • Double rotations occur when the insertion occurs on the inside (left,right or right,left) and is more complex.

44. AVL Tree Examples • Two binary search trees: • (a) an AVL tree • (b) not an AVL tree (unbalanced nodes are darkened)

45. AVL Tree Balance Factors 1 0 -1 0 0 0 -1 0 -1 1 -1 0 0 0 0

46. Which are AVL trees?

47. Unbalanced AVL Trees – How to fix??? -1 -2 -2 -1 2 -1 2 0 0 1 0 -1 0 0

48. Problem cases for AVL add 1. insertion into left subtree of node's left child 2. insertion into right subtree of node's left child ...

49. Insertion problem cases, contd 3. insertion into left subtree of node's right child 4. insertion into right subtree of node's right child

50. AVL tree data structure • potential balancing problems occur when a new element is added or removed • Maintain balance using rotations • the idea: reorganize the nodes of an unbalanced subtree until they are balanced, by "rotating" a trio of parent - leftChild - rightChild • tree will maintain its balance so that searches (contains) will take O(log n)