CSE 326 Disjoint Sets and Dynamic Equivalence

CSE 326Disjoint Sets and Dynamic Equivalence David Kaplan Dept of Computer Science & Engineering Autumn 2001

Today’s Outline • Making a “good” maze • Disjoint Set Union/Find ADT • Up-trees • Maze revisited • Weighted Union • Path Compression • An amazeing complexity analysis CSE 326 Autumn 2001 2

Maze Construction Problem Represent maze environment as graph {V,E} • collection of rooms: V • connections between rooms (initially all closed): E Construct a maze: • collection of rooms: V = V • designated rooms in, iV, and out, oV • collection of connections to knock down: E  E such that a unique path connects any two rooms CSE 326 Autumn 2001 3

The Middle of the Maze • So far, some walls have been knocked down while others remain. • Now, we consider the wall between A and B. • Should we knock it down? • no, if A and B are otherwise connected • yes, if A and B are not otherwise connected A B CSE 326 Autumn 2001 4

Maze Construction Algorithm While edges remain in E • Remove a random edge e = (u, v) from E • If u and v have not yet been connected • add e to E • mark u and v as connected CSE 326 Autumn 2001 5

Equivalence Relations An equivalence relation R has three properties: • reflexive: for any x, xRx is true • symmetric: for any x and y, xRy implies yRx • transitive: for any x, y,and z, xRy and yRz implies xRz Connection between rooms is an equivalence relation (call it C) For any rooms a, b, c aCa (A room connects to itself!) If aCb, then bCa If aCb and bCc, then aCc Other equivalence relations? CSE 326 Autumn 2001 6

find(4) {1,4,8} {6} 8 {7} {2,3,6} {5,9,10} {2,3} union(2,6) Disjoint Set Union/Find ADT Union/Find ADT operations • union • find • create • destroy Disjoint set equivalence property: every element of a DS U/F structure belongs to exactly one set Dynamic equivalence property: the set of an element can change after execution of a union CSE 326 Autumn 2001 7

Disjoint Set Union/FindMore Formally Given a set U = {a1, a2, … , an} • Maintain a partition of U, a set of subsets of U {S1, S2, … , Sk} such that: • each pair of subsets Si and Sj are disjoint: • together, the subsets cover U: • The Si are mutually exclusive and collectively exhaustive • Union(a, b) creates a new subset which is the union of a’s subset and b’s subset NOTE: outside agent decides when/what to union - ADT is just the bookkeeper • Find(a) returns a unique name for a’s subset NOTE: set names are arbitrary! We only care that: Find(a) == Find(b)  a and b are in the same subset CSE 326 Autumn 2001 8

a b c 3 10 2 1 6 d e f 4 7 11 9 8 g h i 12 5 Example Construct the maze on the right Initial state (set names underlined): {a}{b}{c}{d}{e}{f}{g}{h}{i} Maze constructor (outside agent) traverses edges in random order and decides whether to union CSE 326 Autumn 2001 9

3 10 2 1 6 4 7 11 9 8 12 5 Example, First Step a b c {a}{b}{c}{d}{e}{f}{g}{h}{i} find(b) b find(e) e find(b)  find(e) so: add 1 to E union(b, e) {a}{b,e}{c}{d}{f}{g}{h}{i} d e f g h i Order of edges in blue CSE 326 Autumn 2001 10

Up-Tree Intuition Finding the representative member of a set is somewhat like the opposite of finding whether a given item exists in a set. So, instead of using trees with pointers from each node to its children; let’s use trees with a pointer from each node to its parent. CSE 326 Autumn 2001 11

Union-Find Up-Tree Data Structure • Each subset is an up-tree with its root as its representative member • All members of a given set are nodes in that set’s up-tree • Hash table maps input data to the node associated with that data a c h g f i d b e Up-trees are not necessarily binary! CSE 326 Autumn 2001 12

Find find(f) find(e) a b c 10 a c g h d e f 7 f i d b 11 9 8 g h i 12 e Just traverse to the root! runtime: CSE 326 Autumn 2001 13

Union union(a,c) a b c 10 a c g h d e f f i d b 11 9 8 g h i 12 e Just hang one root from the other! runtime: CSE 326 Autumn 2001 14

a b c 3 10 Example (1/11) 2 1 6 d e f 4 7 11 9 8 union(b,e) g h i 12 5 a b c d e f g h i a b c d f g h i e

a b c 3 10 Example (2/11) 2 6 d e f 4 7 11 9 8 union(a,d) g h i 12 5 a b c d f g h i e a b c f g h i d e

a b c 3 10 Example (3/11) 6 d e f 4 7 11 9 8 union(a,b) g h i 12 5 a b c f g h i d e a c f g h i b d e

a c f g h i b d e While we’re finding e, could we do anything else? a b c 10 Example (4/11) 6 d e f 4 7 11 9 8 find(d) = find(e) No union! g h i 12 5

a b c 10 Example (5/11) 6 d e f 7 11 9 8 union(h,i) g h i 12 5 a c f g h i a c f g h b d i b d e e

a b c 10 Example (6/11) 6 d e f 7 11 9 8 union(c,f) g h i 12 a c f g h a c g h b d b d f i i e e

a c g h c g h b d f a f i i b d e Could we do a better job on this union? e a b c 10 Example (7/11) d e f 7 find(e) find(f) union(a,c) 11 9 8 g h i 12

a b c 10 Example (8/11) d e f find(f) find(i) union(c,h) 11 9 8 g h i 12 c g h c g a f a f h i b d b d i e e

a b c 10 Example (9/11) d e f find(e) = find(h) and find(b) = find(c) So, no unions for either of these. 11 9 g h i 12 c g a f h b d i e

a b c Example (10/11) d e f find(d) find(g) union(c, g) 11 g h i 12 c g g c a f h a f h b d i b d i e e

a b c Example (11/11) d e f find(g) = find(h) So, no union. And, we’re done! g h i 12 g a b c c d e f a f h g h i b d i Ooh… scary! Such a hard maze! e

a c g h b d f i e 0 (a)1 (b)2 (c)3 (d)4 (e)5 (f)6 (g)7 (h)8 (i) -1 0 -1 0 1 2 -1 -1 7 Disjoint set data structure • A forest of up-trees can easily be stored in an array. • Also, if the node names are integers or characters, we can use a very simple, perfect hash. Nifty storage trick! up-index: CSE 326 Autumn 2001 26

Implementation typedef ID int; ID find(Object x) { assert(hTable.contains(x)); ID xID = hTable[x]; while(up[xID] != -1) { xID = up[xID]; } return xID; } ID union(ID x, ID y) { assert(up[x] == -1); assert(up[y] == -1); up[y] = x; } runtime: O(depth) or … runtime: O(1) CSE 326 Autumn 2001 27

a c g h b d f i e Room for Improvement:Weighted Union • Always makes the root of the larger tree the new root • Often cuts down on height of the new up-tree c g h a g h a f b d i c i b d f e Could we do a better job on this union? e Weighted union! CSE 326 Autumn 2001 28

Weighted Union Code typedef ID int; ID union(ID x, ID y) { assert(up[x] == -1); assert(up[y] == -1); if (weight[x] > weight[y]) { up[y] = x; weight[x] += weight[y]; } else { up[x] = y; weight[y] += weight[x]; } } new runtime of union: new runtime of find: CSE 326 Autumn 2001 29

Weighted Union Find Analysis • Finds with weighted union are O(max up-tree height) • But, an up-tree of height h with weighted union must have at least 2h nodes •  2max height n and max height  log n • So, find takes O(log n) Base case: h = 0, tree has 20 = 1 node Induction hypothesis: assume true for h < h A merge can only increase tree height by one over the smaller tree. So, a tree of height h-1 was merged with a larger tree to form the new tree. Each tree then has  2h-1 nodes by the induction hypotheses for a total of at least 2h nodes. QED. CSE 326 Autumn 2001 30

a c f g h i b d e Room for Improvement: Path Compression • Points everything along the path of a find to the root • Reduces the height of the entire access path to 1 a c f g h i b d e While we’re finding e, could we do anything else? Path compression! CSE 326 Autumn 2001 31

Path Compression Example find(e) c c g g a f h a f h b e b d d i i e CSE 326 Autumn 2001 32

Path Compression Code typedef ID int; ID find(Object x) { assert(hTable.contains(x)); ID xID = hTable[x]; ID hold = xID; while(up[xID] != -1) { xID = up[xID]; } while(up[hold] != -1) { temp = up[hold]; up[hold] = xID; hold = temp; } return xID; } runtime: CSE 326 Autumn 2001 33

Complexity of Weighted Union + Path Compression Ackermann created a function A(x, y) which grows very fast! Inverse Ackermann function (x, y) grows very slooooowwwly … Single-variable inverse Ackermann function is called log* n How fast does log n grow? log n = 4 for n = 16 Let log(k) n = log (log (log … (log n))) Then, let log* n = minimum k such that log(k) n  1 How fast does log* n grow? log* n = 4 for n = 65536 log* n = 5 for n = 265536 (20,000 digit number!) How fast does (x, y) grow? Even sloooowwwer than log* n (x, y) = 4 for n far larger than the number of atoms in the universe (2300) CSE 326 Autumn 2001 34

Complexity of Weighted Union + Path Compression • Tarjan proved that m weighted union and find operations on a set of n elements have worst case complexity O(m(m, n)) • This is essentially amortized constant time • In some practical cases, weighted union or path compression or both are unnecessary because trees do not naturally get very deep. CSE 326 Autumn 2001 35

SummaryDisjoint Set Union/Find ADT • Simple ADT, simple data structure, simple code • Complex complexity analysis, but extremely useful result: essentially, constant time! • Lots of potential applications • Object-property collections • Index partitions: e.g. parts inspection • To say nothing of maze construction • In some applications, it may make sense to have meaningful (non-arbitrary) set names CSE 326 Autumn 2001 36

CSE 326 Disjoint Sets and Dynamic Equivalence