Memory-Constrained Algorithms

1. Memory-Constrained Algorithms DagstuhlSeminar on Computational Geometry March 2011 Tetsuo Asano School of Information Science, JAIST

2. What is Memory Constrained Algorithm? CPU is faster and Memory is cheaper than before Can handle problems of larger sizes Create new directions of computation such as Data Streaming, Massive-Parallel, etc. Lack of memory space Need Space-Efficient Algorithms with less memory space Main Objective Design space-efficient algorithms Establish good space-time tradeoffs

3. "How and where are input data stored?" Two Different Approaches Stored in a regular array with read/write the input array can be used as a work storage.  In-place algorithms Stored in a read-only array  e.g. log-space algorithms work space of O(log n)-bits work space of O(√n) variables???

4. Why read-only array? Embedded software e.g., digital camera, scanner, wireless sensor array Input data are stored outside with random access. Massive-parallel computing Input data are shared with many processors. Easy to design algorithms with read-only arrays. Flash Memory Reading is fast but writing is very slow. Reentrant Program Work space is allocated in addition to shared memory. Theoretical curiosity.

5. Simple example: Given a pointer to the head of a (singly) linked list, determine whether it contains a cycle (snail type) or not (snake type). And, reform it into a snake type if it contains a cycle. Algorithm using an array: How do you specify its length? Algorithm using a linked list: Too much memory consumed!

6. D: a data structure ・Insertion: Insert an address of a list element. ・Query: Determine whether a query address is in D. Algorithm for determining snake or snail Initialize the data structure D. p = head. Insert(p). while p→next ≠NIL and Query(p→next) = FALSE{ p = p→next. Insert(p). } if p →next = NIL then it is a snake type else p→next =NIL. // p→next has been seen before If D can be implemented by a zero-space data structure, then we have a constant-work-space algorithm.

7. What is a zero(or constant)-space data structure? data structure using no work space, but implementing required operations D: a data structure ・Insertion: Insert an address of a list element. ・Query: Determine whether a query address is in D. Insertion: Just maintain the number of elements to be included. In practice, we just increment a counter C. Query: Traverse the list at most C steps to determine whether a query address is encountered. Quadratic time algorithm using work space of O(log n) bits

8. Improvement to linear time idea: baby-step giant step two pointers of different speeds A: advances one step at a time B: two steps at a time Move A and B until both of them point the same entry A, B Conversion from snail type to snake type can be done in linear time using O(log n) bits.

9. Computing Euclidean Minimum Spanning Tree Definition: Minimum spanning tree for a set S of points is a tree connecting all the points with the least total length. Property: (Euclidean) Minimum spanning tree is a subgraph of the Delaunay triangulation of the point set.

10. Property: u, v: two points of S (u, v) is an edge of the MST(S) they are not connected in the subgraph of the Delaunay triangulation of S consisting of those edges shorter than (u, v). Algorithm for computing the MST whenever we find a Delaunay edge, check whether they are connected in a subgraph defined by all edges shorter than the edge. (st-connectivity)

11. Property 2: If (u, v) is not an edge of MST, then adding (u, v) to the MST creates a cycle which forms a face. Property 3: Let DT(u, v) is a subgraph of the Delaunay triangulation consisting of all edges shorter than (u, v). Then, if (u, v) is not an edge of MST, then adding (u, v) to the subgraph DT(u, v) creates a cycle which forms a face.

12. MST DT(u, v) DT(u, v) + (u,v) Search a path

13. Naive Algorithm for Computing EMST ・Compute the Delaunay triangulation D for a set of input points. ・Construct a data structure for the triangulation (e.g. DCEL). ・for each Delaunay edge (u, v) do ・ Define a subgraphD(u, v) consisting of edges of D shorter than the edge (u, v). ・ if D(u, v) has no path between u and v then (u, v) is an edge of EMST (and thus we output it).

14. Designing a zero-space data structure for Delaunay triangulation ・Find a Delaunay edge. ・Find the next Delaunay edge along a face boundary ・Compute the twin edge with opposite direction Can enumerate all Delaunay edges.

15. Theorem 3: Given a set of n points in the plane, we can compute the Euclidean Minimum Spanning Tree for the point set in O(n3) time using work space of O(log n) bits.

16. Applications to Image Processing: Let G be a color image of npixels, and let f be a function from color space into {0, 1}. example: red: 80 < r < 255 green: 0 < g < 80 blue: 0 < b < 255 for each pixel p in G f(p) = 1 if color values at p are within the ranges above = 0 otherwise Then, the function f defines a binary image f(G). We assume that the function f can be computed in constant time.

17. input image red: 80-255, green: 0-80, blue: 0-255

18. red: 80-255, green: 0-80, blue: 0-255

19. red: 80-255, green: 0-80, blue: 0-255

20. Given a color image G of n pixels and a function f from color space into {0, 1}, we can define a binary image f(G). Problem: Compute the number of connected components in the binary image. (2) Output the binary image in raster order such that g(p) = 1 if and only if f(p) = 1 and p belongs to the largest component in f(G). (3) Output the binary image in raster order such that g(p) = 1 if and only if f(p) = 1 and p belongs to a component of size > K in f(G). (4) Output a label matrix M(f(G))in raster order such that same component  same label different components  different labels value 0  label 0

21. input image red: 80-255, green: 0-80, blue: 0-255

23. Component labeling

24. Component labeling with size > 200 pixels

25. Component labeling : largest component

26. Labeling problem What is a connected component? G: an nxn binary image (matrix) G[p]= 0 or 1 for each pixel p=(x, y) Two 1-pixels are connected if there is a rectilinear sequence of 1-pixels between them. A connected component is a maximal set of 1-pixels any two of which are connected.

27. FILL Algorithm for Connected Component Labeling L = 1 for each pixel ps.t. G[p]=1 in raster order Increment L. G[p] = L. Push p into a stack. while(stack is not empty){ Pop a pixel r from the stack. for each pixel p with G[p]=1 adjacent to r Push p into the stack . g[p] = L. } R. Klette and A. Rosenfeld: ``Digital Geometry: Geometric Methods for Digital Picture Analysis,'' Elsevier, 2004.

28. 1 2 4 3 worst case: work space of O(n log n) bits for the stack and O(n log n) bits for the labeling matrix = 0 = 1

29. Is it possible to reduce the work space by using a queue? = 0 = 1

30. Observation: Given a color image G of n pixels and a function f from color space into {0, 1}, we can compute a labeling matrix for the binary image f(G) in O(n) time using work space of O(n log n) bits. If we need the labeling matrix as the output, the algorithm is optimal. The labeling matrix requires Q(n log n) bits. New situation: don't need to store the labeling matrix. just need to output the matrix elements in raster order. How much work space can be saved?

31. Enumeration of all components together with their areas Given a binary image G, enumerate all connected components together with their areas (number of pixels) each exactly once. There is an algorithm for enumerating all components together with their areas in a read-only binary image in O(n log n) time using work space of O(log n) bits.

32. Key Idea for Counting Basic assumption: Component boundary is directed so that the interior always lies to the left of the boundary External boundary : counter-clockwise order Internal boundary : clockwise order Canonical edge for each boundary: the lowest downward edge

33. Each boundary (internal/external) has a unique canonical edge, which is the lowest downward edge on the boundary.

34. Local configuration around a starting edge 0 0 1 1 0 1 External boundary Internal boundary Checking orientation So easy to check the orientation!!

35. Algorithm for Connected Components Counting counter = 0. Perform Raster Scan whenever we find a candidate downward edge e. Apply Boundary_trace_from(e). if it returns 1 then increment the count Report the count. 0 1 ? 0 Boundary_trace_from(e){ es = e. do{ e = next edge of e on the same boundary. if e is downward and lower than es then return 0. }while(e != es) return 1. }

36. Theorem: The algorithm Connected-Components-Counting correctly computes the number of connected components in a given binary image of n pixels in O(n2) time using only work space of O(log n) bits. worst-case example

37. How to improve the time complexity bidirectional search whenever we find a candidate edge e, search on the boundary in two opposite directions for any lower edge. worst-case example which requires O(n log n) for an image with n pixels

38. Theorem: Given a binary image of n pixels on a read-only array, we can enumerate all components together with their areas in the image in O(n log n) time using work space of O(log n) bits. Counting can be done O(n log n) time. Possible to extend the result to labeling problem? Given a binary image of n pixels, is it possible to output labels of pixels in raster order using work space of O(log n) bits in polynomial time?

39. Summary of Previous Results １．Region Expansion Starting from an arbitrary white pixel, we expand a region by incorporating neighboring white pixels using a queue. Time: O(n) Work space: O(n log n) bits for the queue O(n) bits for mark bits Also random access over entire image is needed.

40. ２．Merging adjacent runs First Step: Scan a given image row by row to partition each row into a sequences of 0-runs and 1-runs. Second Step: Scan the image column by column to merge vertically adjacent 1-runs using union-find tree. Third Step: Scan the image again row by row to put a label at each pixel. Time: O(n a(n)) Work Space: O(n log n) bits for labels O(n log n) bits for union-find tree Access: horizontal and vertical scans

41. ３．Following the boundaries First Step: Follow each boundary (external or internal) to put a label to the right of each downward edge. Second Step: Propagate labels to the right until we encounter boundary edges Time: O(n) Work Space: O(n log n) bits for labels no other array is needed Access: random access over the entire image

42. ４．New Algorithm for Labeling Time: O(n√n) Work Space: O(√n log n) bits． Labels are not stored in an array but just output in order. Access: raster scans √n times Rough Sketch of the Algorithm ・Partition a row into runs (raster scan) ・Keeping labeling information of two rows, vertical adjacencies are also maintained. ・At each row, we partition a run-adjacency graph into connected components to compute labels. ・Assign sequential numbers to the resulting components to output a label of each 1-pixel in raster order.

43. Algorithm for counting the number of connected components in a binary image Problem: Given a binary image of n pixels, compute the number of connected components in the image. Work space of O(n log n) bits count the number of components after labeling O(n) time Work space of O(log n) bits still possible to count them, but taking O(n log n) time Work space of O(√n log n) bits how fast can we count them? O(n) time?

44. Important Observation: For any row, the number of connected components intersecting the row is O(√n). Assume: the image is of size O(√n)xO(√n). Basic Idea of our Algorithm Scan a given image in the raster order while maintaining connected components intersecting each row. Labels of connected components should be reused to save work space. We maintain labels in current two rows during the raster scan. Array A[ ]: labels in the current row Array B[ ]: labels in the next row (to be computed)

45. We maintain labels in current two rows during the raster scan. Array A[ ]: labels in the current row Array B[ ]: labels in the next row (to be computed) Partition a row into runs (of 1-pixels) Assuming that runs on A[ ] have been labeled. Put provisional labels (r1, r2, ..., rk) to runs on B[ ] in order. Examine the adjacency among those runs on the two rows. Case 1： A[] has no run adjacent to a run rion B[ ] This run rimay be a part of a new component, and thus a new label is given to the run.

46. Case 1： A[] has no run adjacent to a run rion B[ ] This run rimay be a part of a new component, and thus a new label is given to the run. Case 2: B[] has no run adjacent to a run of label m on A[] This run terminates in this row. So, increase the count of connected components.

47. Case 1： A[] has no run adjacent to a run rion B[ ] This run rimay be a part of a new component, and thus a new label is given to the run. Case 2: B[] has no run adjacent to a run of label m on A[] This run terminates in this row. So, increase the count of connected components. Case 3: A run on A[] is adjacent to a run on B[]. In this case, the two labels are merged into a single label. label X Merge two labels X and Y

48. Construct a graph representing adjacency among runs and then partition it into connected components. Then, choose a unique label in each component.

49. Algorithm for computing the number of components N=0 // the number of connected components P = {1, 2, ... , c√n} // set of labels available，c is a constant. Initialize the array A[ ] by 0's foreach rowdo Read the next row into the array B[], and partition it into runs. //Assume that the previous rows has been correctly labeled. Compute a graph representing adjacency among labels in A[ ] and runs in B[ ], and then partition it into connected components. for each component Cin the graph do if C consists of a single run on B[ ] then Take a label out of P and put it to the run if C consists of a single label on A[ ] then Increment the value of N and return the label to P． if C consists of more than one elementthen Choose one label and return other labels to P. endfor Exchange the roles of A[ ] and B[ ]. endfor

50. In the algorithm we maintain a set of labels by P = {1, 2, ... , c√n}, c: a constant > 1 It is guaranteed that we can always find an available label if the constant c is large enough. Operations on the label set P can be done in constant time. Theorem： Given a binary image of n pixels, we can count the number of connected components in the image in O(n) time using work space of O(√n log n) bits. Stronger expression： A single raster scan over a given image is enough to count the number of connected components.