Morphological image processing – Part II

1. Morphological image processing – Part II Hit-and-Miss Transform Gray-value morphological processing

2. Links http://www.mmorph.com/resources.html http://cmm.ensmp.fr/~beucher/wtshed.html http://www.cwi.nl/projects/morphology/ http://www.ph.tn.tudelft.nl/Courses/FIP/noframes/fip-Morpholo.html http://www.dai.ed.ac.uk/HIPR2/morops.htm http://www.mathworks.com/access/helpdesk/help/toolbox/images/images.shtml http://www-dsv.cea.fr/thema/shfj/web/demo_extraction/english/cerveau.htm http://cmm.ensmp.fr/~beucher/interpol/interpol.html

3. Rather than simply probing the inside or the outside of objects in a binary image image , it can be fruitful to probe both at the same time. We can accomplish this using the hit and miss transform. This operation requires a matched pair of structuring elements(B1 , B2), that probe the inside and outside, respectively, of objects in the image. Hit-and-Miss Transform The hit-or-missoperator was defined by Serra as follows. Given an image A and two structuring elements B1 and B2, the set definition and Boolean definition are: where B1 and B2 are bounded, disjoint structuring elements. Two sets are disjoint if B1 (AND) B2 = 0 , the empty set.

4. Hit-and-Miss Transform The structuring element used in the hit-and-miss is a slight extension to the type that has been introduced for erosion and dilation , in that it can contain both foreground and background pixels , rather than just foreground pixels, i.e. both ones and zeros. Note that the simpler type of structuring element used with erosion and dilation is often depicted containing both ones and zeros as well, but in that case the zeros really stand for `don't care's', and are just used to fill out the structuring element to a convenient shaped kernel, usually a square. In all our illustrations, these `don't care's' are shown as blanks in the kernel in order to avoid confusion. An example of the extended kind of structuring element is shown in Figure 1. As usual we denote foreground pixels using ones, and background pixels using zeros.

5. Hit-and-Miss Transform The hit-and-miss operation is performed in much the same way as other morphological operators, by translating the origin of the structuring element to all points in the image, and then comparing the structuring element with the underlying image pixels. If the foreground and background pixels in the structuring element exactly match foreground and background pixels in the image, then the pixel underneath the origin of the structuring element is set to the foreground color. If it doesn't match, then that pixel is set to the background color. In an important sense the hit-and-missoperator is the morphological equivalent of templatematching, a well-known technique for matching patterns based upon cross-correlation. ere, we have a template B1 for the object and a template B2 for the background.

6. Hit-and-Miss Transform For instance, the structuring element shown in Figure 1 can be used to find right angle convex corner points in images. Notice that the pixels in the element form the shape of a bottom-left convex corner. We assume that the origin of the element is at the center of the 3×3 element. In order to find all the corners in a binary image we need to run the hit-and-miss transform four times with four different elements representing the four kinds of right angle corners found in binary images. Figure 2 shows the four different elements used in this operation. Figure 2 Four structuring elements used for corner finding in binary images using the hit-and-miss transform. Note that they are really all the same element, but rotated by different amounts.

7. Hit-and-Miss Transform After obtaining the locations of corners in each orientation, We can then simply OR all these images together to get the final result showing the locations of all right angle convex corners in any orientation. Figure 3 shows the effect of this corner detection on a simple binary image. Figure 3

8. Hit-and-Miss Transform – Guidelines to Use The hit-and-miss transform is used to look for occurrences of particular binary patterns in fixed orientations. It can be used to look for several patterns (or alternatively, for the same pattern in several orientations as above) simply by running successive transforms using different structuring elements, and then ORing the results together. The operations of erosion, dilation, opening, closing, thinning and thickening can all be derived from the hit-and-miss transform in conjunction with simple set operations.

9. Hit-and-Miss Transform – Guidelines to Use Figure 4 illustrates some structuring elements that can be used for locating various binary features. Figure 4 Some applications of the hit-and-miss transform. 1 is used to locate isolated points in a binary image. 2 is used to locate the end points on a binary skeleton Note that this structuring element must be used in all its rotations so four hit-and-miss passes are required. 3a and 3b are used to locate the triple points (junctions) on a skeleton. Both structuring elements must be run in all orientations so eight hit-and-miss passes are required.

10. Hit-and-Miss Transform – Guidelines to Use We illustrate two of these applications on an image skeleton. We start with Skeleton The informal definition of a skeleton is a line representation of an object that is: i) one-pixel thick, ii) through the "middle" of the object, and, iii) preserves the topology of the object. which is the skeleton of

11. Hit-and-Miss Transform – Guidelines to Use Another way to think about the skeleton is as the loci of centers of bi-tangent circles that fit entirely within the foreground region being considered. Figure illustrates this for a rectangular shape. Figure Skeleton of a rectangle defined in terms of bi-tangent circles.

12. Summary of the basic operations The results of the application of these basic operations on a test image are illustrated below. In Figure 5 the various structuring elements used in the processing are defined. The value "-" indicates a "don't care". All three structuring elements are symmetric. Figure 5 : Structuring elements B, B1, and B2 that are 3 x 3 The results of processing are shown in Figure where the binary value "1" is shown in black and the value "0" in white. a) Image Ab)Dilation with 2Bc)Erosion with 2B

13. Summary of the basic operations d)Opening with 2Be)Closing with 2Bf) hit-and-Miss with B1 and B2 The opening operation can separate objects that are connected in a binary image. The closing operation can fill in small holes. Both operations generate a certain amount of smoothing on an object contour given a "smooth" structuring element. The opening smoothes from the inside of the object contour and the closing smoothes from the outside of the object contour. The hit-and-miss example has found the 4-connected contour pixels.

14. Thinning It can be used for several applications, but is particularly useful for skeletonization. In this mode it is commonly used to tidy up the output of edge detectors by reducing all lines to single pixel thickness. Thinning is normally only applied to binary images, and produces another binary image as output. Skeletonization is a process for reducing foreground regions in a binary image to a skeletal remnant that largely preserves the extent and connectivity of the original region while throwing away most of the original foreground pixel.

15. Thinning - How It Works Like other morphological operators, the behavior of the thinning operation is determined by a structuring element. The binary structuring elements used for thinning are of the extended type described under the hit-and-miss transform (i.e. they can contain both ones and zeros). The thinning operation is related to the hit-and-miss transform and can be expressed quite simply in terms of it. The thinning of an image I by a structuring element J is: where the subtraction is a logical subtraction defined by .

16. Thinning Thinning - How It Works The thinning operation is calculated by translating the origin of the structuring element to each possible pixel position in the image, and at each such position comparing it with the underlying image pixels. If the foreground and background pixels in the structuring element exactly match foreground and background pixels in the image, then the image pixel underneath the origin of the structuring element is set to background (zero).Otherwise it is left unchanged. Note that the structuring element must always have a one or a blank at its origin if it is to have any effect.

17. Thinning-Guidelines for Use One of the most common uses of thinning is to reduce the thresholded output of an edge detector such as the Sobel operator , to lines of a single pixel thickness, while preserving the full length of those lines (i.e. pixels at the extreme ends of lines should not be affected). A simple algorithm for doing this is the following: Consider all pixels on the boundaries of foreground regions (i.e. foreground points that have at least one background neighbor). Delete any such point that has more than one foreground neighbor, as long as doing so does not locally disconnect (i.e. split into two) the region containing that pixel. Iterate until convergence. This procedure erodes away the boundaries of foreground objects as much as possible, but does not affect pixels at the ends of lines.

18. This effect can be achieved using morphological thinning by iterating until convergence with the structuring elements shown in Figure , and all their 90° rotations (4×2 = 8 structuring elements in total). Thinning-Guidelines for Use In fact what we are doing here is determining the octagonal skeleton of a binary shape --- the set of points that lie at the centers of octagons that fit entirely inside the shape, and which touch the boundary of the shape at at least two points. Note that this skeletonization method is guaranteed to produce a connected skeleton. Figure 6: Structuring elements for skeletonization by morphological thinning.

19. Thinning-Guidelines for Use At each iteration, the image is first thinned by the left hand structuring element, and then by the right hand one, and then with the remaining six 90° rotations of the two elements. The process is repeated in cyclic fashion until none of the thinning produces any further change. As usual, the origin of the structuring element is at the center. Figure 7 :Example skeletonization by morphological thinning of a simple binary shape, using the above structuring elements. Note that the resulting skeleton is connected.

20. Note that skeletons produced by this method often contain undesirable short spurs produced by small irregularities in the boundary of the original object. These spurs can be removed by a process called pruning, which is in fact just another sort of thinning. Thinning-Guidelines for Use Figure 8: Some applications of thinning. 1 simply finds the boundary of a binary object, i.e. it deletes any foreground points that don't have at least one neighboring background point. Note that the detected boundary is 4-connected. 2 does the same thing but produces an 8-connected boundary. 3a and 3b are used for pruning. At each thinning iteration, each element must be used in each of its four 90° rotations. Pruning is normally carried out for only a limited number of iterations to remove short spurs, since pruning until convergence will actually remove all pixels except those that form closed loops.

21. Thinning-Guidelines for Use Note that many implementations of thinning have a particular structuring element `hardwired' into them (usually the skeletonization structuring elements), and so the user does not need to be concerned about selecting one. Thinning is often used in combination with other morphological operators tp extract a simple representation of regions. A common example is the automated recognition of hand-written characters. In this case, morphological operators are used as pre-processing to obtain the shapes of the characters which then can be used for the recognition. We illustrate a simple example using

22. Thinning-Guidelines for Use .Since we want to work on binary images, we start off by thresholding the image at a value of 180, obtaining A simple way to obtain the skeleton of the character is to thin the image with the structuring elements in the next Figure until convergence. The result is shown in The character is now reduced to a single pixel-wide line. However, the line is broken at some locations, which might cause problems during the recognition process.

23. Thinning-Guidelines for Use To improve the situation we can first dilate the image to connect the lines before thinning it. Dilating the image twice with a 3×3 square structuring element yields then the result of the thinning is

24. Thickening Thinning is the dual of thickening, i.e. thickening the foreground is equivalent to thinning the background. If the foreground and background pixels in the structuring element exactly match foreground and background pixels in the image, then the image pixel underneath the origin of the structuring element is set to foreground (one). Applying the 45° convex hull algorithm described above results in

25. Gray Scale Images For a grayscale image, the intensity value is taken to represent height above a base plane, so that the grayscale image represents a surface in three-dimensional Euclidean space. Figure 9 shows such a surface. Then the set of coordinates associated with this image surface is simply the set of three-dimensional Euclidean coordinates of all the points within this surface and also all points below the surface, down to the base plane. Note that even when we are only considering points with integer coordinates, this is a lot of points, so usually algorithms are employed that do not need to consider all the points.

26. Rules for Grayscale Dilation and Erosion DilationThe value of the output pixel is the maximum value of all the pixels in the input pixel's neighborhood. In a binary image, if any of the pixels is set to the value 1, the output pixel is set to 1. ErosionThe value of the output pixel is the minimum value of all the pixels in the input pixel's neighborhood. In a binary image, if any of the pixels is set to 0, the output pixel is set to 0.

27. Gray-value morphological processing The techniques of morphological filtering can be extended to gray-level images. To simplify matters we will restrict our presentation to structuring elements, B, that comprise a finite number of pixels and are convex and bounded. Now, however, the structuring element has gray values associated with every coordinate position as does the image A. * Gray-level dilation, DG(A,B), is given by: Dilation - For a given output coordinate [m,n], the structuring element is summed with a shifted version of the image and the maximum encountered over all shifts within the J x Kdomain of B is used as the result.This new value is recorded at the origin of the structuring element For a flat structuring element, this is equivalent to the maximum filter.

28. Grayscale dilation with a flat disk shaped structuring element will generally brighten the image. Bright regions surrounded by dark regions grow in size, and dark regions surrounded by bright regions shrink in size. Small dark spots in images will disappear as they are `filled in' to the surrounding intensity value. Small bright spots will become larger spots. The effect is most marked at places in the image where the intensity changes rapidly and regions of fairly uniform intensity will be largely unchanged except at their edges. Gray-level dilation

29. Grayscale erosion Gray-level erosion, EG(*), is given by: Where j and k index the pixels of B. If B is a 3 x 3 with its center as the origin, j and k will range form –1 to 1. For a spatial case of a flat structuring element, this calculation has the same effect as the minimum filter .

30. Grayscale erosion with a flat disk shaped structuring element will generally darken the image. Bright regions surrounded by dark regions shrink in size, and dark regions surrounded by bright regions grow in size. Small bright spots in images will disappear as they are eroded away down to the surrounding intensity value, and small dark spots will become larger spots. The effect is most marked at places in the image where the intensity changes rapidly, and regions of fairly uniform intensity will be left more or less unchanged except at their edges. Figure 11 shows a vertical cross-section through a graylevel image and the effect of erosion using a disk shaped structuring element. Note that the flat disk shaped kernel causes small peaks in the image to disappear and valleys to become wider.

31. Opening - Gray-level opening Graylevel opening can be used to select and preserve particular intensity patterns while attenuating others. As a simple example we start with Bright features smaller than the structuring element have been greatly reduced in intensity, while larger features have remained more or less unchanged in intensity. Note that the image does have a more matt appearance than before since the opening has eliminated small secularities and texture fluctuations.

32. gray-level closing Closing - The dark specks in between the bright spots in the hair have been largely filled in to the same color as the bright spots, while the more uniformly colored nose area is largely the same intensity as before. Similarly the gaps between the white whiskers have been filled in.

33. Morphological filtering of gray-level data. In many situations the seeming complexity of gray level morphological processing is significantly reduced through the use of symmetric structuring elements where b[j,k] = b[-j,-k]. The most common of these is based on the use of B = constant = 0. For this important case and using again the domain [j,k] B, the definitions above reduce to: Dilation - Erosion - Opening - Closing -

34. The operations defined above can be used to produce morphological algorithms for smoothing, gradient determination and a version of the Laplacian. All are constructed from the primitives for gray-level dilation and gray-levelerosion and in all cases the maximum and minimum filters are taken over the domain .

35. Morphological smoothing This algorithm is based on the observation that a gray-levelopening smoothes a gray-value image from above the brightness surface given by the function a[m,n] and the gray-levelclosing smoothes from below. We use a structuring element B based on eqs. and . Top-hat transform acts as a detector of peaks and ridges of the gray level surface.

36. Morphological gradient For linear filters the gradient filter yields a vector representation with a magnitude. and direction .The version presented here generates a morphological estimate of the gradientmagnitude:

37. Morphological Laplacian The morphologically-based Laplacian filter is defined by:

38. Summary of morphological filters a) Dilation b) Erosion c) Smoothing d) Gradient e) Laplacian

39. Skeletonization/Medial Axis Transform Skeletonization is a process for reducing foreground regions in a binary image to a skeletal remnant that largely preserves the extent and connectivity of the original region while throwing away most of the original foreground pixels. This line is the skeleton. Under this definition it is clear that thinning produces a sort of skeleton. The terms medial axis transform (MAT) and skeletonization are often used interchangeably but we will distinguish between them slightly. The skeleton is simply a binary image showing the simple skeleton. The MAT on the other hand is a graylevel image where each point on the skeleton has an intensity which represents its distance to a boundary in the original object.

40. Skeleton/MAT - How It Works The skeleton/MAT can be produced in two main ways. The first is to use some kind of morphological thinning that successively erodes away pixels from the boundary (while preserving the end points of line segments) until no more thinning is possible, at which point what is left approximates the skeleton. The alternative method is to first calculate the distance transform of the image. The skeleton then lies along the singularities (i.e. creases or curvature discontinuities) in the distance transform. This latter approach is more suited to calculating the MAT since the MAT is the same as the distance transform but with all points off the skeleton suppressed to zero.

41. Distance Transform The distance transform is an operator normally only applied to binary images. The result of the transform is a graylevel image that looks similar to the input image, except that the graylevel intensities of points inside foreground regions are changed to show the distance to the closest boundary from each point. The distance transform is sometimes very sensitive to small changes in the object. If, for example, we change the above rectangle to