Manipulation of pixel data

1. Manipulation of pixel data • Simple geometric transformations • Arithmetic combinations of images • done on a pixel-by-pixel basis • images must be the same size • Image addition and averaging • averaging multiple noisy images reduces the noise level • scene must be static • Image subtraction • used for motion detection and tracking • use a threshold to ignore insignificant changes

2. Pixel Values Each of the pixels that represents an image stored inside a computer has a pixel value which describes how bright that pixel is, and/or what color it should be. In the simplest case of binary images, the pixel value is a 1-bit number indicating either foreground or background. For a grayscale images, the pixel value is a single number that represents the brightness of the pixel. The most common pixel format is the byte image, where this number is stored as an 8-bit integer giving a range of possible values from 0 to 255. Typically zero is taken to be black, and 255 is taken to be white. Values in between make up the different shades of gray. To represent color images, separate red, green and blue components must be specified for each pixel (assuming an RGB colorspace), and so the pixel `value' is actually a vector of three numbers. Often the three different components are stored as three separate `grayscale' images known as color planes (one for each of red, green and blue), which have to be recombined when displaying or processing. Multi-spectral images can contain even more than three components for each pixel, and by extension these are stored in the same kind of way, as a vector pixel value, or as separate color planes.

3. Levels of computation We can consider image processing algorithms from the viewpoint of locality of computation. (levels of computation) Consider each algorithm is terms of its input-output characteristics. Here our aim is to characterize operations so that we can discuss the nature of input and output and how best to implement these operations . There is a variety of ways to classify and characterize image operations. The reason for doing so is to understand what type of results we might expect to achieve with a given type of operation or what might be the computational complexity associated with a given operation.

4. Operation Characterization Generic Complexity/Pixel * Point - the output value at a specific coordinate is dependent only on the input value at that same coordinate. constant * Local - the output value at a specific coordinate is dependent on the input values in the neighborhood of that same coordinate. P2 * Global - the output value at a specific coordinate is dependent on all the values in the input image. N2 Types of image operations The types of operations that can be applied to digital images to transform an input image A[m,n] into an output image B[m,n] (or another representation) can be classified into three categories as shown in Table 1.

5. Types of image operations Types of image operations. Image size = N x N; neighborhood size = P x P. Note that the complexity is specified in operations perpixel. This is shown graphically in Figure 1. Figure 1: Illustration of various types of image operations

6. Point level operations In a point operation, each output pixel corresponds directly to the input pixel having the same coordinates fB(x,y) = Opoint {fA(x,y)} where fA and fB are input and output images , respectively The simplest and most widely used segmentation method is thresholding. It consists of setting background values for pixels below a threshold value T and a different set values for the foreground. If the input image is f(x,y) and thresholded image is g(x,y), the equation of the thresholding operator is given by:

7. Shown below are the original image and the results after applying different thresholds values (118, 128) to it. Experiment - Thresholding

8. Local level operations - neighbourhood operations Local level operation produces an output image in which the intensity at a point depends on the neighborhood of the corresponding points in the input image fB(x,y) = Olocal {fA(xk,yl) ; (xk,yl) Nx,y)} Local operations produce an output pixel value b[m=mo,n=no] based upon the pixel values in the neighborhood of A[m=mo,n=no]. Some of the most common neighborhoods are the 4-connected neighborhood and the 8-connected neighborhood in the case of rectangular sampling and the 6-connected neighborhood in the case of hexagonal sampling illustrated in Figure 2.

9. Masks, filters, kernels, templates Figure 2a Figure 2b Figure 2c Rectangular sampling Rectangular sampling hexagonal sampling 4-connected 8-connected 6-connected

10. Smoothing and edge detection are local operations

11. Global leveloperations The output of certain operators depends on the whole image. A histogram of intensity values and Fourier transform are global operations. P = Oglobal{f(x,y)} Object Level - Size, average intensity, shape, and other characteristics of an object must be computed for the system to recognize this object.

12. Mathematics-based Operations • Binary operations • Arithmetic-based operations In the binary case there are two brightness values "0" and "1". In the ordinary case we begin with 2B brightness values or levels but the processing of the image can easily generate many more levels. For this reason many software systems provide 16 or 32 bit representations for pixel brightnesses in order to avoid problems with arithmetic overflow. • Arithmetic combinations of images • - done on a pixel-by-pixel basis • - images must be the same size

13. Operation Definition preferred data type ADD c = a + b integer SUB c = a - b integer MUL c = a * b integer or floating point DIV C = a / b floating point LOG c = log(a) floating point EXP c = exp(a) floating point SQRT c = sqrt(a) floating point TRIG. c = sin/cos/tan(a) floating point INVERT c = (2B - 1) - a integer Arithmetic-based operations The gray-value point operations that form the basis for image processing are based on ordinary mathematics and include:

14. Arithmetic-based operations Output images: Usually it will be necessary to pre-define the type or range of the resultant image. For example, if two 256 grey level images are multiplied together, the result for any pixel is highly likely to be outside the 0 to 255 range! Depending on the application, it might be preferable to define the result as a floating point number so that any value is recorded, or a new range of 0 to 255² (i.e. effectively 216 so that we would now have a 16-bit image) so that all the results are within the range. Alternatively a smaller range could be defined, and any results which were outside that range would be clipped, i.e. if the range is A to B, any pixel where the result of the operation was less than A would be given the value A and any pixel where the result was greater than B would be given the value B.

15. Arithmetic-based operations Image - Image Multiplication: This can only be done on two images which have the same dimensions. The value of each pixel in one image is multiplied by the value of the corresponding pixel in the other image. Image - Image Division: The values in the first image are divided by those in the second. Image - Image Addition: The values in the two images are added. Image - Image Subtraction: The values in the two images are added. Note that this should be used with care as it easily results in negative values which are undefined for a gray level image.

16. Pixel Addition -How It Works The addition of two images is performed straightforwardly in a single pass. The output pixel values are given by: Or if it is simply desired to add a constant value C to a single image then:

17. Arithmetic-based operations Image - Image Combining: This is similar to addition but proportions of the corresponding pixels of the two images are added together. For example, 0.6 of image 1 could be added to 0.4 of image 2. We may wish to give more emphasis to one image than the other. This can be done by 'alpha blending' g(x,y) = f1(x,y) + (1- ) f2(x,y) When  in this Equation is 0.5, g(x,y) becomes a simple, evenly-weighted average of the two input images. It is possible for a  to vary, every pixel of an image can have its own , stored in a separate ' alpha channel'.

18. Image Combining a)Image A; b)Image B c)0.5 * A + 0.5 * B

19. The main application of image averaging is noise removal

20. Subtraction of images The difference between two images f1(x,y) and f2(x,y) is obtained by computing the difference between all pairs of corresponding pixels from f1( x,y) and f2(x,y):  g(x,y) = f1(x,y) - f2(x,y) Subtracting two 8-bit greyscale images can produce values between -255 and +255. This necessitates the use of 16-bit signed integers in the output image - unless sign is unimportant, in which case we can simply take the modulus of the result and store it using 8-bit integer g(x,y) = f1(x,y) - f2(x,y) 

21. The main application for image subtraction is in change detection In the simples form, a binary difference image DP1,2(x,y) between frames f1(x,y) and f2(x,y) is obtained by: In a difference picture, pixels, which have value 1 may be considered to be the result of object motion or illumination changes.

22. The main application for image subtraction is in change detection Mask Image Live Image Digital Subtraction Angiography (DSA) Image

23. Binary operations Operations based on binary (Boolean) arithmetic form the basis for a powerful set of tools that will be described here and extended in mathematical morphology. The operations described below are point operations and thus admit a variety of efficient implementations including simple look-up tables. The standard notation for the basic set of binary operations is: The implication is that each operation is applied on a pixel-by-pixel basis.

24. Binary operations For example, . The definition of each operation is:

25. Binary operations These operations are illustrated in Figure where the binary value "1" is shown in black and the value "0" in white. a) Image a b) Image b c) NOT(b) = d) OR(a,b) = a + be) AND(a,b) = a * b f) XOR(a,b) = abg) SUB(a,b) = a \ b

26. Find more applications of point to point image operation