Image Processing

1. Image Processing Rule 1: Always save your primary data Rule 2: Be able to describe quantitatively what you have done

2. 3 types of operations • Point operations: operate on a pixel-by-pixel basis • Neighborhood operations: operate on a small group of adjacent pizels • Reciprocal space operations: deal with image-wide patterns and chartacteristics

3. Point operations (Black-and-White) http://homepages.inf.ed.ac.uk/rbf/HIPR2/pntops.htm Thresholding Gamma Curves Histogram Equalization

4. Point Operations (Color) • Color maps • RGB (More common in scientific imaging) • CMYK (Printers, etc) • Color Balance • HSV

5. Shading correction • Two sources of shading variation in images • Dye binding to background: Image subtraction is appropriate • Camera/light source nonuniformity: Image division is appropriate • Image subtraction • Image division: Divide data image by blank image and normalize • If you don’t have a blank image: erode features and smooth to derive background (“flatten image” in Image Pro Plus)

6. Geometric correction • Geometric distortion: pincushion and barrel distortion • Geometric distortion: trapezoidal distortion • Tiepoints: set of points with known geometric relationships to each other • Set up a matrix of actual geometric positions in the image as a function of pixel coordinates • Interpolate beweennonintegrap pixels positions to get square pixels.

7. Neigborhood operations – allow feature extraction • Convolution operator: a matrix that applies a kernel (say 3X3) to every point in the image • Replaces the central point by the resultant of multiplying that 3X3 matrix by neighboring pixels Molecular Expressions Web Site

8. Averaging kernel • Replace central point with average of neighborhood Most software packages do the normalization automatically, so you can use “1”’s instead of “1/9”’s

9. Smoothing kernel • Gaussian (3X3) • (5x5)

10. Sharpening kernels • Laplacian Approximates a Laplacian operator, which replaces the central value with the differential in x and y

11. Directional kernels • Vertical edge • Diagonal edge Average in vertical direction Difference in horizontal direction

12. Complex neighborhood operations • Median filter: replace central pixel with median of neighborhood • Very effective at removing “shot noise” • Roberts cross: 2 perpendicular directional filter • Sobel: • Calculate derivatives in 2 perpendicular directions • Replace central magnitude with √ ((δB/ δx)2 + (δB/ δy)2 ) • Kirsch: Apply each of 8 directional filters, and replace central value with maximum

13. Complex neighborhood operations • Olympic filter: in each 5X5 neighborhood. Ignote the brightest and darkest 4. Replace the central value with the average of the remaining 17 • Top hat: replace values greater than the average of a neighborhood by the average for that neigborhood • Gray scale opening • First pass: replace central pixel with brightest neighbor • Second pass: replace pixel with darkest neigbor • Net effect: dilation of dark features, and erosion of bright

14. Hybrid – sharpening by difference of Gaussians (unsharp mask) • Apply 2 different size Gaussians to same image • Subtract smaller from larger Gaussian filtered result • Unsharp masking • Photographically: • Image in and out of focus • Invert out-of-focus • Mat reversed image with in-focus • Digitally: subtract blurred from unblurred • http://micro.magnet.fsu.edu/primer/java/digitalimaging/processing/unsharpmask/index.html

15. Character recognition • Instead of regular convolution masks, use masks that represent characters in the image • You get a “hit”, or high match, whenever the mask matches the character! • However, the characters must ba aligned, undistorted, etc.

16. Automatic number plate recognition Wikipedia: Automatic number plate recognition

17. Algorithms for ANPR • There are six primary algorithms that the software requires for identifying a licence plate: • Plate localisation – responsible for finding and isolating the plate on the picture • Plate orientation and sizing – compensates for the skew of the plate and adjusts the dimensions to the required size • Normalisation – adjusts the brightness and contrast of the image • Character segmentation – finds the individual characters on the plates • Optical character recognition • Syntactical/Geometrical analysis – check characters and positions against country specific rules • The complexity of each of these subsections of the program determines the accuracy of the system. During the third phase (normalisation) some systems use edge detection techniques to increase the picture difference between the letters and the plate backing. A median filter may also be used to reduce the visual "noise" on the image. -ibid

18. General object recognition • How do we recognize specific objects (such as tanks in aerial images) using machine vision • Problem of orientation: any orientation may present itself • First, scan image with circularly averaged structures • Then, scan again with specific orietations • Highly computationally expensive, and not terribly effective • We can often do better with Fourier transform techniques.

19. Fourier Transform Image Processing • Any periodic object can be represented by a summation of a series of cosine waves • The Operation of Fourier transformation of an image replaces the image (real space) be a series of amplitudes and frequencies of the cosine waves that make it up • Fourier space is also referred to as frequency space • If there are repeats in the stucture at specific frequencies, these will appear as peaks in Fourier space

20. Fourier Transform Image Processing • High- and low-pass filters • By enhancing or supressing specific frequencies, we can enhance or suppress periodic structures within the image Molecular Expressions Java simulation

21. Examples • Nuclear pore complex • Markham rotation • Fourier transform • Removal of halftone screen noise

22. Dangers of Fourier transforms • Can introduce periodicities where none are present • Edge effects