Java 101: Imaging & Java

1. Java 101:Imaging & Java Ryan Cuprak

2. Java Media APIs • Java2D • Java Advanced Imaging (JAI) • Java Media Framework (JMF) • Java3D

3. Agenda • Digital imaging basics • Java Imaging Foundation • Image I/O • Java Advanced Imaging

4. Java Imaging Foundation

5. Java Imaging Foundation • Digital Image representation • Alpha channel • Transformations • Interpolation • Filters • Convolution kernels • Smoothing filters • Edge detectors • Sharpening

6. Digital Image Representation x Sample 1 Sample 3 y Sample 2 Alpha RGB Grey Scale

7. Java Imaging Models • Push Model (AWT) • Designed for loading images into Applets • Image is rendered as data becomes available • Immediate Mode (Java2D) • Image is loaded into memory • Variety of Filters • Pull Model

8. Processing Images in AWT ImageProducer PixelGrabber MemoryImageSource MemoryImageSource

9. java.awt.image • Image is an abstract class • Image objects are constructed in a platform specific manner • Two types of image objects: • Images created from an image source • Images created by an AWT component • Component.createImage()

10. BufferedImage java.awt.image.BufferedImage Color Model Raster ColorSpace SampleModel DataBuffer Encapsulates the methods for translating a pixel value to color components (ex. red/green/blue) and an alpha component for rendering

11. DataBuffer • java.awt.image.DataBuffer • DataBuffer stores pixel data as one or more arrays of primitive data types. • Java2D buffer types: • Byte • Integer • Short • Unsigned short • JAI buffer types: • Double • Float

12. Sample Model • SampleMode describes how pixels are stored in the DataBuffer. • Packed Models • Single-pixel packed model • Multiple-pixel packed sample model • Component Models • Component sample model

13. ColorModel • Converts raster pixel samples into color components for displaying • Pixel samples are packed into a short integer (16 bits). Each sample would be 5 bits. • If displayed on a color monitor (0-255) per channel, the resulting image would appear dark. • Color model handles the conversion.

14. ColorSpaces • Device independent/dependent • Converts between devices • Supports many color spaces including: • CMYK • HSV (HSB) • CIEXYZ • sRGB • ICC Color Profile • Create ColorSpace via factory

15. Code Example Creating an Image

16. Alpha Channel • A mixing factor to control the linear interpolation of foreground and background colors. • Varies from 0 to 1 • 0 indicates no coverage (clear) • 1 indicates full coverage • Each sample is multiplied by the alpha channel

17. Alpha Channel

18. Transformations • Definitions: • Transformation is a function that maps an object from one point to another. • Affine transform preserves: • Finiteness • Parallelism • Transformations: • Translation • Scaling • Rotation • Reflection • Shear

19. Translation y x

20. Scaling y x

21. Rotation y x

22. Reflection y x

23. Shear y x

24. Transformations • Transformations can be concatenated: • C = Translate * Scale * Rotate * Shear • Java Classes: • AffineTransform ( java.awt.geom ) • AffineTransformOp ( java.awt.image )

25. Interpolation • What is Interpolation? • Interpolation is a process of generating a value of a pixel based on its neighbors. • Types of Interpolation: • Nearest-neighbor • Linear • Bilinear • Trilinear • Cubic

26. Interpolation - Linear d Interpolated Pixel px P(x+1) Px’ P(x+1) Px’ px x x’ x+1 Building Imaging Applications with Java p272

27. Bilinear Interpolation P(x,y) P(x+1,y) dy dx P(x’,y’) P(x+1,y+1) P(x,y+1)

28. Trilinear Interpolation P(x,y,z+1) P(x,y,z) dy dx P(x’,y’,z’) dz P(x,y+1,z)

29. px px+1 px’ px+2 px+3 px+2 px+3 px’ px+1 Pixel value px x x+1 x’ x+2 x+3 Pixel location Cubic Interpolation px = ax3 + bx2 + cx + d

30. Convolution Kernels Image I(i,j) Convolution kernel is a matrix f(m,n) Example: f(m,n) =

31. Convolution Kernel Example I(3,3) = (1*5) + (1*17) + (1*6) + (2*11) + (1*11) + (1*1) + (1*5) + (1*7) = 74 f(m,n)=

32. Smoothing Filters Weighted average of the neighboring pixels. Often approximate a Gaussian function.

33. Edge Detectors Laplacian

34. Sharpening

35. Java2D Image Operations • AffineTransformOp • ConvolveOp • LookupOp • RescaleOp • ColorConvertOp • BandCombineOp (RasterOp)

36. Code Example Interpolation/Edge Detection

37. Image I/O “The Java Image I/O API provides a pluggable architecture for working with images stored in files and accessed across the network. It offers substantially more flexibility and power than the previously available APIs for loading and saving images.”

38. Image I/O Key Features • Read & write multiple image formats • JPG, PNG, GIF, etc. • Expose image metadata uniformly • W3C DOM • Provide Access to thumbnail images • Support multi-image files • Provide pluggable architecture for additional formats • Handle large image files

39. Image I/O Packages • javax.imageio • ImageIO, ImageReader, ImageWriter • javax.imageio.event • javax.imageio.metadata • IIOMetadata, IIOMetadataFormatImpl, IIOMetadataNode • javax.imageio.spi • Javax.imageio.stream

40. Custom Plug-ins • Implement: • Reader - extends ImageReader • Writer - extends ImageWriter • Provider interfaces: • ImageReaderSpi • ImageWriterSpi • Optionally provide • Image and stream metadata classes extending IIOMetadata

41. Image I/O Code Examples • Reading & writing images • Querying for supported image types • Implementing a plug-in

42. Java Advanced Imaging

43. Uses of JAI • Defense and intelligence • Geospatial data processing • Document image processing • Bioinformatics applied research • Digital photography • E-commerce and retail (from java.sun.com)

44. JAI Concepts • Multi-tiled images • Deferred execution • Network images • Image property management • Image operators with multiple sources • Three-dimensional image data

45. Image Processing DAGs constant fileload add rotate Display Widgets

46. JAI Operator Categories • Point Operators • Area Operators • Geometric Operators • Color Quantization Operators • File Operators • Frequency Operators • Statistical Operators • Edge Extraction Operators • Miscellaneous Operators

47. Point Operators

48. Area Operators • Border • BoxFilter • Convolve • Crop • MedianFilter

49. Geometric Operators • Affine • Rotate • Scale • Shear • Translate • Transpose • Warp

50. Color Quantization • ErrorDiffusion • OrderedDither