Pyramid-Based Texture Analysis / Synthesis

1. Pyramid-Based Texture Analysis / Synthesis Presented By : Keren Kahanov

2. Texture Synthesis • Generation of much texture as desired so that any object can be covered • Input texture Output size Output texture • The output texture is synthesized image that match the texture appearance of the given digitized image

3. General Techniques • Texture mapping– repeating the texture • Can lead to unacceptable artifacts in visible seams , visible repetition • Distortion in map to 3-D – often no natural map from planar texture image to the geometry/ topology of the surface. (There are some partial solutions , but no universal solution for mapping an image onto arbitrarily shaped surface.)

4. General Techniques (cont.) • Create (paint) textures by hand directly onto the 3-d surface model • Very labor intensive • Requires considerable artistic skills • Computer-synthesized textures • as much texture can generated as needed • Some of the synthesis techniques produce textures that tile seamlessly.

5. Synthetic textures • The distortion problem has been solved in two different ways: • Some techniques work by synthesizing texture directly on the object surface. • The second solution is to use solid textures (no mapping no distortion)

6. Heeger & BergenPyramid-Based TextureAnalysis / Synthesis • Our lecture today • A technique for synthesizing an image (or solid texture) that matches the appearance of a given texture sample. • The output image (at any size) looks like the original image in its color and texture properties. • The texture is synthesized so that it tiles seamlessly.

7. Heeger & Bergen (cont.) • First order statistics (histograms) • Focuses on the synthesis of stochastic textures • deterministic textures: characterized by a set of primitives and a placement rule (e.g., a tile floor ,bricks). • stochastic textures : do not have easily identifiable primitives (e.g., granite, bark, sand). • Many real-world textures have some mixture of these two (e.g. woven fabric, plowed fields).

8. Heeger & Bergen’s Algorithm • The texture-matching algorithm relies on two main components: • a self-inverting steerable pyramid • a histogram matching process.

9. Pyramids - Reminder • Reminder from “image processing” course by Dr. Hagit Hel-Or

10. Pyramids – Reminder (cont.)

11. (1) Steerable Pyramid • a linear multi-scale, multi-orientation image decomposition • developed in order to overcome the limitations of orthogonal separable wavelet decompositions • The steerable pyramid, unlike wavelet, is non-orthogonal and overcomplete • Overcomplete = the number of pixels in the pyramid is much greater than the number of pixels in the input image

12. Steerable Pyramid (cont.) • The basis functions of the steerable pyramid are directional derivative operators, that come in different sizes and orientations • translation-invariant (i.e., the subbands are aliasing-free, or equivariant with respect to translation) • Translating the image the transform translates energy in each band is preserved (not in wavelet)

13. Aliasing • In steerable pyramid only the low-pass band is subsampled • This is done to minimize the amount of aliasing within each subband. Avoiding aliasing is critical because the algorithm treats each subband independently. Aliasing Properly sampled

14. Steerable Pyramid (cont.) • rotation-invariant = steerable (i.e., the subbands are steerable, or equivariant with respect to rotation) • rotating the image the transform rotates • A set of filters form a steerable basis if (1) they are rotated copies of each other, and (2) a copy of the filter at any orientation may be computed as a linear combination of the basis

15. Steerable filters example 2nd derivative of Gaussian Linear combination filters: Original Image: Same Linear combination outputs:

16. Steerable Pyramid (cont.) • A self-inverting pyramid: • it provides a complete representation of an image • original image can be reconstructed by applying the same filters used to generate the pyramid • A steerable pyramid can capture the variation of a texture in both intensity and orientation

17. Originalimage Steerable Pyramid Build • Needs 7 filters:

18. Steerable Pyramid Build (cont.) Originalimage 4 oriented filters The steerable pyramid

19. Steerable Pyramid- Fourier domain • The steerable pyramid performs a polar-separable decomposition in the frequency domain, thus allowing independent representation of scale and orientation 1 43 2 1 4 2 3 3 2 4 1 wavelet

20. (2) Histogram Matching • A generalization of histogram equalization. • modify an image (im1) to have the same histogram as another image (im2). • Using two lookup tables: • the cumulative distribution function (cdf) of one image. • The inverse cumulative distribution function (inverse cdf) of the other image

21. Cdf / inverse Cdf • The cdf • maps from the interval [0,256] to the interval [0,1]. (cdf = accumulated histogram) • The inverse cdf • maps back from [0,1] to [0,256]. • constructed by resampling (with linear interpolation) the cdf so that its samples are evenly spaced on the [0,1] interval. • Note! the values in the noise and target images need not lie in the range[0,256]

22. Histogram Matching Algorithm

23. התאמת ההיסטוגרמה של תמונה זו לזו של תמונה זו Histogram matched תוצאה סופית

25. Image 1 Image 2 Image 1 histogram matched to image 2 histogram

26. Enough with all the talks about Pyramids & Histograms !!! • It’s time to learn the real thing – Heeger & Bergen’s AlgorithmforTexture Analysis / Synthesis

27. Heeger & Bergen’s Algorithm

28. TheAlgorithm - steps • Input1 = sample texture image • Input2 = sample of uniform random noise ( noise image) • Its dimensions = desired size of output image • Match noise image’s histogram to the input sample’s histogram. ( modified noise image) • The texture sample is decomposed into an oriented bandpass pyramid

29. TheAlgorithm – steps (cont.) • Loop for several iterations: • Make pyramid from the modified noise image • Histogram matching process is employed between corresponding subbands of the two pyramids. • The noise pyramid is collapsed • The histogram matching process is again applied to the new noise image and the texture sample After a few iterations, the output image, which began as random noise, begins to resemble the input texture.

30. When are two textures similar? Textures appear visually similar if… they have similar distributions of filter outputs*. (Filters : orientation and spatial-frequency) FROM: “Statistical TextureSynthesis” Presented by:Zeev Litichever & Tal Hassner , Weizmann Institute

31. Laplassian \ steerable pyramid FROM: “Statistical TextureSynthesis” Presented by:Zeev Litichever & Tal Hassner , Weizmann Institute

32. Using histograms for synthesis Output image with similar distributions to sample. Iteratively coarse noise image to have sample histogram via histogram matching. Output pyramid Noise image of output size: Apply filters And create histograms FROM: “Statistical TextureSynthesis” Presented by:Zeev Litichever & Tal Hassner , Weizmann Institute

33. Results Input1 = texture image Input2 = noise image Output

34. More Results • In each pair : left= input (original) , right =output (synthesized)

35. Edge Handling • Proper edge handling in the convolution operations is important. • synthesis pyramid: use circular convolution. Given that the synthesis starts with a random noise image, circular convolution guarantees that the resulting synthetic texture will tile seamlessly. (synthesis pyramid = noise pyramid)

36. Edge Handling (cont.) • analysis pyramid: circular convolution would typically result in spuriously large filter responses at the image borders. This would, in turn, introduce artifacts in the synthesized texture. (analysis pyramid = original image pyramid) • A reasonable border handler for the analysispyramid is to pad the image with a reflected copy of itself. Reflecting at the border usually avoids spurious responses.

37. Color • Heeger and Bergen’s Algorithm can be applied for both : • gray level texture images • color texture images

38. G R Color Space RGB • RGB components of a typical image are not independent • cannot directly apply the algorithm to each channel individually. correlation Looking for B ? B is on a vacation , in another dimension (helped to simplify the graph)

39. Color Space Transform • First, transform the color space of the input texture to independent channels SVD algorithm : The decorrelation transform is :

40. R Color Space Transform (cont.) Diagonalmatrix G No correlation in transformed color space

41. New Color Space • each independent channel can be processed separately. • After processing, the channels are transformed back into RGB values, using yielding the synthesized texture.

42. Solid Textures • The algorithm can also be used to make isotropic 3-d solid textures. • Same as before, except for the choice of pyramid: use a 2-d Laplacian pyramid for analysis and a 3-d Laplacian pyramid for synthesis.

43. Solid Textures - examples

44. Texture Mixtures • Texture mixtures synthesized by choosing the color palette from one image and the pattern from a second image.

45. Texture Mixtures (cont.) • tiling different textures as input to Heeger and Bergen’s algorithm • Output: combine the properties of multiple textures.

46. Algorithm – no. iterations • An Implementation of Heeger and Bergen’s Texture Analysis/Synthesis Algorithm by Thomas F. El-Maraghi • The human observerwould find it difficult, or even impossible, to discern any change after the 4th iteration.

47. No. of orientation • The effect of changing the number of oriented subbandsin the pyramid on the synthesized texture. • the synthesized texture looks more like the original as the number of orientations is increased. original 1 orientation 2 orientations 4orientations

48. Homogenous / Inhomogoneousinput • Homogoneous input texture success • Inhomogoneous input texture produces blotchy synthetic texture failure

49. Failures

50. More Failures