Evolutionary Computation and Image Re-Coloring

1. Evolutionary Computation and Image Re-Coloring Gary R. Greenfield Mathematics & Computer Science University of Richmond Computational Aesthetics in Graphics, Visualization, and Imaging Dagstuhl Seminar, 2006

2. Motivation • Is there a role for evolutionary computation (EC) in computational aesthetics? • Example: Image Re-coloring.

3. Background (1 of 4) • E. Reinhard, M. Ashikhmin, B. Gooch, P. Shirley, Color transfer between images, IEEE CG&A, 2001, 34-41. • B. Meier, A. Spalter, D. Karelitz, Interactive color palette tools, IEEE CG&A, 2004, 64-72. • G. Greenfield, D. House, Palette-driven color transfer, Computational Aesthetics 2005, 91-99. • M. Grundland, N. Dodgson, Color search and replace, Computational Aesthetics 2005, 100-109.

4. Background (2 of 4) • L. Neumann, A. Neumann, Color style transfer techniques using hue, lightness and saturation histogram matching, Computational Aesthetics 2005, 111-122. • A. Hertzmann, C. Jacobs, N. Oliver, B. Curless, D. Salesin, Image Analogies, Proc. SIGGRAPH ’01, 2001, 327-340. • T. Welsh, M. Ashikhmin, K. Mueller, Transferring color to grayscale images, Proc. SIGGRAPH ’02, 2002, 277-280. • Y. Chang, S. Saito, M. Nakajima, A framework for transfer colors based on the basic color categories, Proc. Computer Graphics International, 2003, 176-181.

5. Background (3 of 4) • L. Neumann, A. Nemcsics, A. Neumann, Computational color harmony based on Coloroid system, Computational Aesthetics 2005, 231-240. • A. Levin, D. Lischinski, Y. Weiss, Colorization using optimization, Proc. SIGGRAPH ’04, 2004, 689-694. • R. Irony, D. Cohen-Or, D. Lischinski, Colorization by example, Rendering Techniques, 2005, 201-210. • A. Gooch, S. Olsen, J. Tumblin, B. Gooch, Color2Gray: salience preserving color removal, Proc. SIGGRAPH ’05, 2005, 634-639.

6. Background (4 of 4) • G. Greenfield, An algorithmic palette tool, UR Technical Report, 1994. • J. Barallo, V. de Spinadel, Colouring algorithms and fractal art, J. of Math. and Design, 2001, 27-32. • C. Farsi, C. Collins, Visual synthesis and esthetic values, Proc. Generative Arts Conference, 2004, 40-45. • G. Greenfield, Image re-coloring using multi-objective optimization, Proc. Art+Math=X Conference, 2005, 83-87.

7. Color Look-Up Tables • Fix a “small” set S = {C1,…,CM} consisting of M <HSV | RGB | YIQ |…> colors. • A color look-up table (CLUT) of length L is a list T = (c1,…,cL) of not necessarily distinct colorschosen from S. • The image I can be re-colored by T if each pixel P can be identified with some integer I such that 1 <= I <= L (viz. re-color P using color cI).

8. Evolving CLUT’s • One way to evolve CLUTs is by randomly initializing CLUT segments and then using crossover as the recombination operator coupled with mutation operators such as: • “Flow” • “Extend” • “Copy” • “Blot”

9. An Evaluation Framework • Unfortunately, the problem of how to evaluate the aesthetic quality of a CLUT re-coloring still remains. • The following re-colorings were done by applying evolutionary multi-objective optimization (EMO) using certain color and geometric characteristics of low res segmented versions of the re-colored images. For example, F1(I) = A2,6 Jm/2,m + C4,5 F2(I) =min(T1,4.2,T2,3.7) B1,4

10. Re-Coloring Results

17. The Rogue’s Gallery

21. Lessons Learned • The fewer colors that are required, the more successful the re-coloring will be. • Images with many large, juxtaposed, and differently colored regions “confuse” the re-coloring algorithm. • Luminance criteria should be taken into consideration.

22. Other Ideas and Suggestions • Introduce penalty terms for prohibiting color neighbor mismatches in the re-coloring. • Define CLUT’s as discretizations of continuous curves (e.g. evolve the spline points for such curves). • Adopt a SETI Approach – Use on-line voting schemes to evolve re-colorings… and learn principles and preferences?!