Color2Gray: Salience-Preserving Color Removal

1. Color2Gray: Salience-Preserving Color Removal Amy Gooch Sven Olsen Jack Tumblin Bruce Gooch

2. Grayscale Color New Algorithm

3. Problem Volume of displayable CIE L*a*b* Colors

4. Isoluminant Colors Color Grayscale

5. Converting to Grayscale… • In Color Space • Linear • Nonlinear • In Image Space • Pixels (RGB) • Using colors in the image • Different gray for different color • Relative difference • Using colors in the image and their position in image space • Colors can map to same gray…..

6. Traditional Methods: Luminance Channels Problem can not be solved by simply switching to a different space CIE CAM 97 Photoshop LAB CIE XYZ YCrCb

7. Photoshop Grayscale Traditional Methods • Contrast enhancement & Gamma Correction • Doesn’t help with isoluminant values PSGray + Auto Contrast New Algorithm

8. Simple Linear Mapping Luminance Axis

9. Luminance Axis Principal Component Analysis (PCA)

10. Problem with PCA Worst case: Isoluminant Colorwheel

11. Non-linear mapping

12. Contemporaneous Research • Rasche et al. [2005, IEEE CG&A and EG] Color Image Luminance Only Rasche et al.'s Method

13. Goals • Dimensionality Reduction • From tristimulus values to single channel Loss of information • Maintain salient features in color image • Human perception

14. Relative differences Color Illusion by Lotto and Purves http://www.lottolab.org

15. Challenge 1:Influence of neighboring pixels

16. Challenge 2:Dimension and Size Reduction 120, 120 100 0 -120, -120

17. Challenge 3:Many Color2Gray Solutions Original . . .

18. color2gray For first pixel L1 + e1,2 L2 1 2 L1 + e1,2 L2 + e2,1 For Nth pixel L2 + e2,1 L1 L1 L1 L1 L2 L2 L2 Algorithm Intuition Look at DC . . . i = 1, j = 2 luminance Look at DC

19. Algorithm Overview • Convert to Perceptually Uniform Space • CIE L*a*b* • Adjust g to incorporate both luminance and chrominance differences • Initialize image, g, with L channel • For every pixel • Compute Luminance distance • Compute Chrominance distance dij

20. Color2Grey Algorithm Optimization: minSS ( (gi - gj) - di,j)2 i+m i j=i-m

21. Parameters Map chromatic difference to increases or decreases in luminance values m : Radius of neighboring pixels a : Max chrominance offset q :

22. m : Neighborhood Size • = 300o a = 10 m = 2 m = 16 • = entire image • = 49o a = 10

23. m : Neighborhood Size m = 16 • = entire image

24. a: Chromatic variation maps to luminance variation a -a crunch(x) = a * tanh(x/a) a = 5 a = 10 a = 25

25. Perceptual Distance DLij = Li - Lj • Luminance Distance: • Chrominance Distance: ||DCij|| Problem: ||DCij|| is unsigned

26. Map chromatic difference to increases or decreases in luminance values +b* C2 -a* +a* Color Space C1

27. +Db* Color Difference Space +DC1,2 vq= (cos q, sin q) + - vq q -Da* +Da* + - sign(||DCi,j||) = sign(DCi,j .vq ) -Db*

28. q = 45 q = 225 Photoshop Grayscale

29. q = 135 q = 45 q = 0 Grayscale

30. How to CombineChrominance and Luminance dij = DLij (Luminance)

31. How to CombineChrominance and Luminance (Luminance) if |DLij| > ||DCij|| DLij dij = ||DCij|| (Chrominance)

32. 120, 120 100 a -a crunch(x) = a * tanh(x/a) 0 -120, -120 How to CombineChrominance and Luminance DLij if |DLij| > crunch(||DCij||) d (a)ij = crunch(||DCij||)

33. Grayscale . . . How to CombineChrominance and Luminance DLij if |DLij| > crunch(||DCij||) d(a,q)ij = if DCij . nq ≥ 0 crunch(||DCij||) otherwise crunch(-||DCij||)

34. Color2Grey Algorithm Optimization: minSS ( (gi - gj) - di,j)2 i+m i j=i-m If dij == DL then ideal image is g Otherwise, selectively modulated by DCij

35. Color2Grey + Color Results Photoshop Grey Original Color2Grey

36. Original PhotoshopGrey Color2Grey Color2Grey+Color

37. Original PhotoshopGrey Color2Grey

38. Original PhotoshopGrey Color2Grey

39. Implementation Performance • Image of size S x S • O(m2 S2) or O(S4) for full neighborhood case • 12.7s 100x100 image • 65.6s 150x150 image • 204.0s 200x200 image • GPU implementation • O(S2) ideal, really O(S3) • 2.8s 100x100 • 9.7s 150x150 • 25.7s 200x200 Athlon 64 3200 CPU NVIDIA GeForce GT6800

40. Future Work • Faster • Multiscale • Animations/Video • Smarter • Remove need to specify q • New optimization function designed to match both signed and unsigned difference terms • Image complexity measures

41. Validate "Salience Preserving" Original PhotoshopGrey Color2Grey Apply Contrast Attention model by Ma and Zhang 2003

42. Validate "Salience Preserving" Original PhotoshopGrey Color2Grey

43. Thank you www.color2gray.info • SIGGRAPH Reviewers • NSF • Helen and Robert J. Piros Fellowship • Northwestern Graphics Group • MidGraph2004 Participants • especially Feng Liu • (sorry I spelled your name wrong in the acknowledgements)

45. Original Color2Grey Color2Grey+Color

46. Original Color2Grey Color2Grey+Color

47. Color2Grey Original Color2Grey+Color

48. Original PhotoshopGrey Color2Grey

49. Original Color2Grey Color2Grey+Color

50. Original PhotoshopGrey Color2Grey