Automatic Color Gamut Calibration

1. Automatic Color Gamut Calibration Cristobal Alvarez-Russell Michael Novitzky Phillip Marks

2. Inspiration • G. Klein and D. Murray, Compositing for Small Cameras, ISMAR'08

3. Motivation • Calibrate and compensate for: • Color distortions of a small video camera • Lighting conditions of environment • Purpose: • Augmented Reality • Matching the color gamut of virtual objects to video camera image • Robotics • Calibrating a video camera for particle-filter-based object tracking (i.e. orange ball in robot soccer)

4. Approach • GretagMacbeth ColorChart • Diffuse material • Color samples under daylight • RGB values are known

5. Approach (cont.) • Start with picture of a scene with the chart • Locate the squares of the chart in the image • Unproject and crop the chart • Sample the colors in the chart • Adjust the color of the entire image (and subsequent ones)

6. Locating the Chart • Failed Attempts • Swain’s Histogram Back-projection • Color constancy a big problem • Tried color constancy approximations • Not good for color chart • Too many histogram matches -> false positives • Only returned a point within the square • We hoped it would be an estimation of the center of the chart • No information useful for unprojection

7. Locating the Chart (cont.) • Original image

8. Locating the Chart (cont.) • Color constancy • Color normalization

9. Locating the Chart (cont.) • False positives • Ratios high because of wide chart histogram

10. Locating the Chart (cont.) • Result not useful for feature extraction • Not a good estimate of the center of the chart

11. Locating the Chart (cont.)

12. Locating the Chart (cont.) • Current approach • First step: User interface • User clicks and labels squares • Flood fill • Uses histogram • Create screen-aligned bounding box

13. Locating the Chart (cont.)

14. Locating the Chart (cont.)

15. Locating the Chart (cont.)

16. Locating the Chart (cont.) • Second step: Connected components • Sweep through the image • Label neighboring pixels that are activated • Choose the connected component with the highest vote

17. Locating the Chart (cont.)

18. Locating the Chart (cont.)

19. Locating the Chart (cont.) • Third step: Recognize regions • We need to find the corners of the region within the bounding box • First attempt: Draw lines from bounding box corners and vote on likelihood of region edge • Failed! • Second attempt: Look for region corner iteratively from bounding box corner • Success!

20. Locating the Chart (cont.)

21. Locating the Chart (cont.)

22. Locating the Chart (cont.)

23. Locating the Chart (cont.)

24. Unprojecting the chart • Start with corners of some color regions • Construct a matrix A composed of image and world point correspondences • Compute homography matrix from null space of A • SVD to compute it • Use inverse homography to unproject each pixel

25. Unprojecting the chart (cont.) • Problems: • OpenCV matrices are not good for numerical methods • Switched to GSL • Noise in region corner positions • Remove smallest eigenvalue of singular matrix • Squares in the middle of the chart better

26. Unprojecting the chart (cont.)

27. Unprojecting the chart (cont.)

28. Sampling the chart • We sample at square centers • Squares centers estimated by predefined, specific ratios of the chart • We assume the homography and the unprojection are good enough • Stochastic sampling • We average several samples to reduce noise influence

29. Sampling the chart (cont.)

30. Sampling the chart (cont.)

31. Sampling the chart (cont.)

32. Adjusting the color gamut • Step 1: Adjust white balance of the samples • Simple linear scale • Using White 9.5 and Black 32 from color chart • Both in chart in image and known RGB values

33. Adjusting the color gamut (cont.)

34. Adjusting the color gamut (cont.)

35. Adjusting the color gamut (cont.) • Step 2: Adjust chromaticity • Use color samples as a distribution • Linear scale of every pixel color according to mean and standard deviation of distribution • Color samples from chart do not map to themselves • Approach 1: Marginal Distribution • Three 1D distributions (one per channel) • Treat channels independently from each other • Approach 2: Joint Distribution • Treat colors as 3D points in RGB cube • Standard deviation is a 3D distance from the mean color

36. Adjusting the color gamut (cont.)

37. Adjusting the color gamut (cont.)

38. Adjusting the color gamut (cont.)

39. Future Work • Locating the color chart • Use SIFT-like descriptors with point matching according to the color chart structure • Use grid-pattern algorithms like the ones used in fiducial-based tracking (i.e. ARToolkit) • Chart unprojection • Try iterative homography estimation • Color gamut adjustment: • Interpolate colors using a tetrahedral mesh • Try using color spaces that separate chromaticity from intensity (HSV, YUV, etc.)

40. The end!

41. Another example

42. Another example (cont.)

43. Another example (cont.)

44. Another example (cont.)

45. Another example (cont.)

46. Another example (cont.)

47. Another example (cont.)

48. Another example (cont.)

49. Another example (cont.)