Scene illumination and surface albedo recovery via L1-norm total variation minimization

1. Scene illumination and surface albedo recovery via L1-norm total variation minimization Hong-Ming Chen hc2599@columbia.edu Advised by: John Wright

2. Decomposition of a scene = .* scene illumination Reflectance (albedo) .* : Matlab element multiplication operation

3. Image Formation Pixel i = .* signals integration illumination reflectance scene intensity response Sensor response Light source power spectrum Object reflectance Sensor response (camera or eyes) :shutter speed, aperture size, quantization factor etc

4. It is VERY HARD to directly model / simulate / solve this problem!

5. Narrowing down our target problem • Simplification: • mean wavelength response (impulse response) • Assumption (on surface reflectance): • Lambertian Surface (Perfect diffuse reflection, no specular light) • Simulation (of light source model): • We need a formula to describe the behavior of the light source • Blackbody radiation: parameterize the light source with: • Light color (color temperature) • Light intensity

6. Problem formulation:

7. Assume: λRλG λG are known If there are N pixels in an image: 3Nobservations 5N unknowns (I, T, ref ) + 3 quantize factors log underdetermined system!

9. Recovering unknown x • Previous approach • Introducing regularization terms into objective function • Current approach • Minimizing L1-norm total variation

10. Previous Approach A result of: Intrinsic images by entropy minimization , Finlayson, ECCV2004 s p 1-D grayscale visualization 255 ps A segmentation-like result pp 0

11. Drawbacks of this approach • There are at least 2 parameters (λ, σ) to be fine tuned. • The results of Finlayson’s approach heavily affects the accurateness of our prior. • 1. Its Achilles heel: projection problem • 2. it is still an open problem to find the best rotation angle.

12. (λ =50 , σ = 10) (λ =10 , σ = 30) (λ =120 , σ = 5) (λ =120 , σ = 8)

13. A brief review of Finlayson’ solution Its Achilles heel:

14. L1 norm Total Variation Minimization Image From Wikipedia

15. L1 norm Total Variation Minimization • Widely used in image denoise / Compressive sensing • E(x, y) + λTV(y). Image From Wikipedia

16. Current approach: L1 TV norm • Applying L1-norm total variation on albedo term, • The L1-norm encourages a spiky result on gradient • Which means: we want most of the albedo gradients are 0 unless necessary => when albedo changes

17. Results Original image Albedo (reflectance) image Light intensity image Light color (temperature) image

18. Results Original image Albedo (reflectance) image Light intensity image Light color (temperature) image

19. Results Original image Albedo (reflectance) image Light color (temperature) image

20. Results Original image Albedo (reflectance) image Light color (temperature) image

21. Editing Average T = 3940 Average T-1000 Original image Average T+1000 Average T+2000 Average T+3000 Average T+4000

22. THANK YOU