1 / 33

What do color changes reveal about an outdoor scene ?

Harvard University. Adobe Systems. What do color changes reveal about an outdoor scene ?. *. *. IEEE Conference on Computer Vision and Pattern Recognition 2008. Outdoor time-lapse. =. £. Intrinsic Images. [ Weiss 2001, Matsushita et al. 2003,… ]. Time-lapse sequence.

jacquelinedunn
Download Presentation

What do color changes reveal about an outdoor scene ?

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Harvard University Adobe Systems What do color changes reveal about an outdoor scene? * * IEEE Conference on Computer Vision and Pattern Recognition 2008

  2. Outdoor time-lapse

  3. = £ Intrinsic Images [ Weiss 2001, Matsushita et al. 2003,… ] Time-lapse sequence Time-varying Shading Texture Editing [ Sunkavalli et al. 2007 ] Geo-location [ Jacobs et al. 2007 ] Time-lapse in Computer Vision

  4. Coherence in time-lapse • Most previous work makes use of only the temporal structure in time-lapse. • Outdoor time-lapse also has a colorimetric structure. overcast sky clear sky sunset

  5. Outline • A physics-based model. • A method to fit the model to data. • Applications of the model.

  6. Outline • A physics-based model. • A method to fit the model to data. • Applications of the model.

  7. A model for outdoor time-lapse • Assumption 1 : Static scene

  8. A model for outdoor time-lapse • Assumption 2 : Linear Transforms for Re-illumination observed pixel values scene color under canonical lighting color transform ¸ ¸ ¸

  9. sky illumination sun illumination mixing weights A model for outdoor time-lapse • Assumption 3 : Ambient sky and direct sun illumination

  10. A model for outdoor time-lapse • Assumption 4 : 2-d subspace for daylight spectra [ Judd et al. 1964, Sastri and Das 1968, …, Hernández-Andrés et al. 2000 ]

  11. A model for outdoor time-lapse • Assumption 4 : 2-d subspace for daylight spectra

  12. A model for outdoor time-lapse illumination coefficients

  13. A model for outdoor time-lapse scene color under canonical lighting (normalized)

  14. A model for outdoor time-lapse

  15. A model for outdoor time-lapse shadows shadows

  16. A model for outdoor time-lapse shading

  17. A model for outdoor time-lapse 1-d projection of normals onto solar plane

  18. Outline • A physics-based model. • A method to fit the model to data. • Applications of the model.

  19. illum. bases illum. coeffs. angular velocity albedo shadows brightness normals Fitting the model over-constrained system 320 X 240 images, 100 frames 23,040,000 measurements 8,141,220 parameters

  20. Fitting the model 1. Estimate albedo and illumination bases 2. Estimate shadows 3. Estimate remaining parameters Reconstruct

  21. Fitting results original reconstruction 240 x 360 images, 120 frames, ~5 mins. interval, 7.36% RMS error

  22. Fitting results original error x 3 240 x 360 images, 120 frames, ~5 mins. interval, 7.36% RMS error

  23. Outline • A physics-based model. • A method to fit the model to data. • Applications of the model.

  24. Application I : Color Constancy [ D’Zmura and Lennie 1986, Forsyth 1990, Funt et al. 1991, Finlayson and Funt 1994, Brainard and Freeman 1997, Barnard et al. 1997, Finlayson et al. 2001, Ebner 2004,… ]

  25. Application I : Color Constancy original color corrected

  26. Application II : Scene Reconstruction

  27. Application II : Scene Reconstruction

  28. Application II : Scene Reconstruction

  29. Application III : Geo-location

  30. Application III : Geo-location

  31. Summary • Outdoor time-lapse sequences are ubiquitous. • They are a rich source of scene information. • By using both temporal and colorimetric coherence we can access that scene information.

  32. Future Work • Robust Fitting (mutual illumination, non-lambertian surfaces, foreground clutter). • Multiple viewpoints. • Estimating weather and atmospheric conditions.

  33. Thank you!

More Related