1 / 49

Inferring Reflectance Functions from Wavelet Noise

This paper explores the estimation of reflectance functions from wavelet noise, using techniques such as image-based relighting, environment matting, and incident illumination. It discusses examples of reflectance functions, methods for computing relit images, and the direct observation of reflectance functions through controlled incident illumination.

warden
Download Presentation

Inferring Reflectance Functions from Wavelet Noise

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Inferring Reflectance Functionsfrom Wavelet Noise June 30th 2005 Pieter Peers Philip Dutré Department of Computer Science

  2. Image-based Relighting / Environment Matting Scene(fixed viewpoint)

  3. Image-based Relighting / Environment Matting + Scene(fixed viewpoint) … Novel Incident Illumination

  4. Image-based Relighting / Environment Matting + = Scene(fixed viewpoint) … … Compute Relit Image Novel Incident Illumination

  5. Image-based Relighting / Environment Matting + = ReflectanceFunction Scene(fixed viewpoint) … … Compute Relit Image Novel Incident Illumination

  6. Examples of Reflectance Functions Specular Ball Diffuse Ball

  7. Examples of Reflectance Functions Specular Ball Diffuse Ball

  8. Examples of Reflectance Functions Specular Ball Diffuse Ball Reflectance Function Reflectance Function

  9. Reflectance Functions (frequency domain) Specular Ball Diffuse Ball Reflectance Function (frequency domain) Reflectance Function (frequency domain)

  10. Reflectance Functions (wavelet domain) Specular Ball Diffuse Ball Reflectance Function (wavelet domain) Reflectance Function (wavelet domain)

  11. Relight a Pixel Relit pixel value? Specular Ball Novel Incident Illumination Reflectance Function (wavelet space)

  12. Relight a Pixel Specular Ball Novel Incident Illumination Reflectance Function (wavelet space) Incident Illumination (wavelet space)

  13. Relight a Pixel Specular Ball Novel Incident Illumination ) (  Reflectance Function (wavelet space) Incident Illumination (wavelet space)

  14. Relight a Pixel Specular Ball Novel Incident Illumination Onlynon-zero coefficients ) (  Reflectance Function (wavelet space) Incident Illumination (wavelet space)

  15. Directly Observing Reflectance Functions Photograph of Specular Ball Controlled Incident Illumination Emit(e.g. from CRT)

  16. Directly Observing Reflectance Functions Observed pixel Photograph of Specular Ball Controlled Incident Illumination ReflectanceFunction(unknown) Controlled Incident Illumination (wavelet space)

  17. Directly Observing Reflectance Functions Photograph of Specular Ball Controlled Incident Illumination ) (  Unknown Reflectance Function (wavelet space) Controlled Incident Illumination (wavelet space)

  18. Directly Observing Reflectance Functions Observed coefficient Photograph of Specular Ball Controlled Incident Illumination Onlynon-zero coefficients ) (  Unknown Reflectance Function (wavelet space) Controlled Incident Illumination (wavelet space)

  19. Number of Observations Specular Ball Incident Illumination #Photographs=#Illumination pixels Reflectance Function (wavelet space)

  20. Number of Observations Problem 1000x1000 Specular Ball Incident Illumination #Photographs=#Illumination pixels Reflectance Function (wavelet space)

  21. Wavelet Noise Illumination • Wavelet Noise • Normal distribution of wavelet coefficients • Mean : 0.0 • Standard deviation : 1.0 • Rescale Wavelet Noise Pattern to fit into [0..1] range Wavelet Noise Pattern • Advantages • Arbitrary number of different patterns possible • Any reflectance function gives a non-zero response • Constant average luminance Wavelet Noise Pattern (wavelet space)

  22. Estimating Wavelet Coefficients Assume: positions of are knownQuestion: what are the magnitudes? ) =  ( Observed Pixel Value (Unknown)Reflectance Function Wavelet Noise

  23. Estimating Wavelet Coefficients )  = ( Wavelet Noise (linearized) Observed Pixel Value Reflectance Function(linearized) Leave out zero coefficients(of the reflectance function)

  24. Estimating Wavelet Coefficients … …  = # observations # emitted patterns Observed PixelValues Wavelet Noise Reflectance Function Multiple observations matrix-vector multiplication

  25. Estimating Wavelet Coefficients … …  = Observed PixelValues Wavelet Noise Reflectance Function Finding magnitudes : Linear Least Squares problem

  26. Estimating Wavelet Coefficients … …  = Observed PixelValues Wavelet Noise Reflectance Function Estimation error when onlya part is approximated?

  27. Partial Estimation … … …  = + … = ObservedPixel Values Wavelet Noise Reflectance Function

  28. Partial Estimation … … …  = + … = ObservedPixel Values Wavelet Noise Reflectance Function According to a normal distribution

  29. Partial Estimation … … …  = + … = ObservedPixel Values Wavelet Noise Reflectance Function Normal distribution According to a normal distribution

  30. Partial Estimation … NoIse …  = + … = ObservedPixel Values Wavelet Noise Reflectance Function Finding the best approximation for : Linear Least Squares problem

  31. Inferring Reflectance Functions Reflectance Function(2D wavelet space) Priority Queueof Candidates

  32. Inferring Reflectance Functions Reflectance Function(2D wavelet space) Priority Queueof Candidates

  33. Inferring Reflectance Functions Reflectance Function(2D wavelet space) Priority Queueof Candidates

  34. Inferring Reflectance Functions Reflectance Function(2D wavelet space) Priority Queueof Candidates

  35. Inferring Reflectance Functions Reflectance Function(2D wavelet space) Priority Queueof Candidates

  36. Inferring Reflectance Functions Reflectance Function(2D wavelet space) Priority Queueof Candidates

  37. Inferring Reflectance Functions Reflectance Function(2D wavelet space) Priority Queueof Candidates

  38. Inferring Reflectance Functions Reflectance Function(2D wavelet space) Priority Queueof Candidates

  39. Overview Record photographs Predetermined number of photographs Emit Wavelet Noise

  40. Overview Reflectance Function Infer Reflectance Functions Record photographs Progressive Algorithm For each pixel

  41. Overview Infer Reflectance Functions Record photographs Compute Relit Image Relight Incident Illumination

  42. Results 64 Haar Wavelet Coefficients256 Photographs Reference Photograph

  43. Results 64 Haar Wavelet Coefficients256 Photographs Reference Photograph

  44. Results 64 Haar Wavelet Coefficients256 Photographs Reference Photograph

  45. Results 64 Haar Wavelet Coefficients256 Photographs Reference Photograph

  46. Results 64 Haar Wavelet Coefficients256 Photographs Reference Photograph

  47. Results 128 Haar Wavelet Coefficients512 Photographs Reference Photograph

  48. Results: High Frequency Illumination

  49. Conclusion & Future Work Inferring Reflectance Functions from Wavelet Noise • No restriction on material properties • Stochastic illumination patterns • Trade-off quality versus acquisition time Future Work • Noise filtering • Higher-order wavelets

More Related