Frequency Domain Normal Map Filtering

Frequency DomainNormal Map Filtering Charles Han Bo Sun Ravi Ramamoorthi Eitan Grinspun Columbia University

Normal Mapping • (Blinn 78)

Normal Mapping • (Blinn 78) • Specify surface normals

Normal Mapping

? A Problem… • Multiple normals per pixel • Undersampling • Filtering needed

Supersampling • Correct results • Too slow

MIP mapping • Pre-filter • Normals do not interpolate linearly • Blurring of details

Comparison supersampled MIP mapped

a single vector is not enough how do we represent multiple surface normals? Representation

no general solution Previous Work • Gaussian Distributions • (Olano and North 97) • (Schilling 97) • (Toksvig 05) • Mixture Models • (Fournier 92) • (Tan, et.al. 05) 3D Gaussian 2D covariance matrix 1D Gaussian mixture of Phong lobes mixture of 2D Gaussians

Our Contributions • Theoretical Framework • Normal Distribution Function (NDF) • Linear averaging for filtering • Convolution for rendering • Unifies previous works • New normal map representations • Spherical harmonics • von Mises-Fisher Distribution • Simple, efficient rendering algorithms

Normal Distribution Function (NDF) • Describes normals within region • Defined on the unit sphere • Integrates to one • Extended Gaussian Image (Horn 84)

Normal Distribution Function normal map NDF

NDF Filtering normal map

NDF Filtering • NDF averaging is linear • Store NDFs in MIP map

normal, lights BRDF pixel value Rendering Radially symmetric BRDFs • Lambertian: • Blinn-Phong: • Torrance-Sparrow: • Factored: rendered image

samples Supersampling supersampled image Effective BRDF

samples NDF, Effective BRDF

Spherical Convolution • Form studied in lighting • (Basri and Jacobs 01) • (Ramamoorthi and Hanrahan 01) • Effective BRDF = convolution of NDF & BRDF

Spherical Convolution BRDF NDF Effective BRDF

NDF representations Previous Work • Gaussian Distributions • Olano and North (97) • Schilling (97) • Toksvig (05) • Mixture Models • Fournier (92) • Tan, et.al. (05) • Our Work 3D Gaussian 2D covariance matrix 1D Gaussian mixture of Phong lobes mixture of 2D Gaussians spherical harmonics von Mises-Fisher mixtures

Spherical Harmonics • Analogous to Fourier basis • Convolution formula:

BRDF Coefficients • Arbitrary BRDFs • Cheaply represented • Analytic: compute in shader • Measured: store on GPU • Easily changed at runtime

NDF Coefficients • Store in MIP mapped textures • Finest-level NDFs are delta functions, so: • Use standard linear filtering

Effective BRDF Coefficients • Product of NDF, BRDF coefficients • Proceed as usual

Limitations • Storage cost of NDF • One texture per coefficient • O( ) cost • Limited to low frequencies

more concentrated less concentrated von Mises-Fisher Distribution (vMF) • Spherical analogue to Gaussian • Desirable properties • Spherical domain • Distribution function • Radially symmetric

2 1 3 4 5 6 Mixtures of vMFs NDF number of vMFs

data model NDF vMF Mixture Expectation Maximization (EM) • From machine learning • Used in (Tan et.al. 05) • Fit model parameters to data EM

NDF rendered image Rendering • Convolution • Spherical harmonic coefficients • Analytic convolution formula • Extensions to EM • Aligned lobes (Tan et.al. 05) • Colored lobes

Conclusion • Summary • Theoretical Framework • New NDF representations • Practical rendering algorithms • Future directions • Offline rendering, PRT • Further applications for vMFs • Shadows, parallax, inter-reflections, etc.

Thanks! Tony Jebara, Aner Ben-Artzi, Peter Belhumeur, Pat Hanrahan, Shree Nayar, Evgueni Parilov, Makiko Yasui, Denis Zorin, and nVidia. http://www.cs.columbia.edu/cg/normalmap

Frequency Domain Normal Map Filtering