Slang: Extensible Real-Time Shading System Design

1. Slang: Language Mechanisms for Building Extensible Real-Time Shading Systems github.com/shader-slang/slang Yong He (Carnegie Mellon University) Kayvon Fatahalian (Stanford University) Tim Foley (NVIDIA)

2. Images Rendered in Unreal Engine 4

3. Geometry / Animation Features StaticMesh SkeletalAnim Displacement Material Features Cloth Glass Metal Skylight Light Features PointLight SpotLight

4. Geometry / AnimationFeatures void myShader(...) { <geom>.computeGeometry(); float4 color = <material> .computeMaterial(); <lighting>.computeLighting(color); } StaticMesh SkeletalAnim Displacement Material Features Cloth Glass Metal Skylight Light Features PointLight SpotLight

5. What features should be in a modern shading language?

6. Shading System Design Goals • Modular: Implement a library of shading features as modules • Composable: Assemble a full shader from shader “templates” • Extensible: Add new features quickly • Explicit: Define interfaces clearly and explicitly • Performant: high performance on both GPU and CPU

7. GPU Performance: generate highly specialized shader code Geometry / AnimationFeatures void myShader(...) { <geom>.computeGeometry(); float4 color = <material> .computeMaterial(); <lighting>.computeLighting(color); } StaticMesh SkeletalAnim Displacement Material Features Cloth Glass Metal Light Features PointLight SpotLight Skylight

8. Existing Approaches

9. Reflecting the graphics mental model in system decompositions Material Clay Plastic Skin Matte Light PointLight SpotLight AreaLight

10. Custom shader code generation tools Bungie TFX [Tatarchuk 2017] Unreal Engine Material Editor

11. Preprocessor Techniques for Shader Code Generation • Static “uber shaders” • #include HLSL code snippets

12. Static uber shader struct V2F { #if defined(NORMAL_MAPPING) float3 tangent, bitangent; #endif ... }; #if defined(NORMAL_MAPPING) Texture2D normalMap; #endif V2F myVertexShader(...) { #if defined(NORMAL_MAPPING) v2f.tangent = ...; v2f.bitangent = ...; #endif ... } float3 myFragmentShader() { #if defined(NORMAL_MAPPING) float3 normal = applyNormalMap(normalMap, v2f.tangent, v2f.bitangent, v2f.uv); #endif ... }

13. Achieving modularity: implement shading features in separate files Materials Lights MetalMaterial.hlsl PointLight.hlsl SkinMaterial.hlsl ShProbeLight.hlsl ClothMaterial.hlsl AreaLight.hlsl HairMaterial.hlsl

14. Shading System Implementation Strategies • C++ style object hierarchies • Dynamic dispatch, low performance • Custom Engine Specific Shader Compilation Tools • Engine specific, ad-hoc replication of standard compiler functionality • Preprocessor Techniques in plain HLSL • Difficult to maintain and extend • Error prone: lack of clearly defined interfaces

15. Contribution: Slang Shading Language • Modern shading system goals can be achieved by extending HLSL with a small set of general programming language constructs • Modular • Composable • Extensible • Explicit • Performant • Demonstration of Slang’s performance and extensibility benefits

16. Slang: extension of HLSL with modern language mechanisms • New language features include: • Interface constrained generics • Associated types • ParameterBlock<T> • Retroactive extensions • Global generic parameters • Module import GitHub Page: https://github.com/shader-slang/slang

17. Slang supports first class interfaces interface ILight { float3 computeLighting(); } struct Spotlight : ILight { float3 computeLighting() {...} } struct Skylight : ILight {...} ILight Skylight Spotlight

18. Support specialization via generics Shader entry-points are generic functions void forwardPassShader<TGeometry, TMaterial, TLight> ( TGeometrygeom, TMaterial material, TLight light) { ... } StaticMesh Displacement Skeletal Metal Cloth Glass forwardPassShader<Displacement, Metal, Skylight> Skylight Spotlight

19. Type parameters are constrained by interfaces interface IGeometry { ... } interface IMaterial { ... } interface ILight { float3 computeLighting(); } ILight Skylight Spotlight void forwardPassShader<TGeometry : IGeometry, TMaterial : IMaterial, TLighting : ILight > (TGeometrygeom, TMaterial mat, TLighting lighting) { . . . lighting.computeLighting(); } Similar to Haskell’s type classes, Rust’s type traits and C# interfaces.

20. Material shading returns a BRDF closure BRDF 1. Material Shading f = evalMaterial(P) 2. Lighting Integration Lo = integrate(Li, f, Wi, Wo);

21. Material shading returns a BRDF closure Interface IBRDF{...} interface IMaterial{...} float3 forwardShader<M:IMaterial> (M material) { IBRDF f = material.evalMaterial(p); float3 l = 0; for (int i = 0; i<N; i++) l += f.eval(lights[i], p); return l; } IMaterial Metal

22. Material shading returns a BRDF closure Interface IBRDF{...} interface IMaterial{...} float3 forwardShader<M:IMaterial> (M material) { IBRDF f = material.evalMaterial(p); float3 l = 0; for (int i = 0; i<N; i++) l += f.eval(lights[i], p); return l; } IMaterial Metal

23. Material shading returns a BRDF closure Interface IBRDF{...} interface IMaterial{...} float3 forwardShader<M:IMaterial> (M material) { IBRDF f = material.evalMaterial(p); float3 l = 0; for (int i = 0; i<N; i++) l += f.eval(lights[i], p); return l; } IMaterial Metal

24. Associated types as interface requirement interface IMaterial { associatedtype B : IBRDF; B evalMaterial(Position p); } struct MetalMaterial : IMaterial { typedef DisneyBRDF B; B evalMaterial(Position p) {...} } struct SkinMaterial : IMaterial { typedef SkinBRDF B; B evalMaterial(Position p) {...} } As in Haskell (as type families), Rust, Swift

25. Summary Achieved both modularity and static specialization through two extensions to HLSL: • Interface Constrained Generics • Associated Types • ParameterBlock<T> type

26. Rearchitecting a shading system using Slang

28. Falcor’s shading system • Contains over 5000 lines of shader code, supporting: • A flexible, layered material system • Point, spot, directional, and ambient light type • Glossy reflection using an environment map • Cascaded, exponential and variance shadow map algorithms • Post processing effects, such as screen space ambient occlusion and tone mapping

29. Benefits of adopting Slang • Cleaner Code Structure • Easier to extend with new features • Achieves higher CPU and GPU performance

30. Cleaner Code Structure

31. Falcor implements a layered material library #if USE_DIFFUSE_LAYER computeDiffuseLayer(mat, ...); #endif #if USE_SPECULAR_LAYER computeSpecularLayer(mat, ...); #endif #if USE_EMISSIVE_LAYER computeEmissiveLayer(mat, ...); #endif Emissive Specular Diffuse

32. Make entry point specializable on materials 1. Define an IMaterial interface 2. Change shader entry point to use a generic material parameter interface IMaterial { ... } void main<TMaterial : IMaterial>(ParameterBlock<TMaterial> material) { ... }

33. Generics also useful for layered material specialization #if USE_DIFFUSE_LAYER computeDiffuseLayer(mat, ...); #endif #if USE_SPECULAR_LAYER computeSpecularLayer(mat, ...); #endif #if USE_EMISSIVE_LAYER computeEmissiveLayer(mat, ...); #endif struct LayeredMaterial < let hasDiffuse : bool, let hasSpecular : bool, let hasEmissive: bool > : IMaterial { BRDF evalMaterial(…) { if (hasDiffuse) computeDiffuseLayer(mat, ...); if (hasSpecular) computeSpecularLayer(mat, ...); if (hasEmissive) computeEmissiveLayer(mat, ...); } }

34. Refactoring lighting to enable specialization interface ILight { float3 computeLighting(TSurface surf); } struct PointLight : ILight { ... } struct DirectionalLight : ILight { ... }

35. Easier to Extend

36. New light type: polygonal area lights [Heitz et al. 2016]

37. Fewer changes to code in Slang code base Changes are localized in one place in refactored branch, but distributed at 7 different places in original branch Localization of a feature’s implementation is a sign of good extensibility

38. Fewer changes to code in Slang code base Lines of code is similar in both branches, Refactored architecture is not introducing significant boiler-plate

39. Higher Rendering Performance

40. Performance Evaluation: Environment Setup Goal: measure the CPU and GPU time required to draw each frame using both original and refactored Falcor branch Bistro Exterior Temple Bistro Interior Images rendered at 1920 * 1080 resolution, on Intel i7-5820 CPU and NVIDIA Titan V GPU

41. Higher GPU performance due to light specialization Original Branch Refactored Branch

42. Refactored branch achieved 30% speed-up in CPU time Original Branch Refactored Branch

43. Falcor main branch is using Slang exclusively • Over 17,000 lines of shader code currently compiled by Slang • 20+ Researchers and developers from various groups within NVIDIA using Falcor to prototype rendering techniques are interacting with Slang code

44. Next Steps in Shading Languages

45. C++ for shaders ?? • Should HLSL adopt C++ features or consider other modern language constructs?

46. Next Steps in Real-Time Shading Languages • Shading language support for real-time ray tracing

47. Next Steps in Shading Languages • Shading language support for ray tracing • More support for dynamic dispatch behavior • Unified CPU-GPU programming • Slang is currently a language for GPU code only, and interacts with CPU code via compiler runtime API • Next step: Unified programing language for CPU-GPU graphics programming

48. Thank you! github.com/shader-slang/slang Support: NVIDIA Research National Science Foundation Thank you to Nir Benty for valuable conversations.

49. Adding light specialization to original branch brings GPU performance on par with refactored branch Original Branch Original Branch (+Light Specialization) Refactored Branch

50. Refactored branch achieved 30% speed-up in CPU time Original Branch Original Branch (+Light Specialization) Refactored Branch