Downloaded 889 times
![2SIGGRAPH 2015 – Advances in Real-Time Rendering course
Context
Physically based rendering in Frostbite
See [Lagarde & de Rousiers 2014]
Huge increase in visual quality](https://cdn.statically.io/img/image.slidesharecdn.com/frostbitepbandunifiedvolumetrics-150820015056-lva1-app6891/75/Physically-Based-and-Unified-Volumetric-Rendering-in-Frostbite-2-2048.jpg)
![5SIGGRAPH 2015 – Advances in Real-Time Rendering course
Previous work
Billboards
Analytic fog [Wenzel07]
Analytic light scattering [Miles]
Light shaft
Post process [Mitchell07]
Epipolar sampling [Engelhardt10]
[Mitchell07]
[Miles]](https://cdn.statically.io/img/image.slidesharecdn.com/frostbitepbandunifiedvolumetrics-150820015056-lva1-app6891/75/Physically-Based-and-Unified-Volumetric-Rendering-in-Frostbite-5-2048.jpg)
![6SIGGRAPH 2015 – Advances in Real-Time Rendering course
Previous work
Splatting
Light volumes
[Valliant14][Glatzel14][Hillaire14]
Emissive volumes [Lagarde13]
Volumetric fog [Wronski14]
Sun and local lights
Heterogeneous media
[Valliant14]
[Lagarde13]](https://cdn.statically.io/img/image.slidesharecdn.com/frostbitepbandunifiedvolumetrics-150820015056-lva1-app6891/75/Physically-Based-and-Unified-Volumetric-Rendering-in-Frostbite-6-2048.jpg)
![11SIGGRAPH 2015 – Advances in Real-Time Rendering course
Our approach: clip space volumes
Frustum aligned 3D textures [Wronski14]
Frustum voxel in world space => Froxel
Note: Frostbite is a tiled-based deferred lighting
16x16 tiles with culled light lists
Align volume tiles on light tiles
Reuse per tile culled light list
Volume tiles can be smaller (8x8, 4x4, etc.)
Careful correction for resolution integer division
Default: 8x8 volume tiles, 64 Depth slices
Screen X
ScreenY](https://cdn.statically.io/img/image.slidesharecdn.com/frostbitepbandunifiedvolumetrics-150820015056-lva1-app6891/75/Physically-Based-and-Unified-Volumetric-Rendering-in-Frostbite-11-2048.jpg)
![14SIGGRAPH 2015 – Advances in Real-Time Rendering course
Participating media material definition
Follow the theory [PBR]
Absorption 𝝈 𝒂 (m-1)
Scattering 𝝈 𝒔 (m-1)
Phase 𝒈
Emissive 𝝈 𝒆 (irradiance.m-1)
Extinction 𝝈𝒕 = 𝝈 𝒔 + 𝝈 𝒂
Albedo 𝛒 = 𝝈 𝒔 / 𝝈𝒕
Artists can author {absorption, scattering} or {albedo, extinction}
Train your artists! Important for them to understand their meaning!](https://cdn.statically.io/img/image.slidesharecdn.com/frostbitepbandunifiedvolumetrics-150820015056-lva1-app6891/75/Physically-Based-and-Unified-Volumetric-Rendering-in-Frostbite-14-2048.jpg)
![18SIGGRAPH 2015 – Advances in Real-Time Rendering course
Froxel integration: Sun/Ambient/Emissive
Indirect light on local fog volume
From Frostbite diffuse SH light probe
1 probe at volume centre
Integrate w.r.t. phase function
as a SH cosine lobe [Wronski14]
Sun light
Sample cascaded shadow maps](https://cdn.statically.io/img/image.slidesharecdn.com/frostbitepbandunifiedvolumetrics-150820015056-lva1-app6891/75/Physically-Based-and-Unified-Volumetric-Rendering-in-Frostbite-18-2048.jpg)
![22SIGGRAPH 2015 – Advances in Real-Time Rendering course
Temporal volumetric integration
Solution: Temporal integration
Jittered samples (Halton)
Same offset for all samples along view ray
Jitter scattering AND material samples in sync
Re-project previous scattering/extinction
5% Blend current with previous
Exponential moving average [Karis14]
Out of Frustum: skip history
Frame N
Frame N-1](https://cdn.statically.io/img/image.slidesharecdn.com/frostbitepbandunifiedvolumetrics-150820015056-lva1-app6891/75/Physically-Based-and-Unified-Volumetric-Rendering-in-Frostbite-22-2048.jpg)
![25SIGGRAPH 2015 – Advances in Real-Time Rendering course
Temporal volumetric integration
Remaining issues
Material animation leaves trails
Re-project using velocity?
What about multiple volumes intersecting?
What about animated volumes? (e.g. fluid simulation)
Moving lights leave trails
Use neighbour clamping? [Karis14]
Challenging R&D area!](https://cdn.statically.io/img/image.slidesharecdn.com/frostbitepbandunifiedvolumetrics-150820015056-lva1-app6891/75/Physically-Based-and-Unified-Volumetric-Rendering-in-Frostbite-25-2048.jpg)
![27SIGGRAPH 2015 – Advances in Real-Time Rendering course
Final PM volume
Integrate froxel {scattering, extinction} along view ray
Solves {���𝒊 𝒙, 𝝎𝒐 , 𝑻𝒓 𝒙, 𝒙𝒔 } for each froxel at position 𝒙𝒔
float4 accumScatteringTransmittance = float4(0.0, 0.0, 0.0, 1.0);
for (uint textureDepth = 0; textureDepth < volumeDepth; ++textureDepth)
{
uint4 coord = uint4(DispatchThreadId.xy, textureDepth,0);
float4 scatteringExtinction = g_ ScatteringExtinctionVolume.Load(coord);
const float transmittance = exp(-scatteringExtinction.a*stepLen);
accumScatteringTransmittance.rgb += scatteringExtinction.rgb*accumScatteringTransmittance.a;
accumScatteringTransmittance.a *= transmittance;
g_FinalScatteringTransmittanceVolumeOut[coord.xyz] = accumScatteringTransmittance;
}
Wrong](https://cdn.statically.io/img/image.slidesharecdn.com/frostbitepbandunifiedvolumetrics-150820015056-lva1-app6891/75/Physically-Based-and-Unified-Volumetric-Rendering-in-Frostbite-27-2048.jpg)
![42SIGGRAPH 2015 – Advances in Real-Time Rendering course
Particle/Sun interaction
High quality scattering and self-shadowing for sun/particles interactions
Fourier opacity Maps [Jansen10]
Used in production now](https://cdn.statically.io/img/image.slidesharecdn.com/frostbitepbandunifiedvolumetrics-150820015056-lva1-app6891/75/Physically-Based-and-Unified-Volumetric-Rendering-in-Frostbite-42-2048.jpg)
![43SIGGRAPH 2015 – Advances in Real-Time Rendering course
Physically-based sky/atmosphere
Improved from [Elek09] (Simpler but faster than [Bruneton08])
Collaboration between Frostbite, Ghost and DICE teams.
In production: Mirror’s Edge Catalyst, Need for Speed and Mass Effect Andromeda
Mass Effect Andromeda, Bioware Need for Speed, Ghost](https://cdn.statically.io/img/image.slidesharecdn.com/frostbitepbandunifiedvolumetrics-150820015056-lva1-app6891/75/Physically-Based-and-Unified-Volumetric-Rendering-in-Frostbite-43-2048.jpg)
![46SIGGRAPH 2015 – Advances in Real-Time Rendering course
References
[Lagarde & de Rousiers 2014] Moving Frostbite to PBR, SIGGRAPH 2014.
[PBR] Physically Based Rendering book, http://www.pbrt.org/.
[Wenzel07] Real time atmospheric effects in game revisited, GDC 2007.
[Mitchell07] Volumetric Light Scattering as a Post-Process, GPU Gems 3, 2007.
[Andersson11] Shiny PC Graphics in Battlefield 3, GeForceLan, 2011.
[Engelhardt10] Epipolar Sampling for Shadows and Crepuscular Rays in Participating Media with Single Scattering, I3D 2010.
[Miles] Blog post http://blog.mmacklin.com/tag/fog-volumes/
[Valliant14] Volumetric Light Effects in Killzone Shadow Fall, SIGGRAPH 2014.
[Glatzel14] Volumetric Lighting for Many Lights in Lords of the Fallen, Digital Dragons 2014.
[Hillaire14] Volumetric lights demo
[Lagarde13] Lagarde and Harduin, The art and rendering of Remember Me, GDC 2013.
[Wronski14] Volumetric fog: unified compute shader based solution to atmospheric solution, SIGGRAPH 2014.
[Karis14] High Quality Temporal Super Sampling, SIGGRAPH 2014.
[Jansen10] Fourier Opacity Mapping, I3D 2010.
[Salvi10] Adaptive Volumetric Shadow Maps, ESR 2010.
[Elek09] Rendering Parametrizable Planetary Atmospheres with Multiple Scattering in Real-time, CESCG 2009.
[Bruneton08] Precomputed Atmospheric scattering, EGSR 2008.](https://cdn.statically.io/img/image.slidesharecdn.com/frostbitepbandunifiedvolumetrics-150820015056-lva1-app6891/75/Physically-Based-and-Unified-Volumetric-Rendering-in-Frostbite-46-2048.jpg)
![51SIGGRAPH 2015 – Advances in Real-Time Rendering course
Particles effect volumetric lighting
We already have shadow from sun
Cascaded translucent shadow
See [Andersson11]
Local lights: volumetric shadow maps
Cast shadows onto
opaque surfaces, other effects and transparents
participating media
Need to voxelize our particles](https://cdn.statically.io/img/image.slidesharecdn.com/frostbitepbandunifiedvolumetrics-150820015056-lva1-app6891/75/Physically-Based-and-Unified-Volumetric-Rendering-in-Frostbite-51-2048.jpg)
Uploaded byElectronic Arts / DICE
Physically Based and Unified Volumetric Rendering in Frostbite
The document outlines advancements in physically-based volumetric rendering techniques implemented in the Frostbite engine, presented by Sebastien Hillaire at SIGGRAPH 2015. It discusses the integration of scattering, lighting, and shadowing interactions to enhance visual quality in real-time rendering, along with the use of 'froxel' technology and voxelizations for efficient data handling. The work demonstrates a comprehensive approach to volumetric rendering, targeting improved realism in games powered by the Frostbite engine.
Physically Based and Unified Volumetric Rendering in Frostbite
- 1. Physically Based andUnified Volumetric Rendering in Frostbite SEBASTIEN HILLAIRE - ELECTRONIC ARTS / FROSTBITE
- 2. 2SIGGRAPH 2015 –Advances in Real-Time Rendering course Context Physically based rendering in Frostbite See [Lagarde & de Rousiers 2014] Huge increase in visual quality
- 3. 3SIGGRAPH 2015 –Advances in Real-Time Rendering course Context Volumetric rendering in Frostbite was limited Global distance/height fog Screen space light shafts Particles
- 4. 4SIGGRAPH 2015 –Advances in Real-Time Rendering course Real-life volumetric Atmosphere and clouds Scattering events More fog Scattering occlusion Varying density
- 5. 5SIGGRAPH 2015 –Advances in Real-Time Rendering course Previous work Billboards Analytic fog [Wenzel07] Analytic light scattering [Miles] Light shaft Post process [Mitchell07] Epipolar sampling [Engelhardt10] [Mitchell07] [Miles]
- 6. 6SIGGRAPH 2015 –Advances in Real-Time Rendering course Previous work Splatting Light volumes [Valliant14][Glatzel14][Hillaire14] Emissive volumes [Lagarde13] Volumetric fog [Wronski14] Sun and local lights Heterogeneous media [Valliant14] [Lagarde13]
- 7. 7SIGGRAPH 2015 –Advances in Real-Time Rendering course Scope and motivation Increase visual quality and give more freedom to art direction! Physically based volumetric rendering Meaningful material parameters Decouple material from lighting Coherent results Unified volumetric interactions Lighting + regular and volumetric shadows Interaction with opaque, transparent and particles
- 8. 8SIGGRAPH 2015 –Advances in Real-Time Rendering course Results
- 9. 9SIGGRAPH 2015 –Advances in Real-Time Rendering course Outline Volumetric rendering Volumetric shadows More volumetric rendering in Frostbite Conclusion
- 10. 10SIGGRAPH 2015 –Advances in Real-Time Rendering course Volumetric rendering: single scattering 𝑳𝒊 𝒙, 𝝎𝒊 = 𝑻𝒓 𝒙, 𝒙𝒔 𝑳𝒔 𝒙𝒔, 𝝎𝒐 + 𝟎 𝒔 𝑻𝒓 𝒙, 𝒙𝒕 𝝈𝒕 𝒙 𝑳𝒔𝒄𝒂𝒕 𝒙𝒕, 𝝎𝒊 𝒅𝒕 𝑻𝒓 𝒙, 𝒙𝒔 = 𝒆𝒙𝒑(− 𝟎 𝒔 𝝈𝒕 𝒙 𝒅𝒕) 𝒙 𝒙𝒔 𝑽𝒊𝒔 𝒙, 𝒍 = 𝒔𝒉𝒂𝒅𝒐𝒘𝑴𝒂𝒑 𝒙, 𝒍 ∗ 𝒗𝒐𝒍𝒖𝒎𝒆𝒕𝒓𝒊𝒄𝑺𝒉𝒂𝒅𝒐𝒘𝑴𝒂𝒑 𝒙, 𝒍 𝑳𝒔𝒄𝒂𝒕 𝒙𝒕, 𝝎𝒊 = 𝛒 𝒍=𝟎 𝒍𝒊𝒈𝒉𝒕𝒔 𝒇 𝒗, 𝒍 𝑽𝒊𝒔 𝒙, 𝒍 𝑳𝒊(𝒙, 𝒍)
- 11. 11SIGGRAPH 2015 –Advances in Real-Time Rendering course Our approach: clip space volumes Frustum aligned 3D textures [Wronski14] Frustum voxel in world space => Froxel Note: Frostbite is a tiled-based deferred lighting 16x16 tiles with culled light lists Align volume tiles on light tiles Reuse per tile culled light list Volume tiles can be smaller (8x8, 4x4, etc.) Careful correction for resolution integer division Default: 8x8 volume tiles, 64 Depth slices Screen X ScreenY
- 12. 12SIGGRAPH 2015 –Advances in Real-Time Rendering course Our approach: data flow 1. Material properties 2. Froxel Light Scattering 3. Final integration Participating media entities ClipspacevolumesInputdata Lighting and shadowing information
- 13. 13SIGGRAPH 2015 –Advances in Real-Time Rendering course Our approach: data flow 1. Material properties 2. Froxel Light Scattering 3. Final integration Participating media entities ClipspacevolumesInputdata Lighting and shadowing information
- 14. 14SIGGRAPH 2015 –Advances in Real-Time Rendering course Participating media material definition Follow the theory [PBR] Absorption 𝝈 𝒂 (m-1) Scattering 𝝈 𝒔 (m-1) Phase 𝒈 Emissive 𝝈 𝒆 (irradiance.m-1) Extinction 𝝈𝒕 = 𝝈 𝒔 + 𝝈 𝒂 Albedo 𝛒 = 𝝈 𝒔 / 𝝈𝒕 Artists can author {absorption, scattering} or {albedo, extinction} Train your artists! Important for them to understand their meaning!
- 15. 15SIGGRAPH 2015 –Advances in Real-Time Rendering course Participating Media (PM) properties voxelization PM sources Depth fog Height fog Local fog volumes With or W/o density textures Voxelize PM properties into V-Buffer Add Scattering, Emissive and Extinction Average Phase g (no multi lobe) Wavelength independent 𝝈𝒕 (for now) V-Buffer (per Froxel data) Format Scattering R Scattering G Scattering B Extinction RGBA16F Emissive R Emissive G Emissive B Phase (g) RGBA16F With density texturesWithout density textures
- 16. 16SIGGRAPH 2015 –Advances in Real-Time Rendering course Our approach: data flow 1. Material properties 2. Froxel Light Scattering 3. Final integration Participating media entities ClipspacevolumesInputdata Lighting and shadowing information
- 17. 17SIGGRAPH 2015 –Advances in Real-Time Rendering course Froxel integration Per froxel 1. Sample PM properties data 2. Evaluate 1. Scattered light 𝑳𝒔𝒄𝒂𝒕 𝒙𝒕, 𝝎𝒐 2. Extinction Scattered light: 1 sample per froxel Integrate all light sources: indirect light + sun + local lights Scattering/Transmittance Buffer Format Extinction RGBA16FScattered light to camera RGB
- 18. 18SIGGRAPH 2015 –Advances in Real-Time Rendering course Froxel integration: Sun/Ambient/Emissive Indirect light on local fog volume From Frostbite diffuse SH light probe 1 probe at volume centre Integrate w.r.t. phase function as a SH cosine lobe [Wronski14] Sun light Sample cascaded shadow maps
- 19. 19SIGGRAPH 2015 –Advances in Real-Time Rendering course Froxel integration: Local lights Local lights Reuse tiled-lighting code Use forward tile light list post-culling No scattering? skip local lights Shadows Regular shadow maps Volumetric shadow maps
- 20. 20SIGGRAPH 2015 –Advances in Real-Time Rendering course Temporal volumetric integration 1 scattering/extinction sample per frame Under sampling with very strong material Aliasing under camera motion Shadows make it worse
- 21. 21SIGGRAPH 2015 –Advances in Real-Time Rendering course Temporal volumetric integration
- 22. 22SIGGRAPH 2015 –Advances in Real-Time Rendering course Temporal volumetric integration Solution: Temporal integration Jittered samples (Halton) Same offset for all samples along view ray Jitter scattering AND material samples in sync Re-project previous scattering/extinction 5% Blend current with previous Exponential moving average [Karis14] Out of Frustum: skip history Frame N Frame N-1
- 23. 23SIGGRAPH 2015 –Advances in Real-Time Rendering course With Temporal volumetric integration
- 24. 24SIGGRAPH 2015 –Advances in Real-Time Rendering course With Temporal volumetric integration
- 25. 25SIGGRAPH 2015 –Advances in Real-Time Rendering course Temporal volumetric integration Remaining issues Material animation leaves trails Re-project using velocity? What about multiple volumes intersecting? What about animated volumes? (e.g. fluid simulation) Moving lights leave trails Use neighbour clamping? [Karis14] Challenging R&D area!
- 26. 26SIGGRAPH 2015 –Advances in Real-Time Rendering course Our approach: data flow 1. Material properties 2. Froxel Light Scattering 3. Final integration Participating media entities ClipspacevolumesInputdata Lighting and shadowing information
- 27. 27SIGGRAPH 2015 –Advances in Real-Time Rendering course Final PM volume Integrate froxel {scattering, extinction} along view ray Solves {𝑳𝒊 𝒙, 𝝎𝒐 , 𝑻𝒓 𝒙, 𝒙𝒔 } for each froxel at position 𝒙𝒔 float4 accumScatteringTransmittance = float4(0.0, 0.0, 0.0, 1.0); for (uint textureDepth = 0; textureDepth < volumeDepth; ++textureDepth) { uint4 coord = uint4(DispatchThreadId.xy, textureDepth,0); float4 scatteringExtinction = g_ ScatteringExtinctionVolume.Load(coord); const float transmittance = exp(-scatteringExtinction.a*stepLen); accumScatteringTransmittance.rgb += scatteringExtinction.rgb*accumScatteringTransmittance.a; accumScatteringTransmittance.a *= transmittance; g_FinalScatteringTransmittanceVolumeOut[coord.xyz] = accumScatteringTransmittance; } Wrong
- 28. 28SIGGRAPH 2015 –Advances in Real-Time Rendering course Final PM volume Non energy conservative integration: Single scattered light sample 𝑆 = 𝑳𝒔𝒄𝒂𝒕 𝒙𝒕, 𝝎𝒐 OK Single transmittance sample 𝑻𝒓 𝒙, 𝒙𝒔 NOT OK Integrate lighting w.r.t. transmittance over froxel depth D 𝝈 𝒔 = 5 𝝈 𝒔 = 50 𝝈 𝒔 = 5000 𝝈 𝒔 = 5000 swapped scattering/transmittance code 0 𝐷 ��−𝝈𝒕 𝑥 × 𝑆 𝑑𝑥 = 𝑆−𝑆×𝑒−𝝈𝒕 𝐷 𝝈𝒕 𝝈 𝒔 = 5000
- 29. 29SIGGRAPH 2015 –Advances in Real-Time Rendering course Final PM volume Also improves with volumetric shadows Without fixed integration: light leaking With improved integration:
- 30. 30SIGGRAPH 2015 –Advances in Real-Time Rendering course Final PM volume rendering on scene {𝑳𝒊 𝒙, 𝝎𝒐 , 𝑻𝒓 𝒙, 𝒙𝒔 } Similar to pre-multiplied color/alpha Applied on opaque surfaces per pixel Evaluated on transparent surfaces per vertex, applied per pixel Camera view point New view with locked PM volumes
- 31. 31SIGGRAPH 2015 –Advances in Real-Time Rendering course Result validation Compare results to references from Mitsuba Physically based path tracer Same conditions: single scattering only, exposure, etc. Scene 1:
- 32. 32SIGGRAPH 2015 –Advances in Real-Time Rendering course Result validation - scene 1 Frostbite Mitsuba Render (light above) Luminance gradient Render (light inside) Luminance gradient
- 33. 33SIGGRAPH 2015 –Advances in Real-Time Rendering course Result validation - scene 2 G=0 G=0.9 Render Luminance gradient Render Luminance gradient Frostbite Mitsuba
- 34. 34SIGGRAPH 2015 –Advances in Real-Time Rendering course Performance Sun + shadow cascade 14 point lights 2 with regular & volumetric shadows 6 local fog volumes All with density textures
- 35. 35SIGGRAPH 2015 –Advances in Real-Time Rendering course Performance: PS4, 900p 64 depth slices Plan for your use cases High or low frequency media? Local lights needed? Emissive needed? Etc. Volume tile resolution 8x8 16x16 PM Material voxelization 0.45 ms 0.15 ms Light scattering 2.00 ms 0.50 ms Final accumulation 0.40 ms 0.08 ms Application (Fog pass) +0.1 ms +0.1 ms Total 2.95 ms 0.83 ms Light scattering components 8x8 Local lights 1.1 ms +Sun scattering +0.5 ms +Temporal integration +0.4 ms
- 36. 36SIGGRAPH 2015 –Advances in Real-Time Rendering course Outline Volumetric rendering Volumetric shadows More volumetric rendering in Frostbite Conclusion
- 37. 37SIGGRAPH 2015 –Advances in Real-Time Rendering course Volumetric shadow maps Additional extinction volumes 3 levels clip-map oriented on frustum Required for out-of-view shadow casters Store extinction Volumetric shadow maps 3d textures store transmittance Ortho/perspective mapping for point/spot lights
- 38. 38SIGGRAPH 2015 –Advances in Real-Time Rendering course Volumetric shadow maps Part of our common light shadow system Opaque Particles Participating media
- 39. 39SIGGRAPH 2015 –Advances in Real-Time Rendering course Particle volumetric shadows Default High quality option Selectable per emitter trilinear Sphere
- 40. 40SIGGRAPH 2015 –Advances in Real-Time Rendering course Performance: PS4 Ray marching of 323 volumetric shadow maps Spot light: 0.04 ms Point light: 0.14 ms 1k particles voxelization Default quality: 0.03 ms High quality: 0.25 ms
- 41. 41SIGGRAPH 2015 –Advances in Real-Time Rendering course Outline Volumetric rendering Particle volumetric shadows More volumetric rendering in Frostbite Conclusion
- 42. 42SIGGRAPH 2015 –Advances in Real-Time Rendering course Particle/Sun interaction High quality scattering and self-shadowing for sun/particles interactions Fourier opacity Maps [Jansen10] Used in production now
- 43. 43SIGGRAPH 2015 –Advances in Real-Time Rendering course Physically-based sky/atmosphere Improved from [Elek09] (Simpler but faster than [Bruneton08]) Collaboration between Frostbite, Ghost and DICE teams. In production: Mirror’s Edge Catalyst, Need for Speed and Mass Effect Andromeda Mass Effect Andromeda, Bioware Need for Speed, Ghost
- 44. 44SIGGRAPH 2015 –Advances in Real-Time Rendering course Conclusion Physically based volumetric rendering Participating media material definition Lighting and shadowing interactions A more unified volumetric rendering system Handles many interactions Participating media, volumetric shadows, particles, opaque surfaces, etc. Physically-based volumetric rendering framework used for all games powered by Frostbite in the future
- 45. 45SIGGRAPH 2015 –Advances in Real-Time Rendering course Future work Improved participating media rendering Phase function integral w.r.t. area lights solid angle Inclusion in reflection views Graph based material definition, GPU simulation, Streaming Better temporal integration! Any ideas? Sun volumetric shadow Transparent shadows from transparent surfaces? Optimisations V-Buffer packing Particles voxelization Volumetric shadow maps generation How to scale to 4k screens efficiently
- 46. 46SIGGRAPH 2015 –Advances in Real-Time Rendering course References [Lagarde & de Rousiers 2014] Moving Frostbite to PBR, SIGGRAPH 2014. [PBR] Physically Based Rendering book, http://www.pbrt.org/. [Wenzel07] Real time atmospheric effects in game revisited, GDC 2007. [Mitchell07] Volumetric Light Scattering as a Post-Process, GPU Gems 3, 2007. [Andersson11] Shiny PC Graphics in Battlefield 3, GeForceLan, 2011. [Engelhardt10] Epipolar Sampling for Shadows and Crepuscular Rays in Participating Media with Single Scattering, I3D 2010. [Miles] Blog post http://blog.mmacklin.com/tag/fog-volumes/ [Valliant14] Volumetric Light Effects in Killzone Shadow Fall, SIGGRAPH 2014. [Glatzel14] Volumetric Lighting for Many Lights in Lords of the Fallen, Digital Dragons 2014. [Hillaire14] Volumetric lights demo [Lagarde13] Lagarde and Harduin, The art and rendering of Remember Me, GDC 2013. [Wronski14] Volumetric fog: unified compute shader based solution to atmospheric solution, SIGGRAPH 2014. [Karis14] High Quality Temporal Super Sampling, SIGGRAPH 2014. [Jansen10] Fourier Opacity Mapping, I3D 2010. [Salvi10] Adaptive Volumetric Shadow Maps, ESR 2010. [Elek09] Rendering Parametrizable Planetary Atmospheres with Multiple Scattering in Real-time, CESCG 2009. [Bruneton08] Precomputed Atmospheric scattering, EGSR 2008.
- 47. 47SIGGRAPH 2015 –Advances in Real-Time Rendering course Questions? Thanks The Frostbite rendering Team Bioware, DICE, Ghost Andreas Glad, Edvard Sandberg, Gustav Bodare, Fabien Christin, Mikael Uddholm, Simon Edgar and all the early tech adopters and collaborators. Natalya Tatarchuk For further discussions sebastien.hillaire@frostbite.com https://twitter.com/SebHillaire
- 48. 48SIGGRAPH 2015 –Advances in Real-Time Rendering course Bonus slides
- 49. 49SIGGRAPH 2015 –Advances in Real-Time Rendering course Volumetric shadow maps
- 50. 50SIGGRAPH 2015 –Advances in Real-Time Rendering course Volumetric shadows are important! Correct secondary ray shadowing Crucial for heterogeneous media No volumetric shadow: approximate with 𝝈𝒕 at light position With volumetric shadows Without volumetric shadows
- 51. 51SIGGRAPH 2015 –Advances in Real-Time Rendering course Particles effect volumetric lighting We already have shadow from sun Cascaded translucent shadow See [Andersson11] Local lights: volumetric shadow maps Cast shadows onto opaque surfaces, other effects and transparents participating media Need to voxelize our particles
- 52. 52SIGGRAPH 2015 –Advances in Real-Time Rendering course Particle voxelization 1 - clear 2 - voxelize 3 – convert and add Use an intermediate uint cascaded extinction volume Extinction of 1.0f maps to 2048u Voxelize using InterlockedAdd Required for particleCount compute threads coherent write to memory Emitter Extinction volume UINT Extinction volume Float16
- 53. 53SIGGRAPH 2015 –Advances in Real-Time Rendering course Particle voxelization methods Default High quality option Selectable per emitter 2x2x2 cubepoint trilinear Sphere
- 54. 54SIGGRAPH 2015 –Advances in Real-Time Rendering course Discussion Soft shadows No sharp details Shadows can flicker for moving lights Under high extinction Received by opaque, transparent, particles and participating media
- 55. 55SIGGRAPH 2015 –Advances in Real-Time Rendering course Particle voxelization consistency Needs to be “extinction conservative” For large voxel cascades, particles write more often into same voxels Result in overshadow Without extinction normalisation With extinction normalisation - Per voxel, voxelize 𝝈𝒕′ as 𝝈𝒕′ = 𝝈𝒕 𝑣𝑜𝑥𝑒𝑙𝐶𝑜𝑢𝑛𝑡 ∗ 𝑣𝑜𝑥𝑒𝑙𝑉𝑜𝑙𝑢𝑚𝑒 - Per particle, 𝝈𝒕 given for unit cube Distribute extinction per volume:
- 56. 56SIGGRAPH 2015 –Advances in Real-Time Rendering course Results
Editor's Notes
- #2 Hi and welcome to this course. I am Sebastien Hillaire and I will present to you our recent work on Physically Based and Unified Volumetric Rendering in Frostbite.
- #3 But first a bit of context. We recently have moved Frostbite to be a physically based rendering engine. This resulted in a huge raise in quality as you can see in this NFS picture. It is hard to tell which one is from Frostbite or and which one is real.
- #4 Despite all that, our volumetric rendering tech was still restricted to basic fog, screen space light shafts and particles. Nothing wrong with these techniques but we wanted more.
- #5 We would like sky and atmosphere simulation, large scale fog, heterogeneous participating media, light scattering in the air and also the possibility to have occluded scattering resulting in volumetric shadows and light shafts.
- #6 Analytic fog / light scattering Fast! Not shadowed Only homogeneous media Screen space light shaft High quality Sun/sky needs to be visible on screen Only homogeneous media Can go for Epipolar sampling but this won’t save the day
- #7 Splatting methods appeared largely with this new generation of consoles such as light volume splatting. This can result in high quality scattering but usually it does not match the participating media of the scene. More recently Wronski presented a technique called Volumetric Fog, allowing spatially varying participating media and local lights to scatter. We were happy to see this as it was aligned with what we were doing at the time as you will see during this presentation. However it did not seem really physically based at the time and some features we wanted were missing.
- #8 Getting volumetric rendering will allow an increase in visual complexity and will result in more varied visuals and art directions. We want it to be physically based: this means that participating media materials are decoupled from the light sources (e.g. no scattering colour on the light entities). Media parameters are also a meaningful set of parameters. With this we should get more coherent results that are easier to control and understand. Also, because there are several entities interacting with volumetric in Frostbite (fog, particles, opaque&transparent surfaces, etc). We also want to unify the way we deal with that to not have X methods for X types of interaction.
- #9 This video gives you an overview of what we got from this work: lights that generate scattering according to the participating media, volumetric shadow, local fog volumes, etc. And I will show you now how we achieve it.
- #11 As of today we restrict ourselves to single scattering when rendering volumetric. This is already challenging to get right. When a light surface interact with a surface, it is possible to evaluate the amount of light bounce to the camera by evaluating for example a BRDF. But in the presence of participating media, things get more complex. You have to take into account transmittance when the light is traveling through the media Then you need to integrate the scattered light along the view ray by taking many samples For each of these samples, you also need to take into account transmittance to the view point You also need to integrate the scattered light at each position And take into account phase function, regular shadow map (opaque objects) and volumetric shadow map (participating media and other volumetric entity)
- #12 As in Wronski, All our volumes are 3d textures that are clip space aligned (such voxels become Froxels in world space, Credit Alex Evans and Sony ATG , see Learning from Failure: a Survey of Promising, Unconventional and Mostly Abandoned Renderers for ‘Dreams PS4’, a Geometrically Dense, Painterly UGC Game’, Advances in Real-Time Rendering course, SIGGRAPH 2015). This volume is also aligned with our screen light tiles. This is because we are reusing the forward light tile list culling result to accelerate the scattered light evaluation (remember, Frostbite is a tile based deferred lighting engine). Our volume tiles in screen space can be smaller than the light tiles (which are 16x16 pixels). By default we use Depth resolution of 64 8x8 volume tiles 720p requires 160x90x64 (~7mb per rgbaF16 texture) 1080p requires 240x135x64 (~15mb per rgbaF16 texture)
- #13 This is an overview of our data flow. We are using clip space volumes to store the data at different stages of our pipeline. We have material properties which are first voxelised from participating media entities. Then using light sources of our scene and this material property volume we can generate scattered light data per froxel. This data can be temporally upsampled to increase the quality. Finally, we have an integration step that prepares the data for rendering.
- #15 Our participating medias are defined according to the state of the art in physically based rendering. We have the following list of properties: - Absorption describing the amount of light absorbed by the media over a certain path length - Scattering describing the amount of light scattered over a certain path length - Emissive describing emitted light - And a single lobe phase function describing how the light bounces on particles (uniformly, forward scattering, etc.). It is based on Henyey-Greenstein (and you can use the Schlick approximation). As with every physically based component, it is very important for artists to understand them so take the time to educate them.
- #16 Depth/height fog and local fog volumes are entities that can be voxelized. You can see here local fog volumes as plain or with varying density according to a density texture. We voxelize them into a Vbuffer analogous to screen Gbuffer but in Volume (clip space). We basically add all the material parameters together since they are linear. Except the phase function which is averaged. We only also only consider a single lobe for now according to the HG phase function. We have deliberately chosen to go with wavelength independent extinction to have cheaper volumes (material, lighting, shadows). But it would be very easy to extend if necessary at some point. Supporting emissive is an advantage for artist to position local fog volume that emit light as scattering would do but that do not match local light. This can be used for cheap ambient lighting.
- #18 For each froxel, one thread will be in charge of gathering scattered light and extinction. Extinction is simply copied over from the material. You will see later why this is important for visual quality in the final stage (to use extinction instead of transmittance for energy conservative scattering). Extinction is also linear so it will be better to temporally integrate it instead of the non linear transmittance value.
- #19 Then we integrate the scattered light. One sample per froxel. We first integrate ambient the same way as Wronski. Frostbite allows us to sample diffuse SH light probes. We use one per local fog volume positioned at their centre. We also integrate the sun light according to our cascaded shadow maps. We could use exponential shadow maps but we do not as our temporal up-sampling is enough to soften the result. You can easily notice the heterogeneous nature of the local fog shown here.
- #20 We also integrate local lights. And we re-use the tile culling result to only take into account lights visible within each tile. One good optimisation is to skip it all if you do not have any scattering possible according to your material properties. Each of these lights can also sample their associated shadow maps. We support regular shadow maps and also volumetric shadow maps (described later).
- #21 As I said, we are only using a single sample per froxel. This can unfortunately result in very strong aliasing for very thick participating media and when integrating the local light contribution.
- #22 You can also notice it in the video, as well as very strong aliasing of the shadow coming from the tree.
- #23 To mitigate these issues, we temporally integrate our frame result with the one of previous frame. (well know, also used by Karis last year for TAA). To achieve this, we jitter our samples per frame uniformly along the view ray The material and scattered light samples are jittered using the same offset (to soften evaluated material and scattered light) Integrate each frame according to an exponential moving average And we ignore previous result in case no history sample is available (out of previous frustum)
- #24 This video shows the before after result of the temporal integration.
- #25 This video shows how the aliasing of the local light shadow is removed and result in a softer look.
- #26 This is great and promising but there are several issues remaining: Local fog volume and lights will leave trails when moving One could use local fog volumes motion stored in a buffer the same way as we do in screenspace for motion blur But what do we do when two volumes intersect? This is the same problem as deep compositing For lighting, we could use neighbour colour clamping but this will not solve the problem entirely This is an exciting and challenging R&D area for the future and I’ll be happy to discuss about it with you if you have some ideas
- #28 We basically accumulate near to far scattering according to transmittance. This will solve the integrated scattered light and transmittance along the view and that for each froxel. One could use the code sample shown here: accumulate scattering and then transmittance for the next froxel, and this slice by slice. However, that is completely wrong. Indeed there is a dependency on the accumScatteringTransmitance.a value (transmittance). Should we update transmittance of scattering first?
- #29 You can see here multiple volumes with increasing scattering properties. It is easy to understand that integrating scattering and then transmittance is not energy conservative. We could reverse the order of operations. You can see that we get somewhat get back the correct albedo one would expect but it is overall too dark and temporally integrating that is definitely not helping here. So how to improve this? We know we have one light and one extinction sample. We can keep the light sample: it is expensive to evaluate and good enough to assume it constant on along the view ray inside each depth slice. But the single transmittance is completely wrong. The transmittance should in fact be 0 at the near interface of the depth layer and exp(-mu_t d) at the far interface of the depth slice of width d. What we do to solve this is integrate scattered light analytically according to the transmittance in each point on the view ray range within the slice. One can easily find that the analytical integration of constant scattered light over a definite range according to one extinction sample can be reduced this equation. Using this, we finally get consistent lighting result for scattering and this with respect to our single extinction sample (as you can see on the bottom picture).
- #30 You can also see that this fixes the light leaking we noticed sometimes for relatively large depth slices and strongly scattering media even when volumetric shadow are enabled.
- #31 Once we have that final integrated buffer, we can apply it on everything in our scene during the sky rendering pass. As it contains scattered light reaching the camera and transmittance, it is easy to apply it as a pre-multiplied colour-alpha on everything. For efficiency, it is applied per vertex on transparents but we are thinking of switching this to per pixel for better quality.
- #32 Our target is to get physically based results. As such, we have compared our results against the physically based path tracer called Mitsuba. We constrained Mitsuba to single scattering and to use the same exposure, etc. as our example scenes. The first scene I am going to show you is a thick participating media layer with a light above and then into it.
- #33 You can see here the frostbite render on top and Mitsuba render at the bottom. You can also see the scene with a gradient applied to it. It is easy to see that our result matches, you can also recognize the triangle shape of scattered light when the point lights is within the medium. This is a difficult case when participating media is non uniform and thick due to our discretisation of volumetric shadows and material representation. So you can see some small differences. But overall, it matches and we are happy with these first results and improve them in the future.
- #34 This is another example showing very good match for an HG phase function with g=0 and g=0,9 (strong forward scattering).
- #35 I am now going to give the performance we obtain for the following scene.
- #36 You can see that the performance varies a lot depending on what you have enabled and the resolution of the clip space volumes. This shows that it will be important to carefully plan what are the needs of you game and different scenes. Maybe one could also bake static scenes scattering and use the emissive channel to represent the scattered light for an even faster rendering of complex volumetric lighting.
- #37 Now I want to describe how we can get our particles to cast volumetric shadows in Frostbite.
- #38 We also support volumetric shadow maps (shadow resulting from voxelized volumetric entities in our scene) To this aim, we went for a simple and fast solution We first define a 3 levels cascaded clip map volume following and containing the camera. With tweakable per level voxel size and world space snapping This volume contains all our participating media entities voxelized again within it (required for out of view shadow caster, clip space volume would not be enough) A volumetric shadow map is defined as a 3D texture (assigned to a light) that stores transmittance Transmittance is evaluated by ray marching the extinction volume Projection is chosen as a best fit for the light type (e.g. frustum for spot light) Our volumetric shadow maps are stored into an atlas to only have to bind a single texture (with uv scale and bias) when using them.
- #39 Volumetric shadow maps are entirely part of our shared lighting pipeline and shader code. It is sampled for each light having it enabled and applied on everything in the scene (particles, opaque surfaces, participating media) as visible on this video.
- #40 Another bonus is that we also voxelize our particles. We have tried many voxelization method. Point and its blurred version but this was just too noisy. Our default voxelization method is trilinear. You can see the shadow is very soft and there is no popping visible. We also have a high quality voxelization where all threads write all the voxels contained within the particle sphere. A bit brute force for now but it works when needed. You can see the result of volumetric shadows from particle onto participating media in the last video. (See bonus slides for more details)
- #41 Some performance numbers. Point lights are more expensive than spot lights because spot lights are integrated slice by slice whereas a full raytrace is done for each point light shadow voxels. We have ideas to fix that in the near future. Default particle voxelization is definitely cheap for 1K particles.
- #42 If we have more time, let me describe the additional volumetric rendering goodies we have in Frostbite.
- #43 Our translucent shadows in Frostbite (see Andersson11) allows particles to cast shadows on opaque surfaces but not on themselves. This technique also did not support scattering. We have added that support in frostbite by using Fourier opacity mapping. This allows us to have some very high quality coloured shadowing, scattering resulting in sharp silver lining visual effects as you can see on this screenshots and cloud video. This is one special case for the sun (non unified) but it was needed to get that extra bit of quality were needed for the special case of the sun which requires special attention.
- #44 We also have added support for physically based sky and atmosphere scattering simulation last year. This was a fruitful collaboration between Frostbite and Ghost and DICE game teams (Mainly developed by Edvard Sandberg and Gustav Bodare at Ghost). Now it is used in production by lots games such as Mirror’s Edge or Mass Effect Andromeda. It is an improved version of Elek’s paper which is simpler and faster than Bruneton. I unfortunately have no time to dive into details in this presentation. But in the comment I have time . Basically, the lighting artist would define the atmosphere properties and the light scattering and sky rendering will automatically adapt to the sun position. When the atmosphere is changed, we need to update our pre-computed lookup tables and this can be distributed over several frame to limit the evaluation impact on GPU.
- #46 There is lots of room to improve this technology. You have some example here in no special order.
- #47 Some references you can look at online
- #48 Thank you for your attention! Now it is time for questions. And if we do not have time do not hesitate to contact me online.
- #49 Thank you for your attention! Now it is time for questions. And if we do not have time do not hesitate to reach me online.
- #50 A scene showing volumetric our shadow maps debug view for 32x32x32 and 48x48x48 resolution. Perfect voxel intersection are achieved using Amanatide ray marching (A Fast Voxel Traversal Algorithm for Ray Tracing Amanatides&Woo, 1987). Really recommended! This allows us to get clean volumetric debug views that are definitely cheaper than brute force raymarch with small steps.
- #51 Volumetric shadow are mandatory to get correctly shadowed secondary rays. It is also important because it is the only way we can support heterogeneous participating media properties and density. For lights not having volumetric shadows, one could samples extinction at light position and assume it being constant around the light position. This would result in over/under shadow but would still help improve the visuals.
- #52 We already have transparent shadows from the sun for our particles (see Andersson11) but not from local lights. We want our particles to leverage our volumetric shadows to also cast them onto everything and for that we need to voxelize them.
- #53 To do so we will use the cascaded extinction volume but we will need an intermediate UINT volume. Indeed we launch N compute threads, each voxelizing one particle and we do not want any order conflicts when writing. So we rely on atomic operations. The volume is first cleared, then particles are voxelized in each clip map level; finally they are added to the extinction volume to be automatically handled by volumetric shadows later. For accuracy, the uint [0,2048] range maps to [0,1] in float space.
- #54 We have tried many voxelization methods. Point and its blurred versions is just too noisy. Our default voxelization method is trilinear. You can see the shadow is very soft and there is no popping happening. We also have a high quality voxelization where all threads writes all the voxels contained within the particle area of effect.
- #55 We can cast shadows from particle onto everything and it is soft. But this only supports soft shadows though. Also for particles having very strong extinction, the volumetric shadow can still flicker for moving lights for example. The high quality voxelization is a lot more expensive and threads are very divergent. This is an interesting area of investigation. How can we allow to go wide there? Maybe with some hierarchical voxelization pre-pass or on-gpu-compute-work-spawning?
- #56 When running the particle splatting voxelization, we need to have our extinction to be “energy conserving” for each clip map level. Indeed for we will write more often into the same voxels when they are large. And as you can see on the left video, the voxelization result in over shadowed in the coarser cascaded. To solve this we have artists authoring extinction probabilities for a unit cube and this extinction is then divided according to the volume in which it is inserted. You can see on this final video that voxelized particles now generate a consistent shadow whatever cascade it falls into.
- #57 Particles self shadowing and casting shadows on opaque surfaces.