Deep Real-time Volumetric Rendering Using Multi-feature Fusion
In Special Interest Group on Computer Graphics and Interactive Techniques Conference Proceedings (SIGGRAPH '23 Conference Proceedings)
August 06–10, 2023, Los Angeles, CA, USA. ACM, New York, NY, USA, Article No.: 61, Pages 1–10.
Jinkai Hu, Chengzhong Yu, Hongli Liu, Lingqi Yan, Yiqian Wu, Xiaogang Jin
Multi-feature RPNN (MRPNN) renders multi-scattered cloud illumination in real time at 1024×1024 resolution, producing results close to the ground truth (a). Thanks to its novel network design, MRPNN supports configurable shading parameters (b), while the prior work only accepts fixed ones. 𝐺 configures Henyey-Greenstein phase function. In addition, our network correctly handles the shadow boundary (c), which was previously a failure case.
Abstract
We present
Multi-feature Radiance-Predicting Neural Networks (MRPNN), a practical
framework with a lightweight feature fusion neural network for rendering
high-order scattered radiance of participating media in real time. By
reformulating the Radiative Transfer Equation (RTE) through theoretical
examination, we propose transmittance fields, generated at a low cost, as
auxiliary information to help the network better approximate the RTE,
drastically reducing the size of the neural network. The light weight
network efficiently estimates the difficult-to-solve in-scattering term and
allows for configurable shading parameters while improving prediction
accuracy. In addition, we propose a frequency-sensitive stencil design in
order to handle non-cloud shapes, resulting in accurate shadow boundaries.
Results show that our MRPNN is able to synthesize indistinguishable output
compared to the ground truth. Most importantly, MRPNN achieves a speedup of
two orders of magnitude compared to the state-of-the-art, and is able to
render high-quality participating material in real time.