跳到正文
arXiv · virtual production· Adrian Azzarelli·精选AI 评分59

论文提出面向虚拟制作的可重打光高斯泼溅方法,用背景图像驱动光照

Relightable Gaussian Splatting for Virtual Production Using Image-Based Illumination

AI 导读

Adrian Azzarelli 在 arXiv 发表论文,提出一种面向虚拟制作的可重打光高斯泼溅框架,用已知背景图像约束重打光过程,避免依赖环境贴图,并把合成简化为背景图像编辑。

推荐理由

论文针对 LED 墙拍摄中光照与背景绑定的问题,提出用高斯泼溅做可重打光的重建,并给出资源占用与帧率数据。

正文 · 原文

View PDF HTML (experimental)

Abstract:Virtual production (VP) use LED walls to provide both background imagery and image-based lighting. While this enables on-set compositing, it couples lighting to background and scene appearance, limiting flexibility for downstream editing. In addition, inverse rendering conventionally relies on physically-based rendering to estimates 3D geometry and lighting, using environment maps. However, these maps are typically low-resolution and assume far-field lighting. In VP, with near-field and high-resolution image-based lighting, this can lead to inaccuracies and introduce complexities when editing. Addressing this, we propose a VP-specific framework for 3D reconstruction and relighting using Gaussian Splatting. This uses the known background imagery to condition the relighting process. This avoids relying on environment maps and reduces compositing to a background-image editing task. To realize our framework, we introduce a process (and associated dataset) that captures real VP scenes under varying background content and illumination conditions. This data is used to decompose a 3D scene into fixed appearance and variable lighting components. The variable lighting process simulates light transport by parameterizing each primitive with a UV coordinate, intensity value and resolution modifier. Using mipmaps, these directly sample the background texture in image space - implicitly capturing reflections and refractions without physically-based rendering. Combined with the fixed appearance component, this allows us to render relit scenes using a Gaussian Splatting rasterizer. Compared to baselines, our approach achieves higher-quality 3D reconstruction and controllable relighting. The method is efficient (<3 GB RAM, <5 GB VRAM, <2 hours training, ~35 FPS) and supports rendering useful arbitrary output variables including depth, lighting intensity, lighting color, and unlit renders.
Subjects: Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR); Multimedia (cs.MM); Image and Video Processing (eess.IV)
Cite as: arXiv:2605.09024 [cs.CV]
  (or arXiv:2605.09024v1 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2605.09024

arXiv-issued DOI via DataCite

Submission history

From: Adrian Azzarelli [view email]
[v1] Sat, 9 May 2026 16:04:06 UTC (36,813 KB)

来源:arXiv · virtual production · arxiv.org