跳到正文
arXiv · virtual production· Yuanjian He·精选AI 评分58

CineMatte:面向虚拟制作的前景抠像框架与 CineMatte-4K 数据集

CineMatte: Background Matting for Virtual Production and Beyond

AI 导读

Yuanjian He 提出 CineMatte 前景抠像框架,用共享权重的冻结 DINOv3 ViT 分别编码输入帧与背景,再经交叉注意力预测前景,以降低对背景变化的敏感度。

推荐理由

论文提出面向 LED 虚拟制作的前景抠像框架,并公开在专业影棚拍摄的 4K HDR 数据集,可供后期替换背景参考。

正文 · 原文

View PDF HTML (experimental)

Abstract:LED Virtual Production (VP) uses large LED volumes to render backgrounds in real time, enabling in-camera visual effects but making post-shot changes labor-intensive. We address this with CineMatte, a robust background matting framework for VP and beyond. CineMatte employs a cross-attention-conditioned design. Instead of concatenating the background with the input, CineMatte employs a Siamese, frozen DINOv3 Vision Transformer with shared weights to encode the input frame and the captured background separately. A cross-attention module compares the two streams to predict the foreground, preserving pretrained semantics and improving robustness to background shifts. Previous ViT-based matting models use a parallel convolutional "detail branch" to recover fine details, which can cause boundary artifacts in real-world samples due to semantic misalignment with the backbone. We instead replace it with a pretrained, image-guided feature upsampler, which largely mitigates the problem. We also introduce CineMatte-4K, a 4K HDR image-video dataset captured on a professional LED VP stage. To the best of our knowledge, the image subset is the first dataset for VP matting and is non-synthetic, obtained via green-screen insertion; the video subset includes camera motion with tracked trajectories so that arbitrary backgrounds can be rendered later with correct parallax. Across CineMatte-4K and public benchmarks (VideoMatte240K, YouTubeMatte), CineMatte not only excels in VP but also generalizes robustly to real-world footage.
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Cite as: arXiv:2605.18328 [cs.CV]
  (or arXiv:2605.18328v1 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2605.18328

arXiv-issued DOI via DataCite

Submission history

From: Yuanjian He [view email]
[v1] Mon, 18 May 2026 12:45:27 UTC (3,728 KB)

来源:arXiv · virtual production · arxiv.org