arXiv · virtual production· Yuanjian He·精选AI 评分58
CineMatte:面向虚拟制作的前景抠像框架与 CineMatte-4K 数据集
CineMatte: Background Matting for Virtual Production and Beyond
AI 导读
Yuanjian He 提出 CineMatte 前景抠像框架,用共享权重的冻结 DINOv3 ViT 分别编码输入帧与背景,再经交叉注意力预测前景,以降低对背景变化的敏感度。
推荐理由
论文提出面向 LED 虚拟制作的前景抠像框架,并公开在专业影棚拍摄的 4K HDR 数据集,可供后期替换背景参考。
正文 · 原文
Abstract:LED Virtual Production (VP) uses large LED volumes to render backgrounds in real time, enabling in-camera visual effects but making post-shot changes labor-intensive. We address this with CineMatte, a robust background matting framework for VP and beyond. CineMatte employs a cross-attention-conditioned design. Instead of concatenating the background with the input, CineMatte employs a Siamese, frozen DINOv3 Vision Transformer with shared weights to encode the input frame and the captured background separately. A cross-attention module compares the two streams to predict the foreground, preserving pretrained semantics and improving robustness to background shifts. Previous ViT-based matting models use a parallel convolutional "detail branch" to recover fine details, which can cause boundary artifacts in real-world samples due to semantic misalignment with the backbone. We instead replace it with a pretrained, image-guided feature upsampler, which largely mitigates the problem. We also introduce CineMatte-4K, a 4K HDR image-video dataset captured on a professional LED VP stage. To the best of our knowledge, the image subset is the first dataset for VP matting and is non-synthetic, obtained via green-screen insertion; the video subset includes camera motion with tracked trajectories so that arbitrary backgrounds can be rendered later with correct parallax. Across CineMatte-4K and public benchmarks (VideoMatte240K, YouTubeMatte), CineMatte not only excels in VP but also generalizes robustly to real-world footage.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2605.18328 [cs.CV] |
| (or arXiv:2605.18328v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2605.18328 arXiv-issued DOI via DataCite |
Submission history
From: Yuanjian He [view email]
[v1]
Mon, 18 May 2026 12:45:27 UTC (3,728 KB)
来源:arXiv · virtual production · arxiv.org