arXiv · virtual production· Yuan-Yi Fan·AI 评分32
LLM4OSC:面向 Open Sound Control 的配置文件约束自然语言控制与确定性校验
LLM4OSC: Profile-Bound Natural Language Control with Deterministic Validation for Open Sound Control
AI 导读
LLM4OSC 采用本地优先架构,让模型在经人工审核的设备配置文件上生成结构化意图 JSON,再由确定性代码校验、钳制并编码后才通过 UDP 发送。在 Max/MSP hero 配置文件(12 个模式:8 个字面、8 个改写、4 个拒答案例)上,B0–B3 后端均通过冻结评测门槛,达到 100% 语义准确率与 0% 错误发送率。
正文 · 原文
Abstract:Open Sound Control (OSC) is the dominant wire protocol for real-time parametric control in professional audio, live performance, and virtual production. Large language models can emit plausible OSC, but they hallucinate addresses, mishandle type tags, and fail under paraphrase- unacceptable in show-critical contexts. We present LLM4OSC, a local-first architecture in which models propose structured intent JSON over a human-reviewed device profile, and deterministic code validates, clamps, and encodes before any UDP send. We introduce a frozen evaluation harness with CI gates on wrong-send rate: mismatches that would still pass validation and transmit. On a Max/MSP hero profile (12 patterns; 8 literal + 8 paraphrase + 4 refusal cases), after profile tag enrichment, symbolic slot fill, NL refine, and a retrieval confidence gate, backends B0--B3 all pass frozen gates (100% semantic accuracy, 0% wrong-send). B0 (rules) remains the production default at ~0.05ms; LLM backends remain ~3-4s. Historical few-shot B2 accuracy of 62.5% rises to 100% on this suite only after symbolic post-processing- not because the 0.5B model alone becomes show-safe. We argue for propose-validate-send and wrong-send rate as first-class metrics for language-to-control systems.
| Subjects: | Human-Computer Interaction (cs.HC); Sound (cs.SD); Audio and Speech Processing (eess.AS) |
| Cite as: | arXiv:2607.26024 [cs.HC] |
| (or arXiv:2607.26024v1 [cs.HC] for this version) | |
| https://doi.org/10.48550/arXiv.2607.26024 arXiv-issued DOI via DataCite |
Submission history
From: Yuan-Yi Fan [view email]
[v1]
Tue, 28 Jul 2026 17:36:22 UTC (7 KB)
来源:arXiv · virtual production · arxiv.org