Runway Labs 推出 Diffusion-UI:实时生成可交互界面的研究应用
Runway Labs Introduces Diffusion-UI, a Research App That Generates Interactive Interfaces in Real Time and Can Use Structured Data
Runway Labs 推出研究应用 Diffusion-UI,由实时模型逐像素生成可交互界面,并演示了能使用结构化数据、维持状态和生成控件的早期版本。精确地图路线、实时天气或任意 API 可作为 JSON 对象传入,界面按数据源、参考布局、样式和文本提示四层生成,交互时逐帧渲染,悬停时以白色轮廓标出可点击控件。该应用仍属早期研究,未公布定价、访问方式或发布时间,也未给出分辨率、帧率与延迟数据。
VP Land
Nov 19, 2026 · 2 min read
Runway Labs introduced Diffusion-UI, a research application in which a real-time model generates every pixel of an interactive interface, and shared an early look at generated interfaces that can use structured data, maintain states and generate controls. Precise map routes, live weather or any API can be passed in as JSON objects, which gives the model specific information to render a dynamic interface.
- Four layers drive each generation. Runway Labs says Diffusion-UI generates using data sources, reference layouts, styles and text prompts.
- Scenes are laid out by moving a box. Generated interfaces render each frame as you interact, so you can design a scene by moving and resizing a bounding box.
- Text prompts edit the scene live. You can type a prompt to change the scene, add objects or edit anything in real time.
- The model highlights its own controls. On hover, white outlines highlight nearby buttons, sliders and other controls to show which elements are interactive.
- It is an early research application. Runway Labs calls Diffusion-UI a research application and an early look, and the thread shares no pricing, access or release details.
Diffusion-UI's demo clips cover live data, generated controls and 3D game scenes
In the thread, Runway Labs shows each of these abilities in short demo clips, including a generated word-tile board.
Runway Labs also says Diffusion-UI is versatile enough to render complex 3D interfaces, pointing to immersive game scenes as an example.
The thread presents all of this as demo clips. It gives no resolution, frame rate or latency figures, and it doesn't name the model behind Diffusion-UI.
Runway Labs turns Runway's video and world models into prototypes
Diffusion-UI comes out of Runway Labs, an incubator inside Runway that turns video and world models into prototypes people can use.
Runway launched the Labs incubator to explore new applications for generative video and General World Models. It named Co-founder and Chief Innovation Officer Alejandro Matamala Ortiz to lead it with a team dedicated to incubating experimental products.
The thread closes with Runway Labs saying it is exploring tools that make composing and refining real-time interfaces "feel direct and familiar," with more prototypes and tools to share.
Runway's Solaris study favored generated interfaces over coded ones, with text and accessibility still unsolved
Runway has worked on model-generated interfaces before. Its Solaris research post introduced Interface World Models, real-time models that generate the interface itself frame by frame with no code underneath.
Solaris builds on Runway's Gen-4.5 video architecture and generates 720p frames in under 500 milliseconds, and in a 250-person Runway study participants preferred it over Claude Opus 5-coded interfaces 61% to 24% for following the requested interaction, CMSWire reported. For natural behavior within the scene, the split was 71% to 21%.
Runway also acknowledged open problems for Solaris in stable, legible text and long-session coherence, plus accessibility, since a generated interface has no DOM for screen readers to hook into.
The Diffusion-UI thread doesn't say how the new application handles those limits.
来源:VP Land · vp-land.com