产品58

vLLM-Omni 首日支持华为等联合发布的 AURA 2.0 流式视频模型

原标题:🎉 Day-0 support for AURA 2.0 in vLLM-Omni!

vLLM@vllm_project发布时间 (北京时间)

vLLM-Omni 在 AURA 2.0 发布当天提供 Day-0 支持。AURA 2.0 是 9B 流式视频模型,可自主发起对话、结合实时场景调用工具,并用帧对齐框指向讨论对象。

中文译文尚未完成,先为你展示原文。

🎉 Day-0 support for AURA 2.0 in vLLM-Omni!

A 9B streaming video model that watches the scene, starts relevant conversations, and acts on what it sees.

💬 Model-Initiated Conversation: decide when to speak and what to say, without waiting for a question.
🔧 Stream-Aware Tool Calling: use the live scene and conversation to call tools and bring results back into the dialogue.
🎯 Interactive Visual Grounding: point to the objects being discussed with frame-aligned boxes.

vLLM-Omni powers the pipeline from speech recognition through video-language inference to speech generation. Session and prefix reuse, a fast path for silent turns, and sentence-level speech handoff deliver 344 ms median time to the first reply audio packet in the single-session setup with two ~140 GB accelerators.

AURA 2.0 leads the four aggregate benchmark comparisons shown below: IPIBench, StreamArena, ProactiveVideoQA and PhoStream.

Thanks to the AURA team at @Huawei Research, CUHK MMLab and HKUST!

Project & demos: https://aurateam2026.github.io/Aura-2.0/
Weights on @huggingface: https://huggingface.co/aurateam/AURA_2

多模态AI 视频部署工程产品更新#产品更新#多模态#视频#部署/工程#Hugging Face