产品47

vLLM 为 Kimi K3.1 Pro 推出 DSpark 草稿模型

原标题:More accepted tokens. Bounded draft KV memory. 🔥

vLLM@vllm_project发布时间 (北京时间)

vLLM 联合 Inferact 与 Red Hat AI Speculators 团队为 Kimi K3.1 Pro 推出新 DSpark 草稿模型,在 HumanEval 上接受长度较预览版最高提升 30%。

中文译文尚未完成,先为你展示原文。

More accepted tokens. Bounded draft KV memory. 🔥

The new DSpark for @Kimi_Moonshot’s Kimi K3.1 Pro, jointly developed by @Inferact and the @RedHat_AI Speculators team with vLLM + Speculators, brings:

  • Up to 30% higher acceptance length vs. the preview (HumanEval).
  • A 2K sliding window in every draft layer, keeping draft KV memory bounded even at 1M-token context.

Efficient draft architectures need an inference stack that can serve them. vLLM’s redesigned KV management supports P/D disaggregation and different parallelism configurations, bringing sliding-window draft models into production.

Comparison: DSpark k=7, temperature 1.0, top_p 0.95. Acceptance length includes the bonus token.

Weights + vLLM config below. 👇

Kimi / 月之暗面推理能力开源生态部署工程产品更新#产品更新#推理#部署/工程#开源生态

同一件事的其他报道1 篇