产品47
vLLM 为 Kimi K3.1 Pro 推出 DSpark 草稿模型
原标题:More accepted tokens. Bounded draft KV memory. 🔥
vLLM 联合 Inferact 与 Red Hat AI Speculators 团队为 Kimi K3.1 Pro 推出新 DSpark 草稿模型,在 HumanEval 上接受长度较预览版最高提升 30%。
中文译文尚未完成,先为你展示原文。
More accepted tokens. Bounded draft KV memory. 🔥
The new DSpark for @Kimi_Moonshot’s Kimi K3.1 Pro, jointly developed by @Inferact and the @RedHat_AI Speculators team with vLLM + Speculators, brings:
- Up to 30% higher acceptance length vs. the preview (HumanEval).
- A 2K sliding window in every draft layer, keeping draft KV memory bounded even at 1M-token context.
Efficient draft architectures need an inference stack that can serve them. vLLM’s redesigned KV management supports P/D disaggregation and different parallelism configurations, bringing sliding-window draft models into production.
Comparison: DSpark k=7, temperature 1.0, top_p 0.95. Acceptance length includes the bonus token.
Weights + vLLM config below. 👇
