教程42
NVIDIA 发布 GLM-5.3 双 DGX Station 本地部署指南
原标题:.@NVIDIA put together a video walkthrough and deployment guide for running GLM-5.3 NVFP4 locally on two DGX Stations with vLLM.
NVIDIA 为在双 DGX Station 上用 vLLM 本地运行 GLM-5.3 NVFP4 制作了视频演示和部署指南。该方案将全部模型权重保留在 GPU 显存中,通过跨两块 GB300 GPU 的张量并行,经单一 OpenAI 兼容 API 提供服务,并启用 MTP 投机解码、FP8 KV cache 和前缀缓存。
中文译文尚未完成,先为你展示原文。
.@NVIDIA put together a video walkthrough and deployment guide for running GLM-5.3 NVFP4 locally on two DGX Stations with vLLM.
The setup keeps all model weights in GPU memory, using tensor parallelism across two GB300 GPUs to serve requests through a single OpenAI-compatible API. It also enables MTP speculative decoding, FP8 KV cache, and prefix caching.
Watch the walkthrough:
https://www.youtube.com/watch?v=opRXPRnSkxk
Get the deployment guide: https://developer.nvidia.com/blog/when-one-gpu-isnt-enough-serving-glm-5-3-across-two-nvidia-dgx-stations-with-vllm/
