GB200.

16 nodes × 4 accelerators = 64 total

Topology: 5th-gen NVLink and NVLink Switches
Software: TensorRT 10.14, CUDA 13.1

deepseek-r1 · Offline: 468,665 tokens/s · source

deepseek-r1 · Server: 311,531 tokens/s · source

gpt-oss-120b · Offline: 774,539 tokens/s · source

gpt-oss-120b · Server: 743,550 tokens/s · source

18 nodes × 4 accelerators = 72 total

Topology: 5th-gen NVLink andNVLink Switches
Software: TensorRT 10.14, CUDA 13.1

deepseek-r1 · Offline: 307,874 tokens/s · source

deepseek-r1 · Server: 156,781 tokens/s · source

18 nodes × 4 accelerators = 72 total

Topology: 5th-gen NVLink andNVLink Switches
Software: TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentML:mlperf-inf-mm-q3vl-v6.0

deepseek-r1 · Interactive: 240,318 tokens/s · source

deepseek-r1 · Offline: 486,141 tokens/s · source

deepseek-r1 · Server: 336,106 tokens/s · source

gpt-oss-120b · Interactive: 624,929 tokens/s · source

gpt-oss-120b · Offline: 879,542 tokens/s · source

gpt-oss-120b · Server: 899,218 tokens/s · source