16 nodes × 4 accelerators = 64 total

Topology: 5th-gen NVLink and NVLink Switches
Software: TensorRT 10.14, CUDA 13.1

gpt-oss-120b · Offline: 910,468 tokens/s · source

gpt-oss-120b · Server: 956,523 tokens/s · source

2 nodes × 4 accelerators = 8 total

Topology: 5th-gen NVLink and NVLink Switches
Software: TensorRT 10.14, CUDA 13.1

deepseek-r1 · Offline: 78,564 tokens/s · source

deepseek-r1 · Server: 44,589.7 tokens/s · source

1 nodes × 4 accelerators = 4 total

Topology: 5th-gen NVLink andNVLink Switches
Software: TensorRT 10.14, CUDA 13.1

gpt-oss-120b · Offline: 60,220.4 tokens/s · source

gpt-oss-120b · Server: 53,462.7 tokens/s · source

1 nodes × 1 accelerators = 1 total

Topology: 5th-gen NVLink and NVLink Switches
Software: TensorRT 10.14, CUDA 13.1, cuDNN 9.17, Driver 580.105.08, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentML:mlperf-inf-mm-q3vl-v6.0

gpt-oss-120b · Offline: 15,371.7 tokens/s · source

gpt-oss-120b · Server: 14,973.1 tokens/s · source

18 nodes × 4 accelerators = 72 total

Topology: 5th-gen NVLink and NVLink Switches
Software: TensorRT 10.14, CUDA 13.1, cuDNN 9.17, Driver 580.105.08, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentML:mlperf-inf-mm-q3vl-v6.0

deepseek-r1 · Offline: 673,936 tokens/s · source

deepseek-r1 · Server: 575,580 tokens/s · source

gpt-oss-120b · Offline: 1,046,150 tokens/s · source

gpt-oss-120b · Server: 1,096,770 tokens/s · source

2 nodes × 4 accelerators = 8 total

Topology: 5th-gen NVLink and NVLink Switches
Software: TensorRT 10.14, CUDA 13.1, cuDNN 9.17, Driver 580.105.08, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentML:mlperf-inf-mm-q3vl-v6.0

deepseek-r1 · Offline: 76,346.6 tokens/s · source

deepseek-r1 · Server: 64,510 tokens/s · source

72 nodes × 4 accelerators = 288 total

Topology: 5th-gen NVLink andNVLink Switches
Software: TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentML:mlperf-inf-mm-q3vl-v6.0

deepseek-r1 · Offline: 2,494,310 tokens/s · source

deepseek-r1 · Server: 1,555,110 tokens/s · source

18 nodes × 4 accelerators = 72 total

Topology: 5th-gen NVLink andNVLink Switches
Software: TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentML:mlperf-inf-mm-q3vl-v6.0

deepseek-r1 · Interactive: 250,634 tokens/s · source

deepseek-r1 · Offline: 647,076 tokens/s · source

deepseek-r1 · Server: 400,437 tokens/s · source

gpt-oss-120b · Interactive: 677,199 tokens/s · source

gpt-oss-120b · Offline: 1,042,980 tokens/s · source

gpt-oss-120b · Server: 1,072,250 tokens/s · source