LOADING DATASET
Opening the index…
Resolving the selected comparison slice.
Resolving the selected comparison slice.
NVIDIA B300-SXM-270GB · 270 GB · SXM
Logical ID accelerator:nvidia-b300-sxm-270-gb
Content version e7190bee6641c38932bf397020c925f7f1c006f22af7bf99fb6b1193ffa38f41
1 nodes × 8 accelerators = 8 total
Topology: 18x 5th Gen NVLink, 14.4 TB/s aggregated bandwidth
Software: TensorRT 10.14, CUDA 13.1
gpt-oss-120b · Interactive: 26,005.8 tokens/s · source
gpt-oss-120b · Offline: 110,077 tokens/s · source
gpt-oss-120b · Server: 100,656 tokens/s · source
llama3.1-8b · Offline: 166,745 tokens/s · source
llama3.1-8b · Server: 148,067 tokens/s · source
1 nodes × 8 accelerators = 8 total
Topology: 5th Gen NVLink and NVLink Switches
Software: TensorRT 10.13, CUDA 13.0
deepseek-r1 · Offline: 69,251.4 tokens/s · source
deepseek-r1 · Server: 58,553.3 tokens/s · source
gpt-oss-120b · Offline: 111,496 tokens/s · source
gpt-oss-120b · Server: 110,655 tokens/s · source
1 nodes × 8 accelerators = 8 total
Topology: 18x 5th Gen NVLink, 14.4 TB/s aggregated bandwidth
Software: TensorRT 10.14, CUDA 13.0
deepseek-r1 · Offline: 69,021.2 tokens/s · source
deepseek-r1 · Server: 46,513.1 tokens/s · source
1 nodes × 8 accelerators = 8 total
Topology: 18x 5th Gen NVLink, 14.4 TB/s aggregated bandwidth
Software: TensorRT 10.14, CUDA 13.1
deepseek-r1 · Offline: 66,287.9 tokens/s · source
deepseek-r1 · Server: 46,510.6 tokens/s · source
1 nodes × 8 accelerators = 8 total
Topology: 18x 5th Gen NVLink, 14.4 TB/s aggregated bandwidth
Software: TensorRT 10.14, CUDA 13.1
deepseek-r1 · Interactive: 4,934.65 tokens/s · source
deepseek-r1 · Offline: 69,617.4 tokens/s · source
deepseek-r1 · Server: 35,552.6 tokens/s · source
gpt-oss-120b · Offline: 107,696 tokens/s · source
gpt-oss-120b · Server: 106,397 tokens/s · source
llama3.1-8b · Interactive: 128,633 tokens/s · source
llama3.1-8b · Offline: 150,707 tokens/s · source
llama3.1-8b · Server: 141,484 tokens/s · source
1 nodes × 8 accelerators = 8 total
Topology: 18x 5th Gen NVLink, 14.4 TB/s aggregated bandwidth
Software: TensorRT 10.14, CUDA 13.1
gpt-oss-120b · Offline: 109,141 tokens/s · source
gpt-oss-120b · Server: 100,687 tokens/s · source
1 nodes × 8 accelerators = 8 total
Topology: 18x 5th Gen NVLink, 14.4 TB/s aggregated bandwidth
Software: TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentML:mlperf-inf-mm-q3vl-v6.0
deepseek-r1 · Offline: 69,318.9 tokens/s · source
deepseek-r1 · Server: 60,413.4 tokens/s · source
1 nodes × 8 accelerators = 8 total
Topology: 18x 5th Gen NVLink, 14.4 TB/s aggregated bandwidth
Software: TensorRT 10.14, CUDA 13.1, cuDNN 9.17, TensorRT-LLM feat/1.2-mlpinf, NVIDIA Dynamo mlperf-v6.0-dynamo-v0.8.0, vLLM CentML:mlperf-inf-mm-q3vl-v6.0
deepseek-r1 · Offline: 70,325.7 tokens/s · source
deepseek-r1 · Server: 42,721.4 tokens/s · source
gpt-oss-120b · Offline: 103,961 tokens/s · source
gpt-oss-120b · Server: 100,328 tokens/s · source
llama3.1-8b · Offline: 165,432 tokens/s · source
llama3.1-8b · Server: 141,565 tokens/s · source
1 nodes × 8 accelerators = 8 total
Topology: 18x 5th Gen NVLink, 14.4 TB/s aggregated bandwidth
Software: TensorRT 10.14, CUDA 13.1
deepseek-r1 · Offline: 69,263.2 tokens/s · source
deepseek-r1 · Server: 35,435.7 tokens/s · source