AMD says its MI300X AI accelerator is quicker than Nvidia's H100

A hot potato: AMD is combating back at Nvidia's claims about the H100 GPU accelerator, which in keeping with Team Green is quicker than the opposition. But Team Red said Nvidia didn't tell the complete story, and provided similarly benchmark outcomes with enterprise-widespread inferencing workloads.

AMD has eventually launched its Instinct MI300X accelerators, a new era of server GPUs designed to offer compelling overall performance stages for generative AI workloads and different excessive-performance computing (HPC) applications. MI300X is faster than H100, AMD stated in advance this month, however Nvidia attempted to refute the competitor's statements with new benchmarks launched multiple days in the past.

Nvidia examined its H100 accelerators with TensorRT-LLM, an open-source library and SDK designed to efficiently accelerate generative AI algorithms. According to the GPU business enterprise, TensorRT-LLM turned into able to run 2x faster on H100 than on AMD's MI300X with right optimizations.

AMD is now providing its own model of the tale, refuting Nvidia's statements about H100 superiority. Nvidia used TensorRT-LLM on H100, instead of vLLM utilized in AMD benchmarks, at the same time as evaluating overall performance of FP16 datatype on AMD Instinct MI300X to FP8 datatype on H100. Furthermore, Team Green inverted AMD's posted overall performance data from relative latency numbers to absolute throughput.

AMD shows that Nvidia tried to rig the game, at the same time as it is still busy figuring out new paths to unlock performance and uncooked electricity on Instinct MI300 accelerators. The company provided the cutting-edge overall performance tiers achieved through the Llama 70B chatbot version on MI300X, displaying a fair higher aspect over Nvidia's H100.

By the usage of the vLLM language model for both accelerators, MI300X became able to attain 2.1x the performance of H100 thanks to the modern day optimizations in AMD's software stack (ROCm). The agency highlighted a 1.4x performance advantage over H100 (with equivalent datatype and library setup) earlier in December. VLLM became selected because of its huge adoption within the network and the potential to run on each GPU architectures.

Even while the usage of TensorRT-LLM for H100, and vLLM for MI300X, AMD turned into nonetheless able to offer a 1.3x improvement in latency. When the usage of decrease-precision FP8 and TensorRT-LLM for H100, and better-precision FP16 with vLLM for MI300X, AMD's accelerator became apparently able to reveal a performance advantage in absolute latency.

vLLM does not assist FP8, AMD explained, and FP16 datatype become selected for its popularity. AMD said that its results display how MI300X the use of FP16 is similar to H100 even if the use of its excellent performance settings with FP8 datatype and TensorRT-LLM.