The AMD Instinct MI300X is AMD's flagship datacenter accelerator based on the CDNA 3 architecture, announced in June 2023 and shipping in Q4 2023. It represents AMD's most competitive offering against NVIDIA in the AI training and inference market, featuring the largest memory capacity of any GPU accelerator at launch.
The MI300X uses a revolutionary chiplet-based design with 12 chiplets (8 XCD compute dies + 4 I/O dies) manufactured on TSMC's 5nm and 6nm processes, containing 153 billion transistors total. The package integrates 192GB of HBM3 memory across 8 stacks with 5.3 TB/s aggregate bandwidth.
The CDNA 3 architecture includes 304 compute units with 19,456 stream processors optimized for matrix operations. The Matrix Core units support FP8, FP16, BF16, FP32, and FP64 operations with competitive throughput to NVIDIA Tensor Cores. Native support for sparsity provides up to 2x throughput for sparse models.
The 192GB memory capacity is the MI300X's primary differentiator, enabling inference of large language models that would require multiple H100 GPUs. TDP is 750W, requiring liquid cooling for most deployments. The MI300X is deployed by major cloud providers (Microsoft Azure, Oracle Cloud) and enterprises seeking NVIDIA alternatives.