AMD Instinct MI100#
The AMD Instinct™ MI100 is a data-center compute PCIe-form-factor GPU. This document provides MI100-specific prerequisites, health checks, validation steps, and performance acceptance criteria.
Overview#
The AMD Instinct MI100 introduces the first-generation CDNA architecture in a standard full-height, full-length, dual-slot PCIe® add-in card aimed at HPC and accelerated computing workloads. Each MI100 provides 120 compute units with Matrix Core technology, 32 GB of HBM2 memory at up to 1.2 TB/s, and AMD Infinity Fabric™ link support for direct GPU-to-GPU connectivity in 2- and 4-GPU hive configurations. The card is passively cooled with a 300 W TDP and supports PCIe® Gen4 host connectivity.
The MI100 is built on the CDNA architecture (gfx908) with 120 compute units and 32 GB of HBM2 memory per GPU. The MI100 Infinity Fabric™ topology tops out at 4 GPUs per hive, so the validation reference configuration for this document is a single 4-GPU MI100 hive with Infinity Fabric™ bridges providing direct GPU-to-GPU connectivity across all peers. Larger deployments (for example, dual-socket servers with two 4-GPU hives for 8 MI100s total) are common; in those systems, cross-hive traffic traverses the host PCIe fabric and the per-hive criteria below apply to each hive independently.
System requirements#
Operating system support#
For the most up-to-date information on supported operating systems and distributions, refer to the official ROCm documentation:
ROCm System Requirements - Supported Distributions
Note
ROCm docs is the single source of truth for supported versions, distribution compatibility, and required dependencies for the ROCm toolkit.
For general NUMA and OS-level tuning that applies to all AMD Instinct hosts, see OS tuning. MI100-specific BIOS settings are in the BIOS settings section below.
GPU identification#
All MI100 GPUs (PCI vendor:device 1002:738c) should appear in lspci output:
sudo lspci -d 1002:738c
Expected output example (4-GPU MI100 hive):
1d:00.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Arcturus GL-XL [Instinct MI100] (rev 01)
20:00.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Arcturus GL-XL [Instinct MI100] (rev 01)
23:00.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Arcturus GL-XL [Instinct MI100] (rev 01)
26:00.0 Display controller: Advanced Micro Devices, Inc. [AMD/ATI] Arcturus GL-XL [Instinct MI100] (rev 01)
BIOS settings#
For maximum MI100 GPU performance on systems with AMD EPYC™ 7002-series processors (codename “Rome”) and AMI System BIOS, the following BIOS settings have been validated. These settings should be set as default values in the system BIOS. Analogous settings for other non-AMI BIOS providers can be configured similarly. On systems with Intel processors, some settings may not apply.
Note
The BIOS setting locations and names may vary by hardware and BIOS vendor. Consult your system documentation for details.
BIOS setting location |
Parameter |
Value |
Comments |
|---|---|---|---|
Advanced / PCI subsystem settings |
Above 4G decoding |
Enabled |
GPU large BAR support. |
AMD CBS / CPU common options |
Global C-state control |
Auto |
Global C-states. |
AMD CBS / CPU common options |
CCD/Core/Thread enablement |
Accept |
May be necessary to enable the SMT control menu. |
AMD CBS / CPU common options / performance |
SMT control |
Disable |
Set to Auto if the primary application is not compute-bound. |
AMD CBS / DF common options / memory addressing |
NUMA nodes per socket |
NPS 1, 2, or 4 |
See NPS memory configuration below. |
AMD CBS / DF common options / memory addressing |
Memory interleaving |
Auto |
Depends on NPS setting. |
AMD CBS / DF common options / link |
4-link xGMI max speed |
18 Gbps |
Set to highest rate supported by the CPU. |
AMD CBS / DF common options / link |
3-link xGMI max speed |
18 Gbps |
Set to highest rate supported by the CPU. |
AMD CBS / NBIO common options |
IOMMU |
Disabled |
|
AMD CBS / NBIO common options |
PCIe ten bit tag support |
Enabled |
|
AMD CBS / NBIO common options |
Preferred IO |
Manual |
|
AMD CBS / NBIO common options |
Preferred IO bus |
Use |
|
AMD CBS / NBIO common options |
Enhanced Preferred IO mode |
Enabled |
|
AMD CBS / NBIO common options / SMU common options |
Determinism control |
Manual |
|
AMD CBS / NBIO common options / SMU common options |
Determinism slider |
Power |
|
AMD CBS / NBIO common options / SMU common options |
cTDP control |
Manual |
|
AMD CBS / NBIO common options / SMU common options |
cTDP |
240 |
Value in watts. |
AMD CBS / NBIO common options / SMU common options |
Package power limit control |
Manual |