Building a Bare-Metal AI Server

Transforming a list of carefully selected components into a functional server requires a methodical assembly and configuration process. While similar to building a standard desktop computer, constructing a bare-metal AI server involves specific considerations for handling high-power GPUs, ensuring maximum data throughput, and configuring the system for stability under heavy, continuous load. Here are the important steps, from physical assembly to the initial software setup.

A Pre-Flight Check for Compatibility

Before you even pick up a screwdriver, perform a final compatibility check. A mistake here can lead to purchasing incorrect parts or discovering that components physically interfere with each other inside the chassis.

• Motherboard and CPU: Verify the CPU socket type (e.g., LGA 1700, AM5) matches the motherboard.
• Motherboard and GPUs: This is a frequent point of error. Confirm that the physical spacing of the motherboard's x16 PCIe slots can accommodate the width of your GPUs. Many high-performance GPUs occupy three or even four slots, meaning a motherboard with four x16 slots might only fit two such cards.
• Chassis and Components: Check the case specifications for maximum GPU length and CPU cooler height. If you are using a liquid cooling AIO, ensure the chassis has mounting points for the chosen radiator size (e.g., 240mm, 360mm).
• Power Supply Unit (PSU): Examine the total wattage. Count the number of available PCIe 8-pin or 12VHPWR power connectors on the PSU and compare it to the requirements of your GPUs. A server with four GPUs requiring two 8-pin connectors each will need eight dedicated connectors from the PSU.

The Assembly Workflow

The assembly process should follow a logical order to avoid having to uninstall components to make room for others. The following workflow highlights the critical stages for an AI server build.

A typical assembly and configuration sequence for a bare-metal AI server.

The most sensitive parts of this process for an AI server are the GPU installation and subsequent cable management. When installing multiple GPUs, seat them firmly in their PCIe slots one by one. If using NVLink, connect the bridge after the GPUs are secured. Pay close attention to cable routing. Poor cable management is not just an aesthetic issue; it can significantly impede airflow, which is essential for preventing the thermal throttling of your GPUs during long training runs.

Essential BIOS/UEFI Configuration

Before installing the operating system, you must configure a few settings in the motherboard's BIOS/UEFI. These are not optional for a multi-GPU system to function correctly.

• Enable Above 4G Decoding: This setting allows a 64-bit operating system to properly map the large memory address space required by multiple GPUs. Without it, the system may fail to boot or only recognize a single GPU.
• Enable Resizable BAR (Base Address Register): A related feature, ReBAR allows the CPU to access the entire GPU frame buffer (VRAM) at once, rather than in smaller 256MB chunks. This can provide a performance uplift in certain workloads.
• Set PCIe Link Speeds: Find the settings for your PCIe slots and ensure they are not set to a lower generation (e.g., "Gen 3.0" when the slot and GPU both support "Gen 4.0"). Forcing the highest supported speed ensures you are not creating an unnecessary data bottleneck between the CPU and GPUs.

OS and Driver Installation

A stable, minimal operating system is the best foundation. A Long-Term Support (LTS) release of a Linux distribution like Ubuntu Server is a common and reliable choice. Once the OS is installed, the single most important software step is the installation of the proprietary NVIDIA drivers. These drivers contain the CUDA toolkit, which is necessary for machine learning frameworks to utilize the GPUs.

After installing the drivers, you can verify that the system correctly recognizes all hardware. The primary tool for this is the NVIDIA System Management Interface. Open a terminal and run the following command:

nvidia-smi

A successful build will produce output similar to this, confirming that all GPUs are detected, their temperatures are normal, and they are ready for work.

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.65.01    Driver Version: 515.65.01    CUDA Version: 11.7     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA RTX 3090      On   | 00000000:01:00.0 Off |                  N/A |
| 30%   35C    P8    23W / 350W |      2MiB / 24576MiB |      0%      Default |
|                               |                      |                  N/A |
|-------------------------------+----------------------+----------------------+
|   1  NVIDIA RTX 3090      On   | 00000000:21:00.0 Off |                  N/A |
| 30%   34C    P8    21W / 350W |      2MiB / 24576MiB |      0%      Default |
|                               |                      |                  N/A |
+-----------------------------------------------------------------------------+

Seeing all your installed GPUs in this list is the final confirmation of a successful hardware build. The machine is now a blank canvas, ready for the software stack, including Docker and Kubernetes, that you will use to run your machine learning workloads.