Power and Cooling Requirements

High-performance CPUs and GPUs are fundamental to AI infrastructure, but their physical constraints are often underestimated. These components are not just powerful, they are also power-hungry, converting nearly every watt of electricity they consume into heat. Failing to plan for power delivery and heat dissipation is a common misstep that leads to thermal throttling, system instability, and even premature hardware failure. An on-premise system's reliability is built as much on its electrical and cooling infrastructure as it is on its silicon.

Calculating Power Consumption

The first step in planning is to estimate the total power your server will draw under a heavy machine learning workload. A component's Thermal Design Power (TDP) is a good starting point for this calculation. While TDP technically measures the maximum heat a component's cooling system is designed to dissipate, it serves as a reliable estimate for its power consumption under sustained load.

To calculate your server's total power requirement, you must sum the TDP of all major components.

• GPUs: These are typically the most power-intensive components. A high-end data center GPU like an NVIDIA A100 can have a TDP of 400W.
• CPUs: Modern server CPUs also have significant power needs, often ranging from 150W to 280W per processor.
• Motherboard, RAM, and Storage: These components contribute less individually but add up. A safe estimate is around 150-250W for the motherboard, memory, NVMe drives, and system fans combined.

Let's calculate the power budget for a common AI server configuration:

• 4x NVIDIA A100 GPUs: $4 \times 400W = 1600W$
• 2x AMD EPYC CPUs: $2 \times 200W = 400W$
• Motherboard, RAM, Storage, etc.: $\approx 200W$

The total estimated power draw for the components is:

$$1600W + 400W + 200W = 2200W$$

This 2200W represents the direct current (DC) power the components need. However, the server's Power Supply Unit (PSU) draws alternating current (AC) from the wall and converts it to DC, a process that is not 100% efficient. High-quality server PSUs have an "80 Plus" rating, with "Platinum" or "Titanium" ratings indicating efficiencies of 90% or higher.

If our server has a PSU that is 90% efficient, the actual power drawn from the wall will be:

$$\frac{2200W}{0.90} \approx 2444W$$

This is the number you must use for planning your electrical circuits. For production systems, always use redundant PSUs. If one fails, the other can handle the full load, preventing an unexpected shutdown during a long training run.

Planning for Power Delivery

A common mistake is assuming a standard wall outlet can handle a powerful AI server. In North America, a typical household circuit is 120 volts (V) and is protected by a 15-ampere (A) circuit breaker. The maximum continuous power you can safely draw is 80% of the circuit's maximum, which is:

$$120V \times 15A \times 0.80 = 1440W$$

Our example server, drawing 2444W, would immediately trip this breaker. Even a 20A circuit ($120V \times 20A \times 0.80 = 1920W$) is insufficient.

This is why data centers and dedicated server rooms use higher-voltage circuits, typically 208V or 240V. A 208V, 20A circuit can provide:

$$208V \times 20A \times 0.80 = 3328W$$

This provides a safe margin for our 2444W server. When building an on-premise facility, you must work with an electrician to install the appropriate high-power outlets. For server racks, you will use a Power Distribution Unit (PDU), which is essentially a power strip designed to mount in a rack and distribute power from a high-amperage wall circuit to multiple servers.

Power flow from the wall outlet to server components, and the resulting heat generation that must be managed by a cooling system.

Calculating Cooling Requirements

The laws of thermodynamics are unforgiving: nearly all the electricity a server consumes is converted into heat. This heat must be removed from the server and the room to prevent components from overheating. The standard unit for measuring heat is the British Thermal Unit (BTU).

The conversion from watts to BTUs per hour is straightforward:

$$1W \approx 3.412 \text{ BTU/hr}$$

Using the 2444W drawn from the wall, we can calculate our server's heat output:

$$2444W \times 3.412 \text{ BTU/hr} \approx 8339 \text{ BTU/hr}$$

To put this in perspective, a small personal space heater produces about 5,000 BTU/hr. Our single AI server generates significantly more heat than that. A standard office HVAC system is not designed to handle this kind of concentrated heat load. Placing such a server in a small, unventilated room will quickly raise the ambient temperature to levels that cause hardware to throttle its performance or shut down entirely.

Cooling Strategies

Effective cooling requires a two-level approach: managing airflow inside the server and managing the temperature of the room itself.

Server-Level Cooling

Server chassis are engineered for specific airflow patterns, usually front-to-back. Cool air is drawn in from the front, passes over the components, and the hot exhaust is expelled out the back. It is important not to obstruct these vents. High-power servers use high-speed, high-pressure fans (measured in Cubic Feet per Minute, or CFM) that are much louder than those in a desktop PC.

Room-Level Cooling

A dedicated, portable AC unit or a "mini-split" system is often necessary for even a single AI server. The cooling capacity of these units is rated in BTU/hr. For our 8339 BTU/hr server, you would need an air conditioner rated for at least 10,000 BTU/hr to provide a safe margin.

For multiple servers in a rack, data center best practices become relevant. The hot aisle/cold aisle layout is a simple and effective strategy. Racks are arranged in rows where all server fronts (air intakes) face one aisle (the cold aisle), and all server backs (hot exhausts) face the other (the hot aisle). This prevents the servers from drawing in hot exhaust air from their neighbors, dramatically improving cooling efficiency.

Ultimately, power and cooling are foundational to your on-premise infrastructure. They are not just operational details but core design requirements that directly impact performance, reliability, and the total cost of ownership of your system.