High-performance CPUs and GPUs are fundamental to AI infrastructure, but their physical constraints are often underestimated. These components are not just powerful, they are also power-hungry, converting nearly every watt of electricity they consume into heat. Failing to plan for power delivery and heat dissipation is a common misstep that leads to thermal throttling, system instability, and even premature hardware failure. An on-premise system's reliability is built as much on its electrical and cooling infrastructure as it is on its silicon.
The first step in planning is to estimate the total power your server will draw under a heavy machine learning workload. A component's Thermal Design Power (TDP) is a good starting point for this calculation. While TDP technically measures the maximum heat a component's cooling system is designed to dissipate, it serves as a reliable estimate for its power consumption under sustained load.
To calculate your server's total power requirement, you must sum the TDP of all major components.
Let's calculate the power budget for a common AI server configuration:
The total estimated power draw for the components is:
This 2200W represents the direct current (DC) power the components need. However, the server's Power Supply Unit (PSU) draws alternating current (AC) from the wall and converts it to DC, a process that is not 100% efficient. High-quality server PSUs have an "80 Plus" rating, with "Platinum" or "Titanium" ratings indicating efficiencies of 90% or higher.
If our server has a PSU that is 90% efficient, the actual power drawn from the wall will be:
This is the number you must use for planning your electrical circuits. For production systems, always use redundant PSUs. If one fails, the other can handle the full load, preventing an unexpected shutdown during a long training run.
A common mistake is assuming a standard wall outlet can handle a powerful AI server. In North America, a typical household circuit is 120 volts (V) and is protected by a 15-ampere (A) circuit breaker. The maximum continuous power you can safely draw is 80% of the circuit's maximum, which is:
Our example server, drawing 2444W, would immediately trip this breaker. Even a 20A circuit () is insufficient.
This is why data centers and dedicated server rooms use higher-voltage circuits, typically 208V or 240V. A 208V, 20A circuit can provide:
This provides a safe margin for our 2444W server. When building an on-premise facility, you must work with an electrician to install the appropriate high-power outlets. For server racks, you will use a Power Distribution Unit (PDU), which is essentially a power strip designed to mount in a rack and distribute power from a high-amperage wall circuit to multiple servers.
Power flow from the wall outlet to server components, and the resulting heat generation that must be managed by a cooling system.
The laws of thermodynamics are unforgiving: nearly all the electricity a server consumes is converted into heat. This heat must be removed from the server and the room to prevent components from overheating. The standard unit for measuring heat is the British Thermal Unit (BTU).
The conversion from watts to BTUs per hour is straightforward:
Using the 2444W drawn from the wall, we can calculate our server's heat output:
To put this in perspective, a small personal space heater produces about 5,000 BTU/hr. Our single AI server generates significantly more heat than that. A standard office HVAC system is not designed to handle this kind of concentrated heat load. Placing such a server in a small, unventilated room will quickly raise the ambient temperature to levels that cause hardware to throttle its performance or shut down entirely.
Effective cooling requires a two-level approach: managing airflow inside the server and managing the temperature of the room itself.
Server chassis are engineered for specific airflow patterns, usually front-to-back. Cool air is drawn in from the front, passes over the components, and the hot exhaust is expelled out the back. It is important not to obstruct these vents. High-power servers use high-speed, high-pressure fans (measured in Cubic Feet per Minute, or CFM) that are much louder than those in a desktop PC.
A dedicated, portable AC unit or a "mini-split" system is often necessary for even a single AI server. The cooling capacity of these units is rated in BTU/hr. For our 8339 BTU/hr server, you would need an air conditioner rated for at least 10,000 BTU/hr to provide a safe margin.
For multiple servers in a rack, data center best practices become relevant. The hot aisle/cold aisle layout is a simple and effective strategy. Racks are arranged in rows where all server fronts (air intakes) face one aisle (the cold aisle), and all server backs (hot exhausts) face the other (the hot aisle). This prevents the servers from drawing in hot exhaust air from their neighbors, dramatically improving cooling efficiency.
Ultimately, power and cooling are foundational to your on-premise infrastructure. They are not just operational details but core design requirements that directly impact performance, reliability, and the total cost of ownership of your system.
Was this section helpful?
© 2026 ApX Machine LearningAI Ethics & Transparency•