In this explainer
A modern AI rack can look almost disappointingly normal from the aisle: a tall black cabinet, some status lights, a lot of cables. But one current rack-scale system from NVIDIA draws about 120 kilowatts. That single number changes the building around it.
The easiest way to understand why is to stop thinking of a rack as furniture. It is a machine that receives a continuous river of electricity, turns almost all of it into heat, and expects both flows to remain under control every second of the day.
NVIDIA’s DGX GB200 documentation describes a rack with 72 GPUs, compute trays, network switches, power shelves, bus bars, and liquid-cooling manifolds. The company says its power consumption is approximately 120 kW. That is a product-specific rating, not a universal number for every AI cabinet. But it is a useful example of where high-density computing is going.
Power density is the number hiding inside the bigger number
When a new data center is announced, the headline usually focuses on the campus: 100 megawatts, 500 megawatts, perhaps eventually a gigawatt. Those totals matter. Utilities plan around them. Communities debate them. Developers spend years trying to secure them.
Inside the building, however, engineers also care about concentration. A megawatt spread across hundreds of conventional racks is a different problem from a megawatt packed into eight or nine high-density AI racks. The total power can be the same while the cables, airflow, coolant piping, and safety margins are completely different.
Think about rain. An inch falling across an entire town is weather. The same volume forced through one storm drain is a plumbing emergency. Power density is the storm drain problem.
At 120 kW, the rack needs more than an ordinary plug and power strip. Electricity may arrive at the building at utility voltage, pass through transformers and switchgear, cross backup systems, and travel through distribution equipment before it ever reaches the cabinet. Near the rack, large conductors or busways make the final delivery. NVIDIA’s own design uses power shelves to convert alternating current into roughly 50-volt direct current, then distributes it through a bus bar.
Each handoff creates heat, occupies space, and introduces another component that must be monitored. Redundancy adds parallel paths because a data center is not designed like a warehouse where a tripped breaker simply means everyone goes home early.
Every watt becomes somebody’s heat problem
The processors do not make electricity disappear. They use it to move and manipulate information, and the useful work ultimately leaves the hardware as heat. A rack drawing roughly 120 kW therefore creates roughly 120 kW of heat that the cooling system must carry away while it is operating near that load.
That is why the electrical and mechanical designs are really one story. If you add more computing power without adding a way to move the heat, you have not added usable computing capacity. You have built a very expensive space heater.
Traditional server rooms move cool air through the fronts of cabinets and collect hot air from the backs. That approach still matters, including in newer AI systems. But air can only carry so much heat through a practical aisle and cabinet before the fans, ducts, and temperature differences become unwieldy.
Newer high-density systems increasingly bring liquid closer to the chips. In NVIDIA’s rack, liquid runs through manifolds and cold plates attached to the CPUs and GPUs, while components such as networking and storage still use air cooling. This is an important detail: “liquid cooled” does not always mean every component is underwater or that fans disappear. Many designs are hybrids.
The rack loop also does not solve the whole cooling problem. It merely moves heat from the chips into a liquid. That heat must travel through pumps and heat exchangers to a larger facility loop, then finally leave the building through dry coolers, chillers, cooling towers, or another heat-rejection system. The plumbing gets longer. The physics does not change.
Backup power has to cross the same bridge
High density also changes the scale of the backup system. Data centers usually place a short-duration bridge between the utility and standby generation. Batteries or another uninterruptible power supply carry the load for the seconds or minutes needed to ride through a disturbance or start generators.
A dense AI room asks that bridge to carry a great deal of power in a small area. Engineers have to size the batteries, inverters, switchgear, and generators for the intended load and redundancy strategy. They also have to plan what happens during maintenance, when one path is unavailable, and during failures that do not unfold as neatly as a diagram.
This is where the word “capacity” becomes slippery. A building may have enough total utility service on paper but still lack enough distribution or cooling capacity in the exact room where the new racks are supposed to go. The stranded space can look empty and still be unusable.
The rack reshapes the floor plan
Concentrated computing can affect structural loads, ceiling routes, pipe sizes, maintenance clearances, leak detection, and the amount of support equipment outside the white space. The server cabinet is only the visible endpoint.
It also changes construction sequencing. Power and cooling equipment with long lead times may have to be ordered well before the servers. The utility connection can become the schedule. Commissioning becomes a full-system exercise: not only “does the rack turn on?” but “can every power and cooling path behave correctly when something else fails?”
The story is not that one cabinet uses a shocking amount of power. The story is that concentrating that power forces the rest of the facility to become a different kind of machine.
This distinction matters when we talk about the AI buildout. Chip announcements move quickly. Buildings, substations, cooling plants, and utility upgrades do not. You can manufacture a faster processor without instantly manufacturing the electrical path and heat-rejection system it needs.
So when you see a new rack with an impressive GPU count, look one layer outward. Ask how the power reaches it. Ask where the heat goes. Ask which systems are redundant and which equipment sets the schedule. That is where the physical scale of AI becomes visible.
AI may be software when we use it. At the rack, it is electricity, heat, copper, coolant, and a building designed to keep all of them moving.