A modern AI rack can look almost disappointingly normal from the aisle: a tall black cabinet, some status lights, a lot of cables. But one current rack-scale system from NVIDIA draws about 120 kilowatts. That single number changes the building around it.

In short: one NVIDIA GB200 NVL72 rack — 72 GPUs and 36 Grace CPUs in a single cabinet — draws about 120 kW, according to NVIDIA's own hardware guide. That is roughly 16 times the average rack in Uptime Institute's 2025 survey (7.5 kW). HPE and Supermicro both rate their versions of the same rack at 132 kW, so 120 kW is the design nominal and facilities should budget above it. Almost all of that power becomes heat: about 115 kW is removed by liquid cooling and about 17 kW by air (HPE's figures), which is why the rack needs direct-to-chip liquid cooling and a building designed around it.

The easiest way to understand why is to stop thinking of a rack as furniture. It is a machine that receives a continuous river of electricity, turns almost all of it into heat, and expects both flows to remain under control every second of the day.

NVIDIA’s DGX GB200 documentation describes a rack with 72 GPUs, compute trays, network switches, power shelves, bus bars, and liquid-cooling manifolds. The company says its power consumption is approximately 120 kW. NVIDIA's own guide says approximately 120 kW. HPE and Supermicro both rate their versions of the same rack at 132 kW in total, so treat 120 kW as the design nominal and budget above it. The two numbers describe different things — NVIDIA's reference design and the OEMs' configured systems — not a disagreement about the same measurement. That is a product-specific rating, not a universal number for every AI cabinet. But it is a useful example of where high-density computing is going.

Power density is the number hiding inside the bigger number

When a new data center is announced, the headline usually focuses on the campus: 100 megawatts, 500 megawatts, perhaps eventually a gigawatt. Those totals matter. Utilities plan around them. Communities debate them. Developers spend years trying to secure them.

Inside the building, however, engineers also care about concentration. A megawatt spread across hundreds of conventional racks is a different problem from a megawatt packed into eight or nine high-density AI racks. The total power can be the same while the cables, airflow, coolant piping, and safety margins are completely different. For how to read the campus-scale number itself, see 100 MW of AI capacity: five questions behind the number.

Think about rain. An inch falling across an entire town is weather. The same volume forced through one storm drain is a plumbing emergency. Power density is the storm drain problem.

RackPowerWhat the number is
Typical enterprise rack~7.5 kWAverage modal density, Uptime Institute Global Data Center Survey 2025 (excludes facilities above 30 kW; more than 80% of operators have no rack above 30 kW)
H100-class AI rack~40 kWFour air-cooled DGX H100 systems at 10.2 kW each — NVIDIA's published SuperPOD rack configuration (40.8 kW)
B200-class AI rack~57 kWDerived, not a published rack figure: four DGX B200 systems at ~14.3 kW each (NVIDIA product page)
GB200 NVL72~120 kW (NVIDIA) / 132 kW (HPE, Supermicro)NVIDIA's design nominal vs. the OEM configuration rating
GB300 NVL72 (next generation)Up to 142 kWNVIDIA Enterprise Reference Architecture provisioning ceiling

At 120 kW, the rack needs more than an ordinary plug and power strip. Electricity may arrive at the building at utility voltage, pass through transformers and switchgear, cross backup systems, and travel through distribution equipment before it ever reaches the cabinet. Near the rack, large conductors or busways make the final delivery. NVIDIA’s own design uses power shelves to convert alternating current into roughly 50-volt direct current, then distributes it through a bus bar.

Each handoff creates heat, occupies space, and introduces another component that must be monitored. Redundancy adds parallel paths because a data center is not designed like a warehouse where a tripped breaker simply means everyone goes home early.

Every watt becomes somebody’s heat problem

The processors do not make electricity disappear. They use it to move and manipulate information, and the useful work ultimately leaves the hardware as heat. A rack drawing roughly 120 kW therefore creates roughly 120 kW of heat that the cooling system must carry away while it is operating near that load.

That is why the electrical and mechanical designs are really one story. If you add more computing power without adding a way to move the heat, you have not added usable computing capacity. You have built a very expensive space heater.

Traditional server rooms move cool air through the fronts of cabinets and collect hot air from the backs. That approach still matters, including in newer AI systems. But air can only carry so much heat through a practical aisle and cabinet before the fans, ducts, and temperature differences become unwieldy.

Newer high-density systems increasingly bring liquid closer to the chips. In NVIDIA’s rack, liquid runs through manifolds and cold plates attached to the CPUs and GPUs, while components such as networking and storage still use air cooling. This is an important detail: “liquid cooled” does not always mean every component is underwater or that fans disappear. Many designs are hybrids.

Air cooling tops out at roughly 20–40 kW per rack, and sources disagree on the exact ceiling. NVIDIA's own air-cooled H100 SuperPOD design puts four DGX H100 systems — 40.8 kW — in one rack, which is about as far as the air-cooled era reached. The NVL72 is past that line: HPE's data sheet splits its 132 kW rack into about 115 kW removed by liquid cooling and about 17 kW by air. The fans do not disappear — networking and storage still use air — but air alone cannot carry this rack's heat.

To follow that heat from the cold plates to the outdoors, see Following the heat through a liquid-cooled data center.

Whether that heat rejection uses water depends on the outdoor equipment — see How Much Water Does a Data Center Use?.

The rack loop also does not solve the whole cooling problem. It merely moves heat from the chips into a liquid. That heat must travel through pumps and heat exchangers to a larger facility loop, then finally leave the building through dry coolers, chillers, cooling towers, or another heat-rejection system. The plumbing gets longer. The physics does not change.

Backup power has to cross the same bridge

High density also changes the scale of the backup system. Data centers usually place a short-duration bridge between the utility and standby generation. Batteries or another uninterruptible power supply carry the load for the seconds or minutes needed to ride through a disturbance or start generators.

A dense AI room asks that bridge to carry a great deal of power in a small area. Engineers have to size the batteries, inverters, switchgear, and generators for the intended load and redundancy strategy. They also have to plan what happens during maintenance, when one path is unavailable, and during failures that do not unfold as neatly as a diagram. Backup power is only one part of keeping a service running — see Backup power cannot protect every part of an AI service.

This is where the word “capacity” becomes slippery. A building may have enough total utility service on paper but still lack enough distribution or cooling capacity in the exact room where the new racks are supposed to go. The stranded space can look empty and still be unusable.

The rack reshapes the floor plan

Concentrated computing can affect structural loads, ceiling routes, pipe sizes, maintenance clearances, leak detection, and the amount of support equipment outside the white space. The server cabinet is only the visible endpoint.

It also changes construction sequencing. Power and cooling equipment with long lead times may have to be ordered well before the servers. The utility connection can become the schedule. Commissioning becomes a full-system exercise: not only “does the rack turn on?” but “can every power and cooling path behave correctly when something else fails?”

The story is not that one cabinet uses a shocking amount of power. The story is that concentrating that power forces the rest of the facility to become a different kind of machine.

This distinction matters when we talk about the AI buildout. Chip announcements move quickly. Buildings, substations, cooling plants, and utility upgrades do not. You can manufacture a faster processor without instantly manufacturing the electrical path and heat-rejection system it needs. The next step is already specified, not just promised: NVIDIA's reference architecture puts the GB300 NVL72 rack at up to 142 kW, and the building work it needs is the same kind — more power, more liquid cooling, longer lead times.

So when you see a new rack with an impressive GPU count, look one layer outward. Ask how the power reaches it. Ask where the heat goes. Ask which systems are redundant and which equipment sets the schedule. That is where the physical scale of AI becomes visible.

AI may be software when we use it. At the rack, it is electricity, heat, copper, coolant, and a building designed to keep all of them moving.

Questions readers ask

How much power does one GB200 NVL72 rack use?

About 120 kW, according to NVIDIA's hardware guide. HPE and Supermicro both rate their versions of the same rack at 132 kW, so plan for more than the nominal figure.

Why do published figures disagree?

They measure different things: NVIDIA's design nominal (~120 kW), the OEMs' configured-system rating (132 kW), and facility planning figures that add cooling and distribution losses on top. Ask which boundary a number describes before comparing it.

Can the rack be air-cooled?

No. Air cooling tops out at roughly 20–40 kW per rack; the NVL72 is three times that. Liquid runs through cold plates on the CPUs and GPUs, while networking and storage still use air — about 115 kW liquid and 17 kW air in HPE's split.

How does 120 kW compare with an ordinary rack?

Uptime Institute's 2025 survey puts the average rack at 7.5 kW (excluding the small number of facilities above 30 kW). One NVL72 rack draws about as much as sixteen average racks.

What comes next?

NVIDIA's reference architecture for the GB300 NVL72 — the Blackwell Ultra successor — says a full rack can require up to 142 kW. That is a provisioning ceiling for a rack that is shipping, not a roadmap promise.

For what an efficiency ratio can and cannot say about a building like this, see PUE: what an efficiency score reveals—and misses.

Sources and further reading

HPE, NVIDIA GB200 NVL72 by HPE Data Sheet (PSN1014890104); Supermicro, SRS-GB200-NVL72 product page; Uptime Institute, Global Data Center Survey 2025; NVIDIA, DGX SuperPOD Data Center Design featuring DGX H100; NVIDIA, DGX B200 specifications; NVIDIA, Enterprise Reference Architecture — NVL72 AI Factory: System Hardware & Components.