Skip to main content

The Hidden Infrastructure Behind AI: Chips, Power and Water

The hidden infrastructure behind AI is becoming a critical constraint

By Mohammad Muneer Ahmed
Published: Sep 21, 2026
5 mins read
👁️ 26 Unique Views
The Hidden Infrastructure Behind AI: Chips, Power and Water
The scale of inference: Optimized for multimodal workloads.
Premium Insight

Why It Matters

AI growth depends on more than models and software. Data centres require reliable electricity, cooling and water, making infrastructure and energy capacity increasingly important as AI adoption expands.

OpenAI's flagship Stargate site in Texas is being built toward 1.2 gigawatts of power — enough for a small city — and that's just one part of a wider plan to reach 10 gigawatts nationally. In Louisiana, Meta's Hyperion data centre has scaled up to a planned 5 gigawatts, pushing the company's investment there past $50 billion.

These aren't unusual, one-off projects. This is what it now takes to build and run modern AI — specialised chips, huge buildings, enormous amounts of electricity, and often, a lot of water for cooling.

It starts with specialised chips

Regular computer chips can run AI, but specialised ones do it far better. Graphics chips (GPUs), first built for video games, turned out to be great at the kind of maths AI needs, and companies have since built chips made specifically for AI — like Google's TPUs and Microsoft's Maia.

These chips keep getting more efficient — roughly doubling in efficiency every two years, according to research group Epoch AI. But companies mostly use those gains to run even more AI, not to cut energy use. So even as chips improve, total electricity demand keeps climbing.

Data centres built just for AI

AI models are trained and run inside data centres: buildings packed with servers, networking equipment, and cooling systems. Rather than reusing older buildings, major AI companies are mostly building brand-new, custom facilities — which means there's limited room to save energy simply by upgrading old ones.

The scale is enormous. OpenAI's Stargate programme alone is aiming for 10 gigawatts of power — more than Switzerland's entire peak electricity use as a country. A single AI project can now outgrow the power needs of a small nation.

The electricity bill keeps climbing

The International Energy Agency expects global data-centre electricity use to more than double, from about 415 terawatt-hours in 2024 to roughly 945 terawatt-hours by 2030 — just under 3% of all electricity used worldwide, and growing four times faster than overall electricity demand. The agency says AI is the biggest reason for this growth.

In the US, data-centre electricity use rose from 76 terawatt-hours in 2018 to 176 terawatt-hours in 2023, according to US government-backed research. Total power demand from US data centres is expected to nearly triple by 2030, and government estimates suggest data centres could use up to 9% of all US electricity by then, up from about 4% in 2023. On top of that, researchers estimate the single largest AI training runs could each need 4 to 16 gigawatts of power by 2030 — as much as several large power plants combined, just to train one model.

Cooling, and the water it takes

Packed rows of AI chips generate a lot of heat, and keeping them cool is now as big a challenge as building the chips themselves. Many data centres use water-based cooling, which is why AI's environmental footprint is increasingly discussed in terms of water, not just electricity — though actual water use varies a lot by location and system design.

This has pushed companies toward liquid cooling, where coolant runs directly next to the chip instead of using air conditioning. One market analyst firm expects the liquid-cooling equipment market to roughly double in 2025 to nearly $3 billion, then grow to about $7 billion by 2029 — a sign that cooling has become as much of a bottleneck as the chips themselves.

Can this keep scaling?

Not without real changes. Chip efficiency keeps improving, but not fast enough to offset how much more AI is being run — companies tend to use efficiency gains to do more, not to use less power overall. More electricity generation and grid capacity will need to be built near data centres, and this is already straining power grids and raising bills in some areas.

None of this means AI's growth will stop. But power, cooling, and water are now just as important to how far AI can scale as any advance in the models themselves.

Found this analysis insightful?

Share with colleagues, engineers, and your network.

Link copied to clipboard!