AI Inference Is Redrawing the Data Center Map

For the past two years, a data center buyer’s playbook was straightforward: find cheap power, find land, build big, and worry about the network later. That playbook is breaking. As AI workloads shift from one-time model training toward continuous inference and agentic workflows, the infrastructure math buyers relied on — GPU-to-CPU ratios, rack density targets, even which metro to site in — no longer holds. Dell’Oro Group, which tracks data center switching and networking spend, is now projecting close to a 40% CAGR in AI-driven front-end network opportunities, and its own analysts say the assumptions that governed training-era buildouts are already obsolete for what buyers are deploying today.

The 10-to-1 ratio that stopped working

Sameh Boujelbene, Vice President at Dell’Oro Group, put it bluntly in the firm’s August 2026 research note: “As AI infrastructure shifts from large-scale training to inference and agentic workflows, the traditional assumption of a 10-to-1 ratio of XPU to CPU no longer applies across all deployments.” In some inference-heavy environments, that ratio is compressing toward 1-to-1, because CPUs are doing more of the orchestration and data-movement work that keeps an inference pipeline fed. Dell’Oro also points to a less obvious cost center: networking for key-value (KV) cache storage racks, which inferencing applications need to hold the intermediate state of a conversation or a long context window so the model doesn’t recompute it on every token.

This matters to a buyer because it changes what you’re actually procuring. A facility sized and wired for a training cluster — heavy on GPU-to-GPU east-west bandwidth, light on CPU and storage networking — is not automatically a good inference facility. More than half of the coming growth in front-end switch sales is now tied to AI, and that spend is pulling adoption of 800 Gbps and 1.6 Tbps ports into networks that, two years ago, would have been built around much lower speeds. If your colocation or build-to-suit contract was scoped against a training-era bill of materials, you may be paying for the wrong fabric, and it’s worth asking your provider directly for the orchestration, CPU, and KV-cache-related line items in the design, not just the GPU count.

Why the map itself is moving

The more consequential shift is geographic. Training workloads tolerate latency; you can run a model-training job from a stranded-power site in rural Wyoming or West Texas and nobody notices. Inference doesn’t work that way. As infrastructure provider Inflect notes in its 2026 colocation guidance for AI buyers, “inference, unlike training, has to live close to end users” — the firm recommends facilities within roughly 10 to 30 milliseconds of the user population, with most real-time AI applications needing a sub-200-millisecond end-to-end response. That requirement pushes buyers back toward exactly the premium, supply-constrained metros — Ashburn, Dallas, Frankfurt — that the power crunch has made expensive and slow to interconnect, while second-tier markets like Kansas City and Reno get pitched as lower-cost alternatives for latency-tolerant tiers of traffic.

You can see the pattern in how inference-specialist Groq is actually buying capacity. The company doesn’t build its own data centers; it leases from colocation providers including Equinix, DataBank, and Bell Canada, and plans to bring more than a dozen new sites online through the end of 2026, on top of roughly a dozen it deployed in 2025. Its footprint spans the U.S., Canada, the Middle East, and Europe, with Asia deployments underway — a deliberately distributed pattern that looks nothing like a single hyperscale training campus, and is explicitly built around sitting closer to population centers rather than wherever power is cheapest. Groq CEO Jonathan Ross told the Wall Street Journal to expect the site count to “grow rapidly.” For buyers evaluating where to put inference capacity, that’s a real-world signal about how a company whose entire business is inference economics is voting with its lease commitments. It’s also a direct complement to the memory and GPU supply constraints we covered in detail recently — the chip shortage determines how much capacity you can get; the inference shift determines where you should actually put it.

The power and cooling spec sheet has changed too

Rack density assumptions are moving just as fast. Goldman Sachs’ 2025 infrastructure research put typical GPU rack power at roughly 25 kW two years ago, climbing past 130 kW today, with a trajectory that could exceed 600 kW per rack by 2027 as platforms like NVIDIA’s GB200 NVL72 (rated 130 to 600-plus kW) become standard. At that density, air cooling simply stops working — it becomes insufficient somewhere between 30 and 50 kW per rack, according to testing cited by Tom’s Hardware. Rear-door heat exchangers can handle up to roughly 70 kW per rack; above that, direct-to-chip liquid cooling or immersion becomes necessary, and KAD’s 2026 market data shows direct-to-chip already accounts for about 65% of deployed liquid cooling capacity.

The economics here are not cosmetic. Liquid-cooled facilities are running PUEs in the 1.10 to 1.20 range, against 1.40 to 1.60 for air-cooled sites — a gap that compounds every month on a power bill at these densities. McKinsey’s 2025 analysis of colocation economics found that leasing colocation space with owned or leased Blackwell-class hardware can deliver 40% to 70% lower run-rate cost than alternatives once utilization clears about 70%, an estimate built specifically around inference workloads rather than training runs. For a buyer, the practical consequence is that a facility’s advertised PUE and its actual rack-level cooling topology now belong in the same due-diligence conversation as its power contract, because at 100 kW-plus per rack you’re also into 415V or 480V three-phase requirements with A/B redundant feeds — specs that a lot of colocation inventory built even three years ago simply doesn’t meet.

A real market, not just a vibe shift

The dollars back up the pattern. Grand View Research sizes the global edge data center market at $34.8 billion in 2025, projecting $40.0 billion for 2026 and $105.8 billion by 2033 — a 14.9% compound annual growth rate. North America holds the largest share today, at 35.5% of 2025 revenue, but Asia Pacific is the fastest-growing region at a projected 17.3% CAGR through 2033, driven by the same AI-optimized edge buildout — GPU-enabled servers, liquid cooling, high-density power — that’s reshaping metro-tier facilities in North America and Europe. Notably, the same report flags high power consumption and grid constraints in urban and semi-urban sites as the key restraint on how fast this can actually happen, which is a useful reality check: the inference shift wants to move capacity into cities, right at the moment cities are among the hardest places to get new grid capacity approved.

What this doesn’t mean, and what to actually do about it

It would be a mistake to read all this as “training is over, go build small facilities everywhere.” Frontier labs are still standing up enormous training clusters, and hardware like the GB200 NVL72 is explicitly marketed for both training and inference, because large mixture-of-experts models now blur the line — batched, high-throughput inference at scale can look a lot like a training workload in its power and network profile. Some analysts argue that as inference workloads grow larger and more batched, they’ll consolidate back into big hyperscale campuses rather than fragment into edge sites, especially where grid access, not latency, is the binding constraint. The honest picture is that buyers now need to plan for both patterns simultaneously, not pick one and assume it’s permanent.

What that means practically: before signing a colocation or build-to-suit agreement, get specific about which workload the facility is actually being designed around. Ask for the CPU-to-XPU ratio and KV-cache networking provisioned in the design, not just GPU count. Confirm the rack power ceiling and whether the cooling topology (air, rear-door, direct-to-chip, immersion) matches your actual density roadmap through 2027, not your 2025 deployment. And weigh latency requirements against power availability explicitly — a sub-30-millisecond requirement in a supply-constrained metro may cost you more in interconnection delay than it saves you in user experience, depending on your workload mix. The buyers who get this right in the next 12 months will be the ones negotiating from a spec sheet built for 2027 workloads, not one inherited from the last training-cluster deal they signed.

Start that due-diligence process with the Data Center Buyer’s Toolkit, which walks through exactly these power, cooling, and network questions before you sign.

Written by

Raajeev Ratra

Data Center Infrastructure Expert | 15+ Years in DC Design, Operations & Project Management

Raajeev is a seasoned data center professional with hands-on experience in hyperscale facilities, colocation design, power & cooling infrastructure, and global DC operations. He shares practical insights to help engineers and IT leaders build better infrastructure.

Connect on LinkedIn →