Do AI Agents Really Need One CPU per GPU?
TL;DR
- Agents do need more CPU than chatbots did, but no published study shows the right number is one CPU per GPU.
- In real traffic, the CPU work of running code and searching databases took as long as the model itself in over a quarter of requests.
- The chips being sold for this are ordinary server processors. Intel’s densest new one even leaves out the AI math units its higher-end Xeons carry.
- Server chips sell for far more, so Intel has been making fewer PC chips. Consumer chip prices rose 5% to 10% this spring.
At the OCP APAC Summit 2026, Intel and AMD made adjacent arguments about the same shift. AMD said AI agents are pushing the CPU-to-GPU ratio toward one-to-one. Intel, without giving a ratio, said bottlenecks are moving toward CPUs and memory even where accelerator capacity is available.
Neither was new. Both companies have been saying it since spring, and both sell CPUs. So the question worth asking is not whether they said it, but what the number refers to, and whether anyone has measured it.
One curve is really two
The familiar story runs 1:8, then 1:4, then 1:1, as though a single trend were tightening. Two different changes are being blended, and only one is about agents.
The first already happened, before the agentic-rack argument existed. Older eight-GPU servers carried one or two processors, a ratio of 1:8 or 1:4 at the server level. Rack-scale designs shifted that. Vera Rubin NVL72 integrates 72 Rubin GPUs and 36 Vera CPUs across 18 compute trays in a single rack, which is 1:2. Whatever drove that topology, it was settled before anyone was selling CPU racks for agents.
The second is genuinely new, and the vendors disagree about it. Nvidia has built a rack with no accelerators in it. The Vera CPU Rack holds up to 256 Vera CPUs, 22,528 cores, up to 400 TB of memory, 48U and fully liquid-cooled, and its stated job is running the sandboxes that execute code, call tools and evaluate results. The first Vera CPU systems have gone to Anthropic, OpenAI, Oracle Cloud Infrastructure and SpaceXAI; the NVL72 racks beside them are in full production and due to ship in the second half of this year.
Arm reaches the same tier without a new rack. Its AGI CPU puts up to 136 Neoverse V3 cores in a 300-watt part built for standard 1U chassis, quoting 8,160 cores per air-cooled rack and more than 45,000 liquid-cooled. AMD is blunter still, arguing that the density being positioned as future-looking is already exceeded on standard equipment, with “no new rack architecture required.” One vendor sells the tier as a rack, the others sell it as a socket.
Either way, adding processors that sit beside no accelerator is what moves a fleet-wide ratio. That much is arithmetic.
Which lets the number be tested, under a stated assumption. If one-to-one means one physical CPU package for every GPU package across the combined infrastructure, the published rack specifications imply the following. These figures are worked out from those specifications, not disclosed by any vendor.
| Configuration | CPUs : GPUs | Ratio |
|---|---|---|
| One NVL72 rack | 36 : 72 | 1 : 2 |
| One NVL72 plus one CPU rack | 292 : 72 | about 4 : 1 |
| Seven NVL72 racks plus one CPU rack | 508 : 504 | about 1 : 1 |
| Nvidia’s 40-rack POD, before CPU racks | 576 : 1,152 | 1 : 2 |
The last row is Nvidia’s own configuration. The Vera Rubin POD is specified at 40 racks and 1,152 Rubin GPUs, which is sixteen NVL72 racks carrying 576 Vera CPUs between them. Bringing that to one processor per accelerator would need 576 more, or roughly 2.25 fully populated CPU racks. Nvidia does not say how many of the POD’s 40 racks are Vera CPU racks.
Pairing one CPU rack with one GPU rack lands near four processors per accelerator: high, but inside AMD’s own “1+ on the CPU side” range rather than outside it. Exact parity takes roughly one CPU rack for every seven NVL72 racks.
One caveat runs under all four rows. The 36 processors inside an NVL72 and the 256 inside a CPU rack are the same Vera part doing different jobs, one scheduling accelerators and the other running sandboxes. There are Vera cores inside the BlueField-4 storage racks too. Counting packages treats all of them as interchangeable. They are not, and that is most of what the single number hides.
Do the experiments support the trend?
Two research groups have instrumented agent workloads.
A team at the University of Texas at Austin, working with a Microsoft Azure co-author and access to the Azure fleet, published a 24-hour trace of production agent traffic on 5 August. Tool execution took as long as or longer than model inference for more than 27% of requests. A separate study co-authored by Georgia Tech and Intel found exact nearest-neighbor search over a 115 GB document corpus consuming 81% to 83% of end-to-end latency on one test system and up to 89% on the other. Two of that paper’s five authors work at Intel, which does not make it wrong but does mean it is not a neutral referee.
So the mechanism is real. Agents genuinely push work back onto the processor.
The sizing question is where both studies stop, and the reason is structural: neither varied the ratio. The Georgia Tech systems each had one CPU and one GPU. The Azure controlled experiments ran on a single 96-core processor feeding eight A100s, the old 1:8 shape. Each measured how time divides on a configuration it inherited.
The controlled runs also make the case that a single ratio is the wrong instrument. Median host utilization ran from 6% to 31% depending on the framework, and accelerator activity stayed below 55%, though power still ran at 50% to 70% of peak. Utilization sits near 11% through sequential stretches and jumps toward full when one stage fans out into parallel builds and test runs. Sizing to the average leaves nothing for the spike; sizing to the spike strands capacity the rest of the time. The authors’ own conclusion is that no single CPU-to-GPU time ratio characterizes the workload.
That is a measurement of how a request divides its time. It is not a measurement of how many sockets belong in a hall, and nobody has published one of those.
Is an agent CPU different from an ordinary server CPU?
Less than the marketing suggests, and the most interesting evidence points the opposite way from “AI chip means more AI instructions.”
Intel’s 288-core Clearwater Forest is the part most often pointed at this workload. Its dense Darkmont cores leave out AVX-512, AMX and hyperthreading, features that Intel’s own Xeon 6 P-cores carry. The argument is that agentic tool execution does not need them, and the logic holds: running a Python interpreter or a database query does not call for wide matrix math. The subtraction is not the whole design, though. Clearwater’s I/O dies carry sixteen fixed-function accelerators for cryptography, data movement, compression and load balancing, conventional datacentre offload blocks rather than AI instructions, but silicon added rather than removed.
Nvidia argues a different design point, fast per-core performance and predictable loaded latency, and claims up to 1.8x over leading x86 processors on compilation, code analysis and Python. Coherent position, weaker evidence. Independent analysis noted the whitepaper’s “agentic benchmarks” are four standard SPEC CPU 2026 integer components, with no model serving tokens and no runtime choosing tools. The same analysis puts Nvidia’s own estimated SPECrate 2026 Integer Base at 925 for two Vera sockets against 898 for two EPYC 9755 sockets, a three percent system difference presented as 1.8x per-core bars. Per core or per socket is the same choice the ratio argument keeps making without saying so.
What is genuinely new is the tiering, not the silicon. Vendors now ship three distinct families, general-purpose, accelerator head node, and agentic, built on conventional server scaling: more cores, more memory channels, more I/O.
Will processors go short the way memory did?
Not through the same mechanism. Memory has a bit-output problem, since each gigabyte of high-bandwidth memory consumes roughly three times the wafer capacity of DDR5, so extra wafer starts do not yield proportional supply. Processors face allocation limits instead. Intel named leading-edge logic wafers, memory and substrates as its constraints in July while reporting 18A output ahead of plan.
The channel that reaches ordinary buyers is allocation, not fabrication. Server processors run from the high hundreds of dollars into the tens of thousands, against a consumer market an order of magnitude cheaper. Intel has been shifting wafer capacity away from client CPUs since October 2025 and has guided full-year PC unit volumes down by low double digits.
That is one mechanism contributing to tighter client supply and higher prices, alongside the memory and storage demand every data center build creates. The pricing detail is covered separately. What matters here is that the agentic CPU tier competes for the same capacity as the parts in a desktop.
If there is a boom, who collects it?
Not automatically the two companies making the argument.
AWS has said the majority of compute capacity it added in 2025 ran on its own Graviton processors, and marketed Graviton5 explicitly for agentic work in June. Meta is named in Arm’s launch release as lead partner and co-developer of the AGI CPU, for which Arm reports more than $2 billion of customer demand across its FYE27 and FYE28 financial years. Nvidia’s Vera racks are Arm parts too.
Two figures circulate, both true, which is why this stays muddled. Arm designs were 13.2% of server CPU unit shipments in the first quarter; Arm separately claims about half of CPU compute at the largest cloud providers. One counts chips, the other counts capacity at a handful of buyers.
What this means for hardware already on the floor
The orchestration tier is bursty and runs at low average utilization, both measured. Many of the tool workloads that have been profiled do not lean on wide matrix or vector acceleration, the feature separating premium parts from ordinary ones. That describes a good deal of hardware already racked, and a good deal of what comes out when a rack is consolidated.
It also sits in its own cabinet. A head-node processor inside a vendor-configured accelerator rack cannot be substituted; a separate CPU tier is exactly where substitution becomes possible.
Four things cut against it. Rack power budgets are fixed, so older cores cost capacity per watt where power is the binding constraint. Tail latency matters for interactive agents in a way average utilization hides. The failure mode is asymmetric, since the Georgia Tech paper found CPU-side parallelism saturates early and strands the accelerator behind it, which AMD phrases as GPUs waiting when the CPU tier is undersized. And the Azure authors do not conclude that ordinary processors are sufficient; they argue future agent servers would benefit from heterogeneous cores, hardware scheduling support and microarchitectural partitioning.
There is also a competing way to ask the whole question. AMD’s modeled 100 kW rack analysis normalizes EPYC, Xeon and Vera to a fixed power envelope rather than a package count, reporting 2.37x, 1.46x and 1.0x rack throughput respectively, with a Turin deployment carrying more than 27,000 cores per rack against the 22,500 sandbox environments Nvidia claims. AMD adds the caveat itself: sandboxes and cores are not directly equivalent, and the comparison is directional. If sizing belongs in watts rather than packages, the CPU-to-GPU ratio is not merely underspecified, it is the wrong instrument. Package counts still matter for one reason, which is that they determine what physically comes out of a rack at end of life.
Supply of used parts may be tighter than the consolidation story implies. CoreWeave signed an A100 contract running into 2029 at prices at or above where they were years ago, which shows older accelerator capacity staying in service. Whether the same holds for the processors beside it is an inference, not a measurement. Where a consolidation does free up processors, getting used server CPUs valued in bulk is worth doing before they go to scrap.
What would settle it
Three claims are being collapsed into one number. Agents put substantially more work on processors, which is measured and not seriously disputed. Vendors are building separate CPU tiers alongside accelerator racks, which is visible in shipping product specifications. Those tiers imply roughly one CPU per GPU, and this is the one still missing a defined denominator and a published sizing experiment.
What no vendor publishes is a sizing rule. Nvidia specifies a 40-rack POD holding 1,152 Rubin GPUs across five rack types, and AMD publishes rack throughput normalized to a power envelope. Neither state how many CPU racks belong beside a given number of accelerator racks, and no operator has published what its own fleet settled at. Until one does, one-to-one is a conclusion with the arithmetic left out.