A couple of weeks ago Sam Altman said on the Cheeky Pint podcast that "it's quite terrible to have everybody have a bunch of the world's limited supply of DRAM sitting in their own computer," where it gets used "5% of the time." The internet did not take it well, and a few days later he posted that he does "want the world to have tons of memory and people to put as much as they want in their computers" (CNET).
Set the politics aside. The arithmetic in his first comment is correct, and it is the whole cloud business. Hardware that sits in one person's machine is sized for that person's busiest moment and idles the rest of the time. Hardware that many people share only has to be sized for the moment when all of them together are busiest, and that moment is much smaller than the sum of everyone's individual peaks.
The jargon is peak of the sum versus sum of the peaks. The gap between them is what a cloud sells. Here's how big that gap is on real data, and what happened the last time an industry worried about it.
How big is the gap? Real GPU data#
In 2020 Alibaba ran a GPU cluster of about 6,500 GPUs shared by over a thousand internal teams, and later published two months of its job logs (Weng et al., NSDI '22). Each log line says which team ran a job, when, and how many GPUs it asked for. That's enough to answer the question directly.
For 8 weeks of that trace, I added up how many GPUs each of 1,197 teams needed at its own busiest moment. Then I measured the busiest 10 minutes for the cluster as a whole.
Alibaba PAI GPU cluster trace, 2020. GPUs requested by running tasks, 10-minute buckets.
- If every team had bought GPUs for its own peak, they would have needed 48,187.
- Sharing one cluster, they never needed more than 5,858 at once. That's 8.2 times less hardware.
- A team sized for its own peak would have used about 1% of its GPUs on average. The shared cluster ran at 61%.
The lower chart shows how fast the saving comes. Pool 10 random teams and you need about half the GPUs they'd buy separately. Pool 100 and you need about a quarter. Pool everyone and you need an eighth.
Not every mix pools#
Sharing only helps when customers are busy at different times. Microsoft's public trace of two Azure LLM inference services, one for code and one for chat, makes the point from the other side (Stojkovic et al., HPCA '25). Pooled together, those two services need only 10% less capacity than separately, because both follow the same working day.
The extreme case is a customer who reserves its own peak for the whole contract, like a big training run on a dedicated cluster. That customer pools with no one: its line sits at 1.00 forever. It's why I think "neocloud" is too broad a label. A company that rents a reserved cluster to one customer is running a hardware lease. A company that shares a fleet across many customers who peak at different times gets the gap above as margin. They are different businesses, and the market has started to price them differently.
60,000 hard drives#
Around 2007 I bought 60,000 hard drives in a single order at Google. At that time Wall Street was trying to decide whether the cloud was good or bad for the companies that made them.
The bear case was this same arithmetic. Cloud companies run their drives far fuller than a PC or an office server does. If storage moves into shared data centers, the world needs fewer drives for the same data.
The bull case was that cheaper, easier storage would make people keep far more data.
Both were right.
Drives shipped vs. capacity shipped, all makers. Sources: iSuppli, Western Digital, Coughlin Associates, StorageNewsletter/TRENDFOCUS.
- Drives shipped peaked at 651 million in 2010 and fell to about 127 million by 2023 (Tom's Hardware, citing Coughlin). Flash took the drive inside the laptop, and the cloud took the rest of the consumer's storage.
- Capacity shipped went the other way. About 88 exabytes in 2007 (Western Digital, FY2008 Q2 call), about 1,338 exabytes in 2021 (StorageNewsletter). That's roughly 15 times as much storage, sold as far fewer, far bigger drives.
- The customer changed too. Today most of the industry's capacity goes to data centers. In fiscal 2025 Seagate shipped 595 exabytes on its own (10-K), and about 90% of its exabytes now go into mass-capacity drives for cloud and enterprise customers (Nasdaq).
So the cloud didn't shrink storage. It made storage cheap and easy enough that the world kept far more data, and it changed what a good hard drive business looks like: fewer, larger drives sold to a few very large buyers.
So, about RAM#
Altman's point is right: memory spread across billions of devices is badly used, and memory pooled in data centers is used far better. The hard drive history says what probably comes next. Pooling raises utilization, and it doesn't reduce total demand. It lowers the cost of using memory, so people find more things to do with it, and total demand keeps rising. The shape of the market changes, though. More of the memory goes to a few large buyers, sold under long contracts.
If you're trying to judge a memory maker, a GPU cloud or a storage company, ask the hard drive question: how much of its hardware is shared across customers who peak at different times, and how much is one customer's reserved peak? The first is a cloud business. The second is a lease.
Method. Alibaba PAI cluster trace v2020: tasks joined to jobs to get the team (user id); GPUs per task = requested GPU share × instances; demand counted in 10-minute buckets over days 7 to 63 of the trace, skipping its ramp-up and wind-down. "Sum of the peaks" adds each team's busiest bucket; "peak of the sum" is the busiest bucket for all teams together. The pooling curve is the median over 300 random groups at each size. These are GPUs requested, not measured utilization, and the teams are inside one company rather than separate customers, so treat 8.2x as one cluster's number, not a law. Azure: tokens per minute for the code and conversation services over their overlapping five days in May 2024.
