Cloud Economics Are Not What They Seem
Cloud computing delivers enormous value early in a company's life, but at scale, the economics invert: cloud costs become a dominant drag on margins, and repatriation can cut infrastructure costs by half or more.
"You're crazy if you don't start in the cloud; you're crazy if you stay on it." Sarah Wang and Martin Casado, a16z
Across 50 of the top public software companies, an estimated $100 billion of market value is being lost due to cloud's impact on margins. Dropbox saved nearly $75 million over two years by repatriating workloads, improving gross margins from 33% to 67%. The math is straightforward: AWS operates at roughly 30% blended operating margin even after committed-use discounts, meaning customers are paying a substantial premium over the cost of running equivalent infrastructure themselves. At scale, when growth slows and efficiency becomes the key determinant of public market value, this margin compression becomes existential.
The AI era makes this calculus even more extreme. Traditional software had near-zero marginal costs serve one more user, pay essentially nothing extra. AI workloads have real, significant marginal costs: every inference query burns GPU cycles, memory bandwidth, and electricity. The 2025 outlook captures this shift bluntly: "The era of aggregation theory is behind us, and AI is again making technology expensive." Hyperscalers now face extremely high fixed costs and high marginal costs simultaneously a paradigm none of them were built for. Capital that was cheap under ZIRP now must generate returns in a world where software improves with more capital expenditure rather than simply being amortized across users.
The largest opportunity in infrastructure may sit in the gap between cloud hardware and the unoptimized code running on it. Neither full repatriation nor naive cloud usage is the answer. The winners will be those who understand their workload economics deeply enough to place each workload where it runs most efficiently.
Takeaway: Cloud is not inherently cheap or expensive it is a financing and operational model whose economics depend entirely on scale, growth rate, and workload characteristics, and most companies evaluate it far too lazily.
See also: Inference Cost Dominates Training at Scale | Custom Silicon Will Eat General Purpose Computing | Infrastructure Determines Output