Why smarter AI models could drive up compute prices 10x

Overview

This episode explores the economic and technological dynamics of AI compute, focusing on the interplay between AI model advancements, compute costs, and revenue growth. It examines the implications of scaling AI capabilities, the constraints of compute supply, and the potential for power concentration in the AI industry.

Notable Quotes

- If a true human-level software engineer could run on an H100 equivalent, then at today's prices for software engineers, that H100 should rent for over $250K a year.

- I wish we didn’t live in a world with such strong economies of scale for intelligence, because I’m worried about power concentration, but it seems we do.

- At some point, we’ll just have robots that can convert shores of silica sand and mines of copper into new computer chips.

🧮 The Revenue vs. Compute Growth Gap

- Anthropic's revenue has been growing at a staggering 10x year-over-year, while compute capacity only grows at 3x annually. This disparity raises questions about sustainability.

- Three factors could bridge this gap: increased lab margins, higher compute prices, or a shift in compute allocation from training to inference.

- Inference margins for labs like Anthropic have already risen from 40% to 80%, while compute spot prices have increased by over 40% since early 2026.

📈 The Economics of Compute Pricing

- As AI models become more efficient, they can monetize compute more effectively, driving up its value. For instance, a single H100 GPU could theoretically generate 15x its current rental price if it performed at human-level productivity.

- The scarcity of compute resources, coupled with labs’ ability to outbid competitors, could lead to a significant increase in compute costs.

- The Allen effect suggests that more efficient models will command higher margins, as they economize on expensive compute resources.

🏭 Constraints on Compute Supply

- The 3x annual growth in compute capacity is driven by Moore’s Law (1.4x), new fab construction (1.2x), and reallocation of wafer capacity from other industries (1.8x). However, these factors face hard limits:

- Moore’s Law is nearing its physical limits.

- Fab construction is bottlenecked by the production of ASML EUV machines.

- AI has already absorbed most leading-edge wafer capacity, leaving little room for further reallocation.

💡 Implications for AI Applications and Power Dynamics

- As compute becomes more expensive, many popular AI applications may be priced out, leaving only high-value use cases like AI research and automation.

- The strong economies of scale in AI models could lead to power concentration, as top labs dominate the market by leveraging their superior monetization of compute.

- This dynamic mirrors historical debates about resource scarcity, but compute supply is less elastic than traditional commodities, making it harder to absorb demand shocks.

🤖 The Future of Compute Costs

- In the long term, compute costs may drop as automation enables the direct conversion of raw materials into chips. However, in the current pre-singularity phase, the imbalance between compute growth and AI value creation will likely persist.

- The economies of scale in AI training—where a one-time cost creates reusable intelligence—contrast sharply with the inefficiencies of human labor, further amplifying the value of compute.