AI Token Costs Set to Plummet as Next-Generation GPU Technology Arrives
The cost of processing information in artificial intelligence models is on the verge of a dramatic decline, driven by the rollout of Nvidia's new Blackwell GPU systems.
A Leap in Performance
Nvidia's Blackwell GPUs are currently being installed in data centers and are expected to operate at scale in the second half of the year. A recent analysis by SemiAnalysis compared Nvidia's new Blackwell system (GB 300 NVL72) to the previous-generation Hopper HGX 200 system, revealing staggering improvements in performance and efficiency.
Blackwell generates 6,000 tokens per second per GPU, compared to Hopper's 90—a 65x increase.
The efficiency gains are equally profound. When measuring tokens per megawatt of energy, Blackwell produces 2.8 million tokens per second, versus Hopper's 54,000—a 50x improvement.
The Cost Implications
The most significant impact for consumers and developers will be on price. The cost per million tokens is expected to fall dramatically.
- Blackwell: $0.12 per million tokens
- Hopper: $4.20 per million tokens
This represents a 35x reduction in cost, making AI processing far more accessible than ever before.
Industry Leaders Weigh In
An unnamed CEO of an AI infrastructure company noted that new AI models launching later this year will be both better and more efficient than current offerings.
OpenAI CEO Sam Altman has also addressed the issue directly, stating that AI costs have become a huge issue, and the startup will help customers get more value for less spend.
Market Signals
The token spending index, compiled by Silicon Data, has already begun to reflect the anticipated price drops. The index peaked at 2.06 in late May and has since fallen to 1.75 as of June 10.
Analysts expect these systems to drive a massive increase in cheaply generated tokens, leading AI model providers to slash prices.
Gradual Rollout
The installation of Blackwell GPUs has been gradual, largely due to new infrastructure requirements such as water cooling. However, as these systems come fully online, the industry is poised for a significant shift in the economics of artificial intelligence.