The AI Inference Boom Is Just Getting Started

The first wave of the AI revolution was all about construction. Companies like Nvidia $NVDA ( ▼ 1.93% )  and Meta $META ( ▼ 1.56% )  spent billions, literally hundreds of billions on building the brains of the future. This was the "training" phase. You buy the GPUs, you plug them in, and you teach a Large Language Model (LLM) how to understand the world. But training is just the beginning.

I’m watching a massive shift in the market right now. We are moving from the training phase to the inference computing phase. This is the stage where AI models aren’t just being taught; they are being used. Every time you ask ChatGPT for a recipe or use Copilot to write a line of code, you are running an inference workload. By 2026, Deloitte estimates that inference will account for roughly two-thirds of all AI compute.

For investors, this shift is everything. The companies that won the training phase (looking at you, Nvidia) aren't necessarily the only ones who will dominate the inference era. We’re looking at a $11.1 trillion cumulative AI capex cycle through 2029, and the winners are beginning to diverge.

📉 AI Training vs Inference: The Great Rebalancing

To understand why this matters, you have to understand the technical difference between AI training and inference. Think of training like medical school. It’s incredibly intense, requires massive amounts of power, and takes a long time. Once the doctor graduates, they go into the world to treat patients. Treating the patients is "inference." Training is a capital-intensive process. Inference is a continuous, high-volume flow.

As AI agents and copilots become standard in every software application — from MongoDB’s database tools to your local weather app — the demand for inference is going to skyrocket. This is where the real monetization happens for big tech.

The Inference Thesis:

• Scale: Training happens once (or periodically). Inference happens billions of times a day.

• Cost: Inference requires chips that are optimized for power efficiency and "latency" (how fast you get an answer).

• Location: While training happens in massive, centralized data centers, inference is moving to the "edge": your phone, your laptop, and local servers.

📈 The $11 Trillion Infrastructure Supercycle

The numbers here are staggering. SemiAnalysis recently forecasted a cumulative $11.1 trillion in AI IT and data center capex between 2024 and 2029. We aren't just talking about buying a few more chips. We are talking about a complete overhaul of the world’s digital infrastructure.

Goldman Sachs $GS ( ▲ 3.06% )  is signaling even more pressure on the system, projecting that data center power demand will surge 220% to 1,350 TWh by 2030. I’m paying close attention to who can actually handle this load. It’s not just about who has the fastest chip; it’s about who has the memory, the power, and the connectivity to make it work.

🔍 Micron: The HBM Bottleneck

You can’t run a high-end AI model without High-Bandwidth Memory (HBM). It’s the "desk space" where the processor does its work. Micron $MU ( ▲ 8.76% )  is in a legendary position right now. Their HBM3E supply is completely sold out through 2026 and most of 2027. When a company tells you they physically cannot make enough of a product to satisfy demand for the next two years, you listen.

Micron HBM is a critical component for both Nvidia’s Blackwell and AMD’s $AMD ( ▲ 6.93% )  Instinct chips. Because the market is so tight, Micron has incredible pricing power. We’re seeing gross margins head toward the high-60% range. For a memory company, those are "software-like" numbers.

🔍 AMD: The Inference Challenger

While Nvidia owns the training market, AMD is positioning itself as the king of AMD AI inference.

The new Instinct MI325X is a beast designed specifically for these heavy inference workloads. It packs massive amounts of HBM3E, which is exactly what you need for serving models with "long context windows," like when you ask an AI to summarize a 500-page PDF.

I’m watching how hyperscalers (Amazon, Google, Microsoft) begin to mix their chip sets. They don’t want to be 100% dependent on Nvidia. AMD provides a high-performance alternative that is arguably better suited for the high-volume, power-conscious world of inference.

⚠️ The Power Gatekeepers

You can have all the chips in the world, but if you can't plug them in, they’re just expensive paperweights. The inference boom is colliding head-on with a global power shortage. This is why "AI Landlords" are becoming the best growth stocks in the infrastructure space. Companies like TeraWulf $WULF ( ▲ 1.91% )  have pivoted from Bitcoin mining to providing high-density power for AI.

TeraWulf’s recent lease agreement with Anthropic is a massive signal. It proves that the "neocloud" model: where smaller, agile firms provide the physical space and power for AI: is a legitimate multi-billion dollar opportunity. If you want to follow the power story deeper, check out our deep dive on nuclear stocks for AI.

📱 Edge AI: Bringing the Brain to the Device

The final leg of the inference boom is edge AI computing. Right now, your AI requests mostly go to a giant data center in the desert. But that’s slow and expensive. The future is "Edge AI," where the inference happens directly on your iPhone, your MacBook, or your industrial robot.

This is why Apple’s "Apple Intelligence" and the "AI PC" cycle are so significant. It shifts the compute burden away from the cloud and onto the end device. This is a massive tailwind for companies like Cloudflare $NET ( ▲ 0.37% ) , which specialize in moving data quickly to the edge. You can read why I think Cloudflare is a generational opportunity here.

⛓️ The Connectors: Credo

As data centers transition to inference-dense racks, the "plumbing" of the data center has to change. We need faster ways to move data between chips. I’m keeping a close eye on Credo Technology $CRDO ( ▲ 3.73% ) . They are at the forefront of the shift from copper to optical connectivity.

Credo is projecting $600M in optical revenue by 2027. When you have net margins over 50% and are growing revenue at 200% year-over-year, you’ve found a serious pocket of growth in the AI infrastructure stocks sector.

🔍 Growth Investing Strategy: How to Play the Shift

If you’re looking for stock market research that cuts through the noise, the inference boom is where you should be spending your time.

Here is how I’m thinking about my growth investing playbook for the next 18 months:

1. Watch the Memory Bottleneck: Micron and SK Hynix are the gatekeepers. If they can’t ship, the whole AI train slows down.

2. Monitor the Power Contracts: Secured power is the new gold. I'm looking at firms with 5-10 year power agreements already signed.

3. Identify the "Edge" Winners: Apple and the PC manufacturers are about to see a massive upgrade cycle as users realize their old tech can't run the latest AI features.

4. Don't Ignore the "Plumbing": Companies like Credo are essential for making sure these massive clusters don't choke on their own data.

⚠️ Risks to Monitor

No thesis is without risk. I’m staying alert to a few key "red flags":

• Capex Fatigue: If Big Tech doesn't see a clear ROI on their AI spending soon, they might tap the brakes.

• Regulatory Scrutiny: Antitrust moves against the major GPU and chip makers could create short-term volatility.

• Grid Capacity: If the power grid truly can't handle the 220% surge in demand, growth will be capped by physics, not demand.

The inference era is going to be longer, more complex, and potentially more profitable than the training era. We are moving from the "how do we build it?" phase to the "how do we use it?" phase.

And in the world of investing, usage is what drives long-term value.

Stay alert,
George

Disclaimer: This post is for informational purposes only and does not constitute financial advice. Always conduct your own stock market research or consult with a financial advisor before making any investment.