The CUDA Crack

0:00 Edmund Hale — This is Undertow, on Fathom FM. I'm Edmund, with Priya. Today, how changing a single word in a programmer's code could loosen Nvidia's grip on artificial intelligence.

0:10 Priya Raman — One word. Edmund, I have heard that promise in technology before, and it is usually false by the second paragraph.

0:16 Edmund Hale — Then let's test it. According to The Leadsman's piece on what it calls the crack in Nvidia's software moat, Amazon Web Services announced a tool called TorchNeuron on the nineteenth of October, with a public beta the next day.

0:29 Priya Raman — In plain English.

0:31 Edmund Hale — A lot of artificial intelligence code is written in a framework called PyTorch, and the article says that code assumes Nvidia hardware, through Nvidia's software platform, CUDA.

0:40 Edmund Hale — Amazon has its own chip, Trainium. TorchNeuron lets PyTorch treat it as just another device.

0:45 Priya Raman — And the one line?

0:47 Edmund Hale — Where the code says CUDA, you write neuron. According to the article, for most training jobs, that is the entire change.

0:53 Priya Raman — Most. Not all.

0:55 Edmund Hale — The article is careful about that. It is also released under an open licence, Apache two point zero, which allows commercial use and modification without royalties.

1:04 Priya Raman — Trainium has existed for a while. What was the problem before?

1:07 Edmund Hale — It needed a different compilation route, and developers had to adapt their code, because most production PyTorch is written in the eager style, not the style that route demanded.

1:17 Priya Raman — So the old barrier was effort, not arithmetic.

1:19 Edmund Hale — That is the heart of the piece. The article's claim is that Nvidia's moat was never mainly performance. It was switching cost.

1:27 Priya Raman — Why does it say that?

1:28 Edmund Hale — Years of code written in Nvidia's idiom: memory, debugging, distributed training. Even if a rival chip was cheaper, moving a production pipeline typically took, it says, weeks to months, with unpredictable errors to check.

1:40 Priya Raman — Is that measured?

1:41 Edmund Hale — No, and the article admits it. The view is widely shared among engineers, but the timelines are illustrative rather than precisely sourced. High confidence in the shape, not in the numbers.

1:52 Priya Raman — Now the second half. DeepSeek.

1:54 Edmund Hale — On the twentieth, DeepSeek released DeepInfer, a framework for running trained models, built on a tool called vLLM. The article takes this from an earlier The Leadsman piece.

2:03 Edmund Hale — It lists Huawei's Ascend nine fifty chips as co-equal with Nvidia's newest.

2:07 Priya Raman — Co-equal on a list. That is support, not speed.

2:10 Edmund Hale — That is the caveat. TorchNeuron attacks the training side, DeepInfer the serving side, where a trained model is delivered to users.

2:17 Edmund Hale — The article calls it closing the ring, and makes an inference. A team could train on Amazon's chip and serve on Huawei's, and need Nvidia at no point. It flags that no source states that scenario. It is The Leadsman's own reasoning.

2:31 Priya Raman — Built from two projects it says were not coordinated.

2:33 Edmund Hale — One from Seattle, one from Hangzhou. It sees no sign of deliberate coordination, just adjacent halves of the same problem.

2:41 Priya Raman — Now the awkward question. Is the chip any good? Easy to switch to is no use if it is slow when you arrive.

2:46 Edmund Hale — The article is blunt. TorchNeuron does not make Trainium as fast as Nvidia's top chips at peak training throughput for the largest models.

2:55 Priya Raman — So frontier models stay on Nvidia.

2:57 Edmund Hale — It says the biggest training runs need tightly coordinated memory and links across thousands of chips, where Nvidia has no confirmed peer. High confidence, though it adds that whether that holds into next year is uncertain.

3:09 Priya Raman — And benchmarks? If Amazon is confident, show me.

3:12 Edmund Hale — The article says Amazon has not published head-to-head benchmarks at that scale, in the documentation it reviewed. It marks that as established by absence.

3:21 Priya Raman — Established by absence. Honest, but also an admission that we are partly guessing. So on what basis does it say that below the frontier the answer may already be yes?

3:30 Edmund Hale — Performance documentation from Amazon and DeepSeek, as reported by TechCrunch and a technology site. It rates that claim only moderate confidence.

3:38 Priya Raman — So the vendors' own material. That is thin, and I'd want it said out loud.

3:42 Edmund Hale — It is thin, which is why the rating is moderate.

3:45 Priya Raman — So who is this really for?

3:46 Edmund Hale — Not the dozen organisations training the biggest models. It is the thousands doing fine-tuning and mid-sized training, where, it says, Trainium's price was always favourable but switching cost blocked adoption.

3:58 Edmund Hale — Its line is that this does not make Nvidia irrelevant at the frontier. It makes Nvidia substitutable for the large majority of the market below it.

4:05 Edmund Hale — The conclusion is about the premium. Nvidia's non-frontier hardware premium becomes structurally harder to defend, not through better performance, but because staying on CUDA is a weaker default.

4:16 Priya Raman — Let me push on that. Defaults are powerful. Plenty of organisations will never try the one word, because nobody gets blamed for buying the incumbent.

4:25 Edmund Hale — The article does not address that directly. It says the direction is robust, and the size is an assessed threshold, not a precise forecast.

4:33 Priya Raman — What is the number?

4:34 Edmund Hale — Within four quarters of the public beta, Trainium's share of cloud artificial intelligence training compute, measured in instance-hours sold, rises by at least three percentage points.

4:44 Priya Raman — Three points sounds tiny. But from what base?

4:47 Edmund Hale — It says from the pre-announcement baseline, and it does not state that figure.

4:51 Priya Raman — Then I cannot tell you whether three points is bold or trivial.

4:54 Edmund Hale — A fair complaint. It rates the call moderate confidence, direction more certain than magnitude.

5:00 Priya Raman — And how would we check?

5:02 Edmund Hale — Cloud providers' quarterly segment data, next year's third-quarter earnings calls, and analyst reports from firms like Morgan Stanley or Bernstein.

5:10 Edmund Hale — In the meantime, by its own admission, the missing evidence is benchmarks at scale. So watch for those, and for real migration stories, not just the headline.

5:18 Priya Raman — Because the headline says most training jobs, and the footnote says not all.

5:22 Edmund Hale — The article is honest in both directions. It calls the two releases the most significant simultaneous erosion of Nvidia's software lock-in in a decade, while saying neither closes the performance gap.

5:33 Priya Raman — A wall with a crack in it is a wall people start to test.

5:36 Edmund Hale — Whether they do is next year's question. This was Undertow, on Fathom FM.