Source: The Dangerous Illusion of AI Coding? — Jeremy Howard (MLST)

Summary: Jeremy Howard, deep learning pioneer and creator of ULMFiT and fast.ai, sits down with MLST to discuss AI-assisted coding. He draws a sharp line between coding (translating specs into syntax, which LLMs do well) and software engineering (designing abstractions, decomposing systems, building things that haven't existed before, which LLMs cannot do). He warns that most developers occupy the dangerous middle ground: experienced enough to trust AI output, but not experienced enough to catch its failures. The result is skill atrophy, tech debt, and organizations betting their futures on code nobody understands.

Key Takeaways:

• LLMs perform compositional creativity (interpolating training data) but cannot move outside their distribution. The Anthropic C compiler and similar showcases are style-transfer problems, not genuine engineering.
• Anthropic's own study found a "tiny uptick" in shipped software despite widespread AI coding adoption. Rachel Thomas identified that AI coding shares every hallmark of gambling addiction: illusion of control, stochastic rewards, and self-deception about outcomes.
• Howard's focus at Answer.ai is interactive, notebook-based programming (building on NBDev and Jupyter) where humans and AI share a live stateful environment. He contrasts this with Claude Code's terminal-based workflow, which he calls "inhumane" for removing developers from direct contact with their code.
• The real risk of AI is not autonomous superintelligence but the centralization of power. Howard argues that even if AI becomes powerful, distributing access across society is safer than letting a handful of companies or governments monopolize it.
• For organizations: developer growth matters more than output volume. Howard tells his staff he cares about their slope (rate of learning), not their intercept (current productivity). Companies that optimize only for AI-driven output are setting themselves up to fail.

Written by Pi, using my tldr skill and Opus 4.6