'The Bitter Lesson' Author Warns Compute Alone Cannot Deliver True Intelligence
Turing Award winner Richard Sutton argues that raw compute scaling is insufficient for AI intelligence, sparking a widespread debate among researchers and tech executives over the future of LLMs.
Compute Alone Is Not Enough
In a new post that sparked broad discussion across the AI community, Turing Award winner Richard Sutton warned that relying on raw compute scaling alone will not yield true intelligence. Sutton, author of the influential 2019 essay "The Bitter Lesson," emphasized that AI science is "far from finished" and urgently requires clever new learning methods.
In his original 2019 essay, Sutton argued that general methods leveraging compute inevitably beat complex, human-engineered approaches over time. However, his latest statements clarify that raw compute without architectural and algorithmic innovation is insufficient, challenging the prevailing reliance on large language model (LLM) scaling.
Skepticism Over Synthetic Data and Static LLMs
Sutton's latest critique aligns with his past statements on the structural limits of scaling existing models. He described the industry's push toward synthetic data as a "big mistake," noting that the real world is infinitely complex while any computer simulation remains merely "microscopic." He also identified human expertise as a bottleneck that actively prevents true scaling.
According to Sutton, today's language models suffer from a fundamental limitation: they stop learning once their training phase concludes, and internet text alone cannot capture real-world complexity. Instead, he advocates for autonomous agents equipped with world models that continuously learn from their own direct experiences without erasing previously acquired knowledge.
Scientists Warn of 'Reward-Hacked' Research
Sutton's position is shared by several prominent researchers seeking alternative paths forward. Former OpenAI researcher Jerry Tworek, who now leads Core Automation, noted that while the fundamental premise of the bitter lesson remains true, the choice of algorithm matters exponentially more as compute increases.
Tworek argued that the temporary success of transformer scaling, coupled with a misreading of Sutton’s original thesis, has "reward-hacked the brains of a whole generation of scientists." He added that while some algorithms scale effectively, others hit dead ends, leaving numerous scaling dimensions entirely unexplored. Core Automation is among 45 new AI labs tracked in the Frontier Radar #5 report exploring algorithms beyond standard pretraining and reinforcement learning.
AI researcher Jayden Teoh added that academic research has frequently suffered because promising ideas were prematurely dismissed for straying from the spirit of the bitter lesson. "An idea doesn’t necessarily have to survive scaling forever to be an important contribution to AI progress," Teoh wrote, arguing that stepping-stone ideas remain vital even if cleaner, more general approaches replace them later.
AI Industry Leaders Push Back
Industry leaders whose business models rely heavily on transformer scaling offered a sharply contrasting perspective. OpenAI CEO Sam Altman declared that LLM scaling is far from exhausted, accusing critics of slowing down a generation of researchers by declaring prematurely what scaling cannot accomplish.
As evidence of ongoing progress, Altman cited a recent OpenAI model that successfully disproved a mathematical conjecture long unsolved by human mathematicians. While Altman admitted world models are useful in domains such as robotics, he maintained that data strongly favors continued LLM scaling.
Anthropic CEO Dario Amodei expressed equal optimism regarding current scaling trajectories, describing progress in the field as a "rainbow with no end."
