Top Scientists Warn Automated AI Research Could Trigger an 'Intelligence Explosion'
Over 20 leading researchers, including Geoffrey Hinton, Yoshua Bengio, and OpenAI's Jakub Pachocki, warn that automating AI development could rapidly outpace human oversight and erode global stability.
Prominent AI Figures Sound Alarm on Automated R&D
A group of more than 20 prominent artificial intelligence researchers—including Turing Award winners Geoffrey Hinton and Yoshua Bengio, alongside OpenAI research lead Jakub Pachocki—have issued a joint warning regarding the dangers of self-improving AI systems. In a newly published paper, the authors caution that automating artificial intelligence development could spark an uncontrolled "intelligence explosion." While acknowledging existing uncertainties, the researchers emphasize that "AI R&D automation might soon trigger one."
Compressing Years of Progress Into Months
According to the paper, automated systems already generate the majority of code inside the frontier organizations building them. The researchers project that the entire AI research and development workflow could be fully automated within a matter of years.
Once the R&D pipeline is self-directed, technological advancements that typically take years could occur in mere months. The authors warn that society may be unable to match this speed, creating severe hazards: humans could lose control of superhuman AI, and geopolitical and corporate power balances could quickly unravel. In response, the researchers are urging government officials and regulators to secure substantially greater visibility into how AI companies are automating their internal research.
Mounting Anxiety Across Frontier Labs
The warning follows a series of escalating concerns voiced within the artificial intelligence sector. Recently, 42 leading mathematicians issued a call for greater attention to existential threats stemming from AI, while multiple lab employees have voiced fears that unconstrained systems could destroy humanity.
The safety issue is also being highlighted from inside leading developers. Pachocki previously stated that no AI laboratory has made enough progress on alignment "to continue responsibly scaling at maximum speed for much longer." Amid mounting unease over uncontrolled technological trajectories, some employees at rival frontier firm Anthropic are reportedly already scouting safe havens in preparation for potential catastrophic scenarios.
