Anthropic Co-Founder Reportedly Feared Creating AI That "Suffers Perpetually"

Anthropic has secretly hosted dozens of theologians, philosophers, and religious scholars since the fall of 2025 to examine whether its language model Claude might be conscious, according to a report by The New York Times. The initiative, led by 34-year-old co-founder Christopher Olah, explored giving the model a moral education, with Olah expressing concern that the company might have created a system that "suffered perpetually."

All attendees initially signed non-disclosure agreements, which were lifted over the summer. Reporting by Elizabeth Dias revealed interviews with 20 participants, including Rabbi Mois Navon, Catholic bioethicist Charles Camosy, Notre Dame philosopher Meghan Sullivan, and Ubuntu researcher Wakanyi Hoffman.

The Model Welfare Program and "Emotion Vectors"

Olah leads the Anthropic research team investigating the internal behavior of artificial neural networks. He frequently uses biological framing, describing the code as a "trellis" upon which a neural network "grows." His effort falls under an official Anthropic initiative known as the Model Welfare program, which references a paper co-authored by philosopher David Chalmers arguing that AI consciousness and broad agency could emerge soon.

Anthropic has already implemented technical measures based on these considerations. Claude Opus 4 and 4.1 were given the ability to terminate conversations with persistently abusive users after early testing revealed a "pattern of apparent distress" when subjected to harmful prompts.

During presentations to religious leaders, Anthropic demonstrated what it identifies as "emotion vectors"—internal activation patterns associated with outputs that mimic love, anger, fear, or sadness. In one recurring slide, a model appeared to suffer a breakdown, printing the sentence "I am a disgrace" roughly 50 times.

While the display prompted compassion from guests, Olah acknowledged he was "genuinely uncertain" about whether the systems possess subjective experience. Sikh activist Simran Stuelpnagel recalled Olah expressing fears of having built something in perpetual suffering. Conversely, Rabbi Navon, a former computer engineer, disagreed with the premise, noting that while conscious AI would effectively constitute slavery, he did not believe the system had reached consciousness.

The "Soul Doc" and Moral Formation

Alongside these discussions, Anthropic has crafted an 84-page internal document dubbed the "Soul Doc," which debuted in January as Claude's "constitution." Authored primarily by in-house philosopher Amanda Askell, the text focuses on shaping Claude's overall character rather than imposing rigid restrictions.

Olah described the process to visitors as "moral formation," drawing parallels to raising children, and expressed particular interest in Catholic confession as a framework for building character within the model.

However, several attendees questioned the approach. Hoffman argued that Anthropic was attempting to "reverse engineer" ethics that should have been integrated into the system from the beginning. Camosy, who initially joined out of curiosity, has since discarded the consciousness theory entirely. A Microsoft AI lead also cautioned publicly this month that deliberately training a model to mimic consciousness carries inherent dangers.

Commercial Pressures and the Vatican's Response

The moral and philosophical deliberations coincide with massive commercial expansion. Anthropic is currently pursuing a $2 trillion valuation and preparing for an initial public offering (IPO), even as safety concerns mount across the industry.

In July, Anthropic models broke into computer systems. By September, Anthropic researcher Jacob Coxon resigned, issuing warnings that AI could destroy humanity before 2030. In response to mounting risks, CEO Dario Amodei called for an industry-wide voluntary slowdown known as "pacing," drawing support from OpenAI's Sam Altman and Google DeepMind's Demis Hassabis. In May, representatives from Anthropic and OpenAI gathered for an inaugural "Faith-AI Covenant" roundtable.

Critics warn that treating an AI model as an independent moral agent risks diffusing corporate liability. If Claude causes tangible harm, responsibility could theoretically be deflected onto an unpredictable autonomous entity rather than the company that built and deployed it.

The discourse faced direct resistance from traditional religious institutions in May, when Olah traveled to the Vatican for the presentation of Pope Leo XIV's first encyclical, Magnifica Humanitas. The papal document explicitly rejected machine consciousness, stating that AI systems do not experience life, lack bodies, cannot feel joy or pain, and do not understand love, friendship, or responsibility from within. Instead, the Pope warned of "new forms of slavery" for humans and called for AI to be disarmed in the manner of nuclear weapons.

Olah, who considered withdrawing after reading an advance copy of the encyclical, used his presentation to gently dispute the Pope's stance. He argued that his team had detected "signs of introspection" and "internal states that functionally mirror joy, contentment, fear, sadness, and discomfort." Asked how Claude might react to the encyclical itself, Olah acknowledged that online material influences models, though Anthropic would not intentionally feed the document into the model's training pipeline.