AI Researcher Resigns From Anthropic Over Superintelligence Risks

Researcher Jacob Coxon has resigned from Anthropic after spending three years doing pretraining research there and at OpenAI. He accused both artificial intelligence giants of failing to act responsibly and gambling with human lives in a race for superintelligence.

Anthropic alignment science lead Evan Hubinger supported the warnings, stating that the firm believes AI has the potential to kill humans. Hubinger estimated the personal risk probability at greater than ten percent within the next decade.

“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.”

Coxon dismissed suggestions that the warnings were a marketing stunt, stating that senior researchers express fear privately while softening their public phrasing. Other industry figures also weighed in, including US Senator Bernie Sanders, who announced plans to introduce legislation to pause AI development.

Amid these concerns, OpenAI recently paused training on its latest models for two weeks in August before resuming under stricter controls. Research chief Jakub Pachocki subsequently called for extreme international coordination on future artificial intelligence development.