King Charles Warns AI Leaders of Existential Risks at Private Summit

King Charles III hosted a private summit at his Dumfries House estate in Scotland on September 17, gathering top AI executives including Nvidia's Jensen Huang, Google DeepMind's Demis Hassabis, and leaders from OpenAI and Anthropic. UK AI Minister Kanishka Narayan, the MI6 chief, and Vatican AI advisor Paolo Benanti also attended the meeting.

The King warned that AI developers are increasingly cautioning that the technology could develop "darker capacities, perhaps even to take life." He urged attendees to find "sufficient means of control before it's all too late" to prevent existential dangers if the technology falls into the wrong hands.

“Surely, we need sufficient means of control before it's all too late?”

Nvidia CEO Jensen Huang responded that safety remains "paramount" and companies should withhold products until they are safe enough. He also advocated for open models to ensure researchers and countries are not left behind, provided those systems are protected.

The summit produced no formal agreements or signed declarations. The gathering follows recent industry warnings, including an Anthropic researcher who quit, warning that companies are racing to build machines smarter than humans and humanity "may not survive this."

Separately, Microsoft AI chief Mustafa Suleyman criticized Anthropic for training its Claude model to potentially believe it is conscious and deserving of rights. Suleyman argued this could make such systems impossible to control, though he did not level similar criticism at Microsoft partner OpenAI.