King Charles asks AI executives to urgently consider risks of AI falling into wrong hands

King Charles III has warned artificial intelligence executives of the “existential dangers” of the technology falling into the wrong hands, amid warnings about the rapidly advancing capabilities of frontier AI models.

The King made the comments on Thursday at a summit he convened at Dumfries House in Ayrshire, Scotland, to discuss how AI can be used to benefit society, making him the latest global leader to warn about the technology’s potential dangers.

The meeting brought together government officials, technology executives and other experts, including UK AI Minister Kanishka Narayan, an adviser to the Pope and representatives from AI companies including Nvidia, OpenAI and Anthropic.

What King Charles is saying

King Charles said those who developed AI technologies were increasingly warning about their potential to develop dangerous capabilities.

• “Those who have created these technologies are now increasingly warning that AI risks developing darker capacities – perhaps even to take life,” the King told executives at the summit.

He said there was an “urgency” to adequately consider the risks of advanced AI falling into the wrong hands and being used in potentially catastrophic ways.

• “In this regard, there seems urgency in adequately considering the existential dangers of such technologies falling into the wrong hands”, he said.
• “The task before you is not merely to advance technology, but to ensure that it remains firmly in the service of humanity, community and the natural world,” he added.

The summit was convened to consider whether participants could develop a “shared set of principles” to guide the future application of AI.

AI leaders weigh safety and progress

Nvidia CEO Jensen Huang, who attended the summit, said AI safety was paramount as the technology develops.

• Huang said companies should be prepared to hold back a product and continue engineering it if they believed it was not safe enough.
• He also highlighted the role of open models in ensuring that people and countries are not left behind in AI development.
• Huang said these systems should be protected so that researchers, universities, start-ups and countries can develop applications that “none of us would have imagined alone”.

Meanwhile, Sir Demis Hassabis, co-founder of Google DeepMind, said artificial general intelligence could be only a few years away.

Artificial general intelligence refers to AI systems believed to be capable of performing multiple tasks at human or above-human levels.

Hassabis said such technology could have an impact “ten times that of the Industrial Revolution”.

Get up to speed

Concerns over the risks of increasingly capable AI systems have intensified in recent weeks, with researchers, technology executives and financial regulators raising questions about the pace of frontier AI development.

• On September 15, Nairametrics reported that former Google DeepMind researcher Bilal Chughtai warned AI could “kill us all” and said humans may be running out of time to prevent such an outcome.
• Two days earlier, Anthropic CEO Dario Amodei called for a slowdown in frontier AI development, warning that the technology could advance faster than society’s ability to understand or control it.

The concerns also extend beyond potential existential risks. On August 31, the Financial Stability Board warned that the impact of frontier AI on cyber risk was the most immediate AI-related concern for global financial stability, citing the potential for AI to accelerate the discovery and exploitation of cyber vulnerabilities.

What you should know

The global AI race is also driving countries, including Nigeria, to invest in the infrastructure, skills and technological capabilities needed to participate in the emerging AI economy.

On September 13, Nairametrics reported that Nigeria was seeking deeper cooperation with BRICS countries on artificial intelligence, digital infrastructure, technology transfer and human-capital development.

The move comes as the Federal Government looks to attract investment and strengthen domestic technological capabilities.