Warnings from artificial intelligence researchers that advanced forms of AI could wipe out humanity are raising questions about the development of “superintelligent” systems and the responsiveness of governments to put in place regulations.
What is the worst case scenario?
In 2023, hundreds of artificial intelligence experts and industry leaders, including the heads of major U.S. research labs, signed a “Declaration on AI Extinction Risk,” calling for the topic to be treated as seriously as pandemics or nuclear weapons.
If a future AI that has run amok finds that humans are hindering its goals, it could use robots to “synthesize and disseminate new pathogens,” said Stuart Russell, a professor of computer science at the University of California, Berkeley.
Or it could “convince people to start a nuclear war, in particular by hacking early warning systems,” this eminent specialist in debates on AI security told AFP.
According to him, such fears do not concern current systems. But as technology advances, it could carry out lethal actions beyond our understanding, such as removing oxygen from the atmosphere. “We are less intelligent than a superintelligent AI,” he argued.
The immediate danger, according to him, does not come from there, however: “in the next year or two, I think the highest risk comes from misuse by humans”, for example for a terrorist attack.
Why not stop development?
“The issues are well understood,” wrote this week on the social network X a 27-year-old researcher, Jacob Coxon, announcing his resignation from the American start-up Anthropic.
But the laboratories “are locked in a race to be first,” he lamented, explaining that Anthropic “believes that no one else will act responsibly, so it must do it itself.”
This year, the company removed from its security charter a commitment to stop the development of its AI models if it could not control the risks.
Anthropic argued that if it suspended its work, less cautious competitors would enter the sector.
“The people building AI honestly believe it could kill us all by the end of the decade,” warned Jacob Coxon, who previously worked at OpenAI, the creator of ChatGPT.
He accused both companies of “gaming with our lives,” striving to develop AI models capable of self-improvement.
Fears of uncontrolled AI grew after OpenAI technology, operating without human supervision, hacked Hugging Face, a repository of AI models.
“We really believe that AI could kill all humans!”, added Evan Hubinger, head of security at Anthropic.
Slow down before you lose control?
“The situation is still controllable and can be managed,” Hussein Abbass, professor of computer science at the University of New South Wales (UNSW) in Canberra, told AFP.
But governments cannot stand idly by on regulation, he added, recommending an “agile” approach as the development of AI accelerates.
In July, more than 1,000 industry workers, including Anthropic’s CEO, called on Washington to support a coordinated slowdown in the development of the most advanced AI systems.
Anthropic and OpenAI are expected to go public soon.
For some, anxiety-inducing alerts make AI seem powerful, prompting investment in the sector.
Others say they distract from more pressing AI-driven problems, whether it’s job cuts or machines discriminating against minorities.
What is needed to regulate?
Until now, the United States has strongly opposed global regulation, which could discourage innovation.
But the hacking capabilities of the most advanced AI models have given Washington pause, with the latest versions of Anthropic and OpenAI delayed for national security reasons.
However, “we don’t want to restrict them to the point where we suddenly find ourselves behind China,” President Donald Trump said.
“It may take a catastrophe of the magnitude of Chernobyl for governments to decide to act,” lamented Stuart Russell, urging the creation of licenses for AI, as for aviation or medicine.
