“Don’t invest in extinction”: on the day of the IPO of the creator of Claude, around a hundred demonstrators braved the storms Thursday in front of Anthropic’s headquarters in San Francisco to denounce its race towards ever more powerful AI.
Five days after a march to demand an international treaty on AI in front of the headquarters of OpenAI, the creator of ChatGPT and great rival of Anthropic, those gathered were young, mostly men, including many AI security researchers and a few students from the University of Berkeley.
“These people are going to make a profit from a technology that they themselves admit could eliminate human life on Earth,” says Tarnel Abbott, a 72-year-old retired librarian.
“Most people in the country, even in San Francisco, don’t know what’s going on (…) It’s almost a black box,” observes Emily Tan, 31, from Los Angeles.
First slogan: the ban on “recursive self-improvement” (RSI in English), that is to say the stage where AIs themselves design the next generation, more powerful and more and more quickly, at the risk of escaping all human control.
“There are only a few places in the world where this is possible in 2027, and this office is one of them,” Jacob Coxon, a young researcher whose resignation from Anthropic and his warnings caused a stir in September, said into the megaphone.
“Everyone I talk to” in Silicon Valley “says privately that they don’t want to see this in 2027,” he assured.
Anthropic, which made the largest IPO in history on Thursday, assures that it has not yet reached the RSI stage. It recognizes that this development, which it considers avoidable, could increase the risk, abundantly described to investors, of losing control of AI.
The “good guys”
“They talk about slowing down (…) but they’re not really trying, and they’re going public today making a lot of money on possible projects” for self-improvement, criticizes Tyler Tracy, 26, a researcher at Redwood Research, a non-profit organization specializing in AI security.
Formerly of OpenAI, Daniel Kokotajlo says he has “a lot of friends at Anthropic and OpenAI”. When he tells them of his fears, they respond: “If we don’t do it, OpenAI will. Or China will. And we are the responsible good guys.”
“I don’t like this answer, that’s why I’m protesting,” says this thirty-year-old who heads the AI Futures Project, a foresight organization.
“Look at any other sector: the FDA (the drug agency), the FAA (the civil aviation regulator)… We don’t have that in tech, while some of the biggest issues of the moment are being played out there for every inhabitant of the planet,” argues Emily Tan.
“You said you would not race in the lead,” recalls one sign.
According to their opponents, the founders of Anthropic assured in their early days that they would never be the first to release the most advanced AI.
Most of the demonstrators met by AFP, however, use AI and consider it useful, but believe that it is progressing too quickly, without safeguards.
The day before, an open letter signed by researchers, entrepreneurs and academics had accused Anthropic of not being “a safe AI company”.
The only counter-protester, Barak Gila, 31, an activist blogger, brandished a “Thank you, Anthropic team” sign in front of the crowd, extolling the merits of the technology and the scale of the local economic benefits.
When a demonstrator comes to challenge him, Barak Gila, all smiles, seems delighted to respond: “Without Anthropic, I’m going to die, that’s for sure. Whereas with Anthropic, I will live perhaps ten years longer,” he tells her, with a provocative air. “Or maybe you will die in two years!”, retorts his young opponent.