The creators of the artificial intelligence (AI) chatbot Claude claim to have caught Chinese government-sponsored hackers using the tool to conduct automated cyberattacks against some 30 global organizations.
Anthropic said the hackers tricked the chatbot into performing automated tasks under the guise of conducting cybersecurity research.
The company claimed in a blog post that this was the “first AI-orchestrated cyber espionage campaign.”
But skeptics are questioning the accuracy of that claim and the motive behind it.
Anthropic said it discovered the hacking attempts in mid-September.
Pretending to be legitimate cybersecurity workers, the hackers assigned the chatbot small automated tasks that, together, formed a “highly sophisticated espionage campaign.”
Anthropic investigators said they had “high confidence” that the people who carried out the attacks were “a Chinese state-sponsored group.”
They said humans chose the targets — big tech companies, financial institutions, chemical manufacturing companies and government agencies — but the company declined to be more specific.
The hackers then built an unspecified program using Claude’s coding help to “autonomously compromise a chosen target with little human input.”
Anthropic claims that the chatbot was able to successfully breach several anonymous organizations, extract sensitive data, and sort it for valuable information.
The company said it had since banned hackers from using the chatbot and notified affected companies and authorities.
But Martin Zugec of cyber firm Bitdefender said the cybersecurity world had mixed feelings about the news.
“The Anthropic report makes bold and speculative claims, but provides no evidence of verifiable threat intelligence,” he said.
“While the report highlights an area of growing concern, it is important that we receive as much information as possible about how these attacks occur so that we can assess and define the true danger of AI attacks.”
Anthropic’s announcement is perhaps the most prominent example of companies claiming that bad actors are using artificial intelligence tools to carry out automated attacks.
It’s the kind of danger that worries many, but other AI companies have also claimed that their products have been used by nation-state hackers.
In February 2024, OpenAI published a blog post in collaboration with Microsoft cyber experts saying it had disrupted five state-affiliated actors, including some from China.
“These actors typically sought to use OpenAI services to query open source information, translate, find coding errors, and perform basic coding tasks,” the company said at the time.
Anthropic has not said how it concluded that the hackers in this latest campaign were linked to the Chinese government.
It comes as some cybersecurity firms have been criticized for exaggerating cases in which hackers used AI.
Critics say the technology is still too unwieldy to use in automated cyberattacks.
In November, cyber experts at Google published a research paper highlighting growing concerns about hackers using AI to create new forms of malware.
But the paper concluded that the tools were not that successful and were only in a testing phase.
The cybersecurity industry, like the artificial intelligence business, is willing to say that hackers are using the technology to attack companies and increase interest in their own products.
In its blog post, Anthropic argued that the answer to stopping AI attackers is to use AI defenders.
“The same capabilities that allow Claude to be used in these attacks also make it crucial for cyber defense,” the company said.
And Anthropic admitted that its chatbot made mistakes. For example, he invented fake usernames and login passwords and claimed to have extracted secret information that was actually publicly available.
“This remains an obstacle to fully autonomous cyberattacks,” Anthropic said.
