“Agentic AI systems are being weaponized.”
That’s one of the first lines of Anthropic’s new Threat Intelligence report, out today, which details the wide range of cases in which Claude — and likely many other leading AI agents and chatbots — are being abused.
First up: “Vibe-hacking.” One sophisticated cybercrime ring that Anthropic says it recently disrupted used Claude Code, Anthropic’s AI coding agent, to extort data from at least 17 different organizations around the world within one month. The hacked parties included healthcare organizations, emergency services, religious institutions, and even government entities.
“If you’re a sophisticated actor, what would have otherwise required maybe a team of sophisticated actors, like the vibe-hacking case, to conduct — now, a single individual can conduct, with the assistance of agentic systems,” Jacob Klein, head of Anthropic’s threat intelligence team, told The Verge in an interview. He added that in this case, Claude was “executing the operation end-to-end.”
Anthropic wrote in the report that in cases like this, AI “serves as both a technical consultant and active operator, enabling attacks that would be more difficult and time-consuming for individual actors to execute manually.” For example, Claude was specifically used to write “psychologically targeted extortion demands.” Then the cybercriminals figured out how much the data — which included healthcare data, financial information, government credentials, and more — would be worth on the dark web and made ransom demands exceeding $500,000, per Anthropic.
“This is the most sophisticated use of agents I’ve seen … for cyber offense,” Klein said.
In another case study, Claude helped North Korean IT workers fraudulently get jobs at Fortune 500 companies in the U.S. in order to fund the country’s weapons program. Typically, in such cases, North Korea tries to leverage people who have been to college, have IT experience, or have some ability to communicate in English, per Klein — but he said that in this case, the barrier is much lower for people in North Korea to pass technical interviews at big tech companies and then keep their jobs.
With the assistance of Claude, Klein said, “we’re seeing people who don’t know how to write code, don’t know how to communicate professionally, know very little about the English language or culture, who are just asking Claude to do everything … and then once they land the job, most of the work they’re actually doing with Claude is maintaining the job.”
Another case study involved a romance scam. A Telegram bot with more than 10,000 monthly users advertised Claude as a “high EQ model” for help generating emotionally intelligent messages, ostensibly for scams. It enabled non-native English speakers to write persuasive, complimentary messages in order to gain the trust of victims in the U.S., Japan, and Korea, and ask them for money. One example in the report showed a user uploading an image of a man in a tie and asking how best to compliment him.
In the report, Anthropic itself acknowledges that although the company has “developed sophisticated safety and security measures to prevent the misuse” of its AI, and though the measures are “generally effective,” bad actors still sometimes manage to find ways around them. Anthropic says that AI has lowered the barriers for sophisticated cybercrime and that bad actors use the technology to profile victims, automate their practices, create false identities, analyze stolen data, steal credit card information, and more.
Each of the case studies in the report adds to the increasing amount of evidence that AI companies, try as they might, often can’t keep up with the societal risks associated with the tech they’re creating and putting out into the world. “While specific to Claude, the case studies presented below likely reflect consistent patterns of behaviour across all frontier AI models,” the report states.
Anthropic said that for every case study, it banned the associated accounts, created new classifiers or other detection measures, and shared information with the appropriate government agencies, like intelligence agencies or law enforcement, Klein confirmed. He also said the case studies his team saw are part of a broader change in AI risk.
“There’s this shift occurring where AI systems are not just a chatbot because they can now take multiple steps,” Klein said, adding, “They’re able to actually conduct actions or activity like we’re seeing here.”
0 Comments
