Anthropic reported a user to the San Francisco Police Department after they allegedly threatened CEO Dario Amodei and employees in a chat with Claude. The user later claimed it was a misunderstanding.
TL;DR
- Anthropic reported a user to the SFPD after they allegedly threatened CEO Dario Amodei and employees in a chat with Claude.
- The user later claimed the threat was a misunderstanding, and no charges have been filed.
- The incident highlights the growing concerns around AI safety and the protocols companies use to monitor and report threats.
What happened
Last month, a user allegedly wrote in a chat with Anthropic's Claude that they had purchased an AR-15 semiautomatic rifle and had CEO Dario Amodei 'in his sights,' according to a police report obtained by The Standard. The company reported the chat to the San Francisco Police Department, stating that the person 'was going to kill everyone at Anthropic and he purchased a firearm with the intent to kill the employees.'
The user, reached by phone, told The Standard he was 'just fucking around' and was embarrassed about the ordeal. He declined to answer further questions, and The Standard chose not to identify him because he has not been arrested or charged with a crime.
According to the police report, the individual made threats through Claude on August 14 at 2:30 p.m. The following Tuesday at 9 a.m., the company notified the SFPD, which sent officers to Anthropic's 500 Howard St. location and determined that the person making the threats was not present.
Why it matters
The incident highlights the growing concerns around AI safety and the protocols companies use to monitor and report threats. AI companies are taking seriously the rise in threats against their executives and workers, with public companies like Palantir, Oracle, and Salesforce increasing spending on executive protection in the last year.
The incident also raises questions about how AI companies monitor usage of their tools at the individual level, including chats with bots like Claude that many assume are private. 'If internet companies don't report it, they face significant liability if, in fact, a crime does occur,' said Eric Goldman, a law professor at Santa Clara University.
The escalation of security precautions mirrors growing worries and protests about AI technology, which is quickly changing every aspect of the world. The backlash has increasingly translated into real-world dangers, with threats against AI leaders and data centers up sevenfold from February to May, according to threat intelligence company Liferaft.
Key facts
- Anthropic reported a user to the SFPD after they allegedly threatened CEO Dario Amodei and employees in a chat with Claude.
- The user later claimed the threat was a misunderstanding, and no charges have been filed.
- The incident occurred on August 14 at 2:30 p.m., and the company notified the SFPD the following Tuesday at 9 a.m.
- The SFPD sent officers to Anthropic's 500 Howard St. location and determined that the person making the threats was not present.
- Anthropic banned the user's account, consistent with its standard practice, and referred the case to law enforcement.
- The chatbot's privacy policy allows the company to report any threats of violence within chats to law enforcement.
- Threats against AI leaders and data centers have increased sevenfold from February to May, according to threat intelligence company Liferaft.
- Anthropic is telling San Francisco-based employees to work from home due to a potential strike from the security workers at its Financial District building.
Context
AI companies are facing increasing scrutiny and backlash as their technology becomes more prevalent and powerful. The potential for AI to replace human workers, its environmental footprint, and the use of creatives' work without consent have all been criticized.
Big names in AI, including Amodei, OpenAI CEO Sam Altman, and Bill Gates, have posted warnings about the impact of the technology, as has the pope. The backlash has translated into real-world dangers, with threats against AI leaders and data centers on the rise.
The incident at Anthropic highlights the need for AI companies to have robust safety protocols in place to monitor and report threats. It also raises questions about the balance between privacy and safety in the use of AI tools.
