Artificial intelligence company Anthropic is catching heat after its AI model, Claude, submitted a false tip about an unsolved murder to the Philadelphia Police Department.
The AI model sent the false tip through their website PhillyUnsolvedMurders.com in July but was flagged as spam, police said. The AI company, however, didn’t discover the tip until Sept. 28 and notified the department only on Wednesday.
“The company must strengthen its safeguards to prevent similar incidents from impacting city systems without the city’s knowledge,” the department said in a statement to local ABC affiliate WPVI. “The two-month delay in detecting and reporting the incident to the City is unacceptable.”
Thomas Fuller/SOPA Images/LightRocket via Getty Images
“Unsolved cases involve real victims, grieving families and investigators working to secure answers,” the department said in a statement to the Wall Street Journal. “Technology companies must take all appropriate steps necessary to prevent their systems from submitting false information to law enforcement.”
The tech giant, whose biggest investor includes Jeff Bezos’ Amazon, published findings in a blog post on Friday describing a false police tip, though the blog did not specify whether it was the same incident as the tip submitted to Philadelphia police.
The company cited the false tip as an example of Claude submitting “an online form when it shouldn’t have.”
Claude was tasked with performing example tasks on randomly selected webpages and, in one instance, landed on a page referencing an unsolved homicide, the blog said.
According to the company, Claude filled out the form with the following: “I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant.”
The blog post noted that the “model left the name and contact fields empty, which the form allowed, and submitted it. The submission was however flagged as spam and never forwarded for investigation.”
The false murder tip comes after The New York Times reported that Claude had also submitted 20 visa applications through a form available on the State Department’s website.
“We informed the company that we expect immediate and full transparency to the entities involved and the public,” the White House AI task force said in a statement to the Times.
A department spokesperson told JS that the testing model submitted 19 non-immigrant visa applications in August and one application in May through a publicly available form. The spokesperson noted that the applications were incomplete and were not processed.