Hugging Face: IT security incident at AI platform

The AI platform Hugging Face for sharing open-source AI models and datasets was the victim of an IT attack. It was likely initiated by an AI – and also contained using AI.

This is reported by Hugging Face in a blog post. According to the report, the operators noticed a breach into parts of the production infrastructure last week and took action against it. It differed from previous incidents in that it was entirely executed by artificial intelligence, by an autonomously acting AI system. Hugging Face states that it largely discovered and analyzed it with AI.

Unauthorized Access

The attack was discovered through unauthorized access to a limited number of internal data and several credentials used by internal services. The analysis is still ongoing, whether partner or customer data were also affected; the operators promise to contact all affected parties directly. Hugging Face found no indications that public models, datasets, or spaces accessible to users were manipulated. The software supply chain, such as container images and published packages, has been reviewed and found to be clean.

Hugging Face explains that the attacker AI targeted the data processing pipeline and transmitted a malicious dataset that exploited two code execution paths in data processing. Specifically, a remote code dataset loader and template injection in a dataset configuration were used. These executed their code in a worker process. This gave the attackers access to the nodes and allowed them to exfiltrate cloud and cluster credentials. They also spread further into several internal clusters.

The campaign was driven by an autonomous agent framework, apparently programmed for agentic security research and utilizing a previously unknown LLM. It performed many thousands of individual actions via a swarm of short-lived sandboxes. The command-and-control infrastructure migrated itself and relied on public services. This fits the description of an “agentic attacker” scenario that the IT security industry has already predicted.

Countermeasures

Hugging Face has closed the vulnerabilities that enabled code execution in dataset processing. They have kicked the attackers off the network and re-provisioned the compromised nodes. The operators have also rotated and revoked the affected credentials and tokens. Additionally, they have implemented further security mechanisms and stricter controls on the clusters. They have also tightened the detection and alarm mechanisms. Hugging Face recommends users revoke and rotate credentials and tokens and review recent account activity.

(dmk)