Distillation Attacks

When AI labs in China use repeated queries to American frontier models to generate synthetic data, then use that data to train competing models that replicate the original model's capabilities. Anthropic's May 2026 policy paper described how Chinese labs carry out large-scale distillation attacks to harvest innovations from US AI companies.

The technical mechanism involves querying a target frontier model millions of times across diverse inputs, collecting the outputs, and using that data to fine-tune a separate model. The attacking lab effectively trains its own model to imitate the target's capabilities without ever accessing the target's architecture or weights directly. This is distinct from knowledge distillation, which is a legitimate training technique where a smaller model learns from a larger model's outputs.

These attacks are controversial. US companies view them as a form of intellectual property theft, arguing that the model's outputs represent proprietary innovations. Defenders argue that black-box querying is difficult to prevent since APIs are openly accessible. The practice exploits the fact that frontier models are often accessible via public APIs, making it hard to distinguish legitimate use from systematic capability extraction.

The scale of these attacks is hard to verify independently. Anthropic and other US labs claim they are widespread and systematic, while Chinese labs typically do not publicly discuss them. The ethical and legal status remains contested: querying an open API may be technically legal while still being seen as a violation of terms of service or a form of unfair competition.