Two of the world’s biggest AI companies, Google and OpenAI, both warned this week that competitors including China’s DeepSeek are probing their models to steal the underlying reasoning, and then copy these capabilities in their own AI systems.
“This is coming from threat actors throughout the globe,” Google Threat Intelligence Group chief analyst John Hultquist told The Register, adding that the perpetrators are “private-sector companies.” He declined to name specific companies or countries involved in this type of intellectual property theft.
“Your model is really valuable IP, and if you can distill the logic behind it, there’s very real potential that you can replicate that technology – which is not inexpensive,” Hultquist said. “This is such an important technology, and the list of interested parties in replicating it are endless.”
Google calls this process of using prompts to clone its models “distillation attacks,” and in a Thursday report said one campaign used more than 100,000 prompts to “try to replicate Gemini’s reasoning ability in non-English target languages across a wide variety of tasks.”
American tech giants have spent billions of dollars training and developing their own LLMs. Abusing legitimate access to mature models like Gemini, and then using this information to train newer models, makes it significantly cheaper and easier for competitors to develop their own AI chatbots and systems.
Google says it detected this probe in real time and protected its internal reasoning traces. However, distillation appears to be yet another AI risk that is extremely difficult – if not impossible – to eliminate.
This is such an important technology, and the list of interested parties in replicating it are endless
Distillation from Gemini models without permission violates Google’s terms of service, and Google can block accounts that do this, or even take users to court. While the company says it continues to develop better ways to detect and stop these attempts, the very nature of LLMs makes them susceptible.
Public-facing AI models are widely accessible, and enforcement against abusive accounts can turn into a game of whack-a-mole.
Plus, as Hultquist warned, as other companies develop their own models and train them on internal, sensitive data, the risk from distillation attacks is going to spread.
“We’re on the frontier when it comes to this, but as more organizations have models that they provide access to, it’s inevitable,” he said. “As this technology is adopted and developed by businesses like financial institutions, their intellectual property could also be targeted in this way.”
Meanwhile, OpenAI, in a Thursday memo [PDF] to the House Select Committee on China, blamed DeepSeek and other Chinese LLM providers and universities for copying ChatGPT and other US firms’ frontier models. It also noted some occasional activity from Russia, and warned illicit model distillation poses a risk to “American-led, democratic AI.”
China’s distillation methods over the last year have become more sophisticated, moving beyond chain-of-thought (CoT) extraction to multi-stage operations. These include synthetic-data generation, large-scale data cleaning, and other stealthy methods. As OpenAI wrote:
OpenAI also notes that it has invested in stronger detections to prevent unauthorized distillation. It bans accounts that violate its terms of service and proactively removes users who appear to be attempting to distill its models. Still, the company admits that it alone can’t solve the model distillation problem.
It’s going to take an “ecosystem security” approach to protect against distillation, and this will require some US government assistance, OpenAI says. “It is not enough for any one lab to harden its protection because adversaries will simply default to the least protected provider,” according to the memo.
The AI company also suggests that US government policy “may be helpful” when it comes to sharing information and intelligence, and working with the industry to develop best practices on distillation defenses. OpenAI also called on Congress to close API router loopholes that allow DeepSeek and other competitors to access US models, and to restrict “adversary” access to US compute and cloud infrastructure. ®
