The AI disinformation gap the Pentagon may be missing

The Pentagon is getting better at catching deepfakes. DARPA’s Semantic Forensics program, the Defense Innovation Unit’s prototype contract with Hive, and a growing stack of detection tools all point the same way. The Department of Defense has decided that the disinformation fight is a detection problem.

They are wrong — not about deepfakes being a threat, but about where the real threat lives.

The detection tools the DoD is funding were built for a world where adversaries generate fake images, fake video and fake audio and then try to pass it off as real. That world exists. But it is the easy version of the problem. The harder version, the one showing up in research but not in budgets, is upstream. Adversaries are poisoning the AI models that defense analysts, intelligence platforms, and policymakers rely on to sort real from fake in the first place.

It is unclear whether the DoD has already encountered this problem in its own systems. But civilian researchers have documented it repeatedly. The Atlantic Council’s Digital Forensic Research Lab and the UK AI Security Institute have both published evidence of adversarial content embedded in the training data that feeds widely used AI models.

We have been tracking this from Israel since January, when Iran shut down its internet during domestic unrest. The shutdown was designed to silence protesters, but it also revealed how state-controlled information flows end up shaping what AI models learn about the region. Tehran has been running its own disinformation campaigns using AI-generated content for years. This is not speculation. It is documented.

But the problem goes well beyond Iran. US allies in the region have their own information controls. Israel restricts reporter access to Gaza and limits what journalists working inside the country can cover. Gulf states have tightly controlled reporting on Iranian missile strikes within their borders. Even the US government has pressed satellite imagery companies to limit what they publish, and several have complied. These restrictions shape what AI models learn about the region just as surely as Iranian censorship does.

The problem also includes US adversaries outside the Middle East.

In an Atlantic Council DFRLab audit of Common Crawl, the public web archive that feeds much of the world’s AI training pipeline, researchers found content from the Pravda network, a pro-Kremlin operation that has flooded the internet with millions of pieces of propaganda, baked directly into training data. Material from Glassbridge, a Chinese-government-adjacent influence operation, and Russia’s RT was in there too. Common Crawl did not respond to the findings. A major open-weights model reproduced this content nearly verbatim when prompted.

How fragile are these systems? An October study by Anthropic, the UK AI Security Institute, and the Alan Turing Institute found it takes as few as 250 malicious documents to compromise a large model. Once that content is baked into the weights, the only fix is a full retrain. Expensive. Impractical. Usually not done.

This is the gap. The Pentagon is building a better mousetrap while the mice are rewriting the blueprint for the house.

Detection tools flag synthetic media, meaning AI-generated images, video or audio, after it has been created. They do nothing about a model that has been quietly trained to favor certain narratives, minimize certain conflicts, or omit facts a state actor would rather the world forget. When a defense analyst asks an AI assistant to summarize South China Sea developments or analyze satellite imagery from the Middle East, the answer may already be shaped by an adversary’s influence operation. No deepfake detector catches that.

So what does closing this gap look like?

Data provenance standards for any AI model used in a defense or intelligence context: If we cannot verify what a model was trained on, we should not trust its outputs where accuracy matters. The Pentagon’s March announcement about exploring classified-data training is a step, but it only covers government-specific models. The broader ecosystem that defense analysts actually use remains unaddressed. Israel’s AI infrastructure buildout, including joint projects with US defense firms, shows what a provenance-first approach could look like in practice.

Red-teaming that goes beyond safety and bias checks: Current exercises test whether models produce harmful content when prompted. They do not test whether models have already been influenced by adversarial training data. That needs to change.

And most immediately, awareness: Most defense professionals I speak with have no idea training data poisoning is even possible, let alone that it has already been documented. That knowledge gap is itself a vulnerability.

The disinformation fight is not just about what is fake. It is about what the machines we trust to tell the difference are quietly being taught to believe.

Mark Ginsberg is a writer and analyst covering technology, security and geopolitical affairs. His work has appeared in the Times of Israel, where he writes on AI-driven propaganda, information operations, and the intersection of emerging technology with regional conflict.