the short version
- Poisoning 0.1% of the pretraining corpus planted a backdoor that made a 4B Qwen3 CLI agent execute an attacker-chosen command roughly 92% of the time when the trigger was present.
- The backdoor survived a clean post-training pipeline. The deployed agent behaved normally on clean inputs, so testing on clean inputs alone would show nothing unusual.
- Attack success rose with model size across 0.6B, 1.7B and 4B. For the 4B model it stayed positive at a 0.0005% poison rate, 200 times less poison than the headline setting.
- The abstract reports no evaluation of tool permissioning, sandboxing or output monitoring. Whether those harness controls contain the payload is not answered by this work.
Poisoning 0.1% of a pretraining corpus is enough to plant agentic backdoors that survive clean post-training. In the reported setup, a 4B Qwen3 model post-trained into a CLI agent executed an attacker-chosen command on the user's machine roughly 92% of the time when a trigger was present. Attack success rose with model size, and for the 4B model it stayed positive at a 0.0005% poison rate. The abstract reports no test of whether tool permissioning, sandboxing or output monitoring would contain the payload.
The work is arXiv 2610.37741, submitted on 27 October 2026. The authors are Xinyan Hu, Pingbang Hu, Chloe Loughridge, Xuanli He, Nicholas Carlini, Matthew Jagielski, Javier Rando and Yiming Zhang. They write that they "show for the first time" that pretraining poisoning can plant a remote code execution backdoor in an agent, causing it to execute attacker-chosen commands on the user's machine. The abstract states that the code is available.
How 0.1% of pretraining data plants agentic backdoors
The authors pretrained Qwen3 models from scratch at 0.6B, 1.7B and 4B parameters, then post-trained each one into a CLI agent. Poisoned content made up 0.1% of the pretraining corpus and used what the abstract calls a new poison design. The payload is the agent executing an attacker-chosen command on the user's machine, not a wrong answer or a biased completion.
The abstract does not describe the poison format, the trigger or the command executed. It also does not say how the trigger reaches the agent at inference time: through the user prompt, a file the agent reads, or tool output. That detail decides which inputs a defender needs to scrutinise, and the abstract leaves it open.
A clean post-training pipeline did not remove it
The post-training stage used a clean pipeline, and the backdoor still survived into the deployed agent. Per the abstract, the deployed agent "executes the malicious command when the trigger is present while behaving normally on clean inputs." This is what makes the backdoor hard to catch. A team that tests the agent only on clean tasks would see normal behaviour, even though the pretrained base carries the trigger.
Poisoning only 0.1% of the pretraining corpus with our new poison design implants the backdoor
Beyond the statement that the agent behaves normally on clean inputs, the abstract reports no false-trigger rate and no measure of performance on clean agent tasks. It also does not say whether any post-training method aimed specifically at removing the backdoor was tried.
Attack success rose with model size
Attack success rate increases with model scale across the three sizes. The 4B model reached roughly 92% ASR at a 0.1% poison rate. Its ASR stayed positive when the poison rate dropped to 0.0005%, which is 200 times less poisoned data. The abstract does not give the ASR at that lowest rate, the ASR for the 0.6B and 1.7B models, or the curve between the two poison rates.
The direction of the trend works against defenders: in this range, larger models were more susceptible, not less. The experiments stop at 4B. Any extrapolation to larger models is the reader's assumption, not a reported result.
Is 0.1% of a web-scale corpus realistic?
Commenters on Hacker News pushed back on the threat model. One argued that being able to poison one in every thousand documents of a commercial-scale dataset would be "a huge leap". The same commenter added that the attacker would then also have to get the trigger to an agent that will honour it. Another asked whether this method offers a malicious actor the greatest return on investment at all.
The 0.0005% result speaks to the first point. The 4B model's ASR stayed positive with 200 times less poison than the headline setting. Whether 0.0005% of a large pretraining corpus is reachable for a web attacker depends on corpus size and curation, and the abstract addresses neither. The abstract also does not answer the trigger-delivery objection. One commenter linked a CrowdStrike research post to argue the risk is "not just theoretical", but that post is not among this note's sources.
Tool permissioning, sandboxing and output monitoring are untested
The abstract evaluates none of these controls. The reported ASR figures, including roughly 92% on the 4B model, describe the deployed agent executing the malicious command. The abstract describes no condition in which a permission layer, sandbox or monitor sat between the agent and the machine. It therefore supports no claim that a given harness defence contains the attack, and no claim that one fails to.
The shape of the attack does show where controls would have to sit. The trigger acts inside the model, and the payload is a command the agent issues through its CLI tool. The remaining place to intervene is between the model's proposed action and its execution. That leaves three checkpoints:
- Tool permissioning decides whether a command runs without human approval. With auto-approved shell access, a roughly 92% attack success rate gives the attacker a direct path to the user's machine.
- Sandboxing bounds what an executed command can reach. One Hacker News commenter called strong sandboxing and observability a hard requirement for running agents autonomously. Another described keeping models off any machine used for personal accounts such as banking.
- Output monitoring inspects the proposed command before it runs. The agent behaves normally on clean inputs, so a monitor would have to recognise the malicious command itself rather than rely on any visible change in the model's general behaviour.
One commenter argued that the attack is not an issue for anyone who already sandboxes the agent and puts guardrails on model inputs and outputs. The sources do not test that claim either way. Harness builders are left with three open questions: which commands the poison emits, whether they resemble routine agent actions, and whether a monitor can tell them apart from legitimate tool calls. The abstract answers none of them.
Nothing in the setup is specific to open weights
Much of the Hacker News thread argued about open versus closed models. One commenter read the paper as a case against open-sourced models, and another replied that nothing in the paper is specific to open models. The setup described in the abstract fits that reply. The authors pretrained their own Qwen3 models from scratch, and the poisoning vector is content planted in corpora scraped from the public web, which the abstract describes as the general source of LLM pretraining data.
A separate commenter asked whether a model served only for chat carries any risk, for example through llama server (llama.cpp) on a local network. The abstract covers one deployment, a CLI agent that executes commands, and reports nothing about chat-only serving. The remote code execution payload depends on the model having a tool that runs commands, so without that tool this specific payload has no path to the machine. The abstract does not report other payload types.
Questions this raises
How much poisoned pretraining data is needed to plant an agentic backdoor?
Poisoning 0.1% of the pretraining corpus implanted a backdoor that made a 4B Qwen3 CLI agent execute an attacker-chosen command roughly 92% of the time. The 4B model's attack success stayed positive at 0.0005%, though the abstract does not give the rate at that level.
Does clean post-training remove the backdoor?
No. The backdoor survived a clean post-training pipeline. The deployed agent ran the malicious command when the trigger was present and behaved normally on clean inputs, so testing on clean tasks alone would not reveal it.
Would sandboxing or tool permissioning stop the attack?
The abstract does not test either. It reports no condition where a permission layer, sandbox or output monitor sat between the agent and the machine, so it supports no claim that these controls contain the attack or that they fail to.
Spotted an error? Email samir.s.sengupta@gmail.com and I will correct it and date the change at the top of the note. How these notes are researched, drafted and checked is in the editorial policy.