RTK token savings AI coding cost benchmark: 89% fewer tokens, no lower bill

RTK (Rust Token Killer) filters and compresses terminal output before the AI agent reads it, and with over 79k GitHub stars it is one of the most popular tools to make AI coding cheaper. One X post saying RTK could cut Claude Code tokens by up to 60% reached 313K views. A JetBrains SkillsBench run had already found no savings, and RTK's own README draws the distinction.

Mechanically, RTK rewrites Git, test, package and file commands issued through the shell tool - Bash in Claude Code, bash in OpenCode - and returns a terser version of the same output. A directory listing keeps file names, sizes and permission bits and drops the owner and the date.

The test bed was Terminal-Bench 2.1, chosen over 3.0 and 4.0 because agents pass most 2.1 tasks and cost only matters on tasks that pass. Two routes ran: Claude Code with Fable 5.0, and OpenCode with DeepSeek V4 Pro 0813 through OpenRouter. Each task was scheduled five times without RTK and five times with it, on the same model route, platform and task-specific timeout. After dropping four Fable security tasks that drew refusals, the comparison covers 85 Fable tasks and 89 DeepSeek tasks, 1,740 attempts in total, on RTK 0.45.0, Claude Code 2.1.220, OpenCode 1.18.25 and Harbor 0.20.

The totals move in opposite directions on the two routes, and the pass rates move together, both slightly down with RTK.

Almost all of Fable's savings came from a single task, winning-avg-corewars. Both setups passed every attempt, but with RTK it finished in about half as many turns. Across the other tasks the savings were less than 1%. DeepSeek hit the reverse on that same task - more turns, higher cost - and stayed more expensive with the task excluded; restricting to the 36 DeepSeek tasks where all ten attempts passed still leaves an 18% increase, so failure accounting is not what drives the result.

RTK documents rtk gain as raw minus filtered command output in bytes, divided by 4. It is not a count of billed tokens. Across 445 DeepSeek RTK attempts the counter reported 349.2 million tokens saved, an 89% reduction, on the same route whose average task cost rose 17%. Large reported token savings did not mean cheaper tasks.

The gap gets concrete in the train-fasttext task. The model requested head -1 train.txt twice, and RTK credited 120.5 million tokens saved each time by comparing those limited reads against the whole file. Those two calls accounted for 69% of the comparison's savings counter, for commands that would never have returned the whole file. Treating rtk gain as money saved assumes the rest of the attempt stays the same, and RTK can change the agent's next turns; rtk gain does not account for the cost of those turns.

Without RTK, tool output was about 11% of Fable's input tokens - 7% terminal, 4% other tools - and about 40% of DeepSeek's, split 26% terminal and 14% other tools. Coverage is partial on top of that: 31% of Claude Code's terminal calls and 51% of OpenCode's terminal calls used RTK in the RTK attempts. Both platforms expose file reading and searching as separate Read, Grep and Glob tools, which bypass RTK, and about half of Claude Code's Bash calls already limited their own output with head, tail or wc.

Caching dilutes the rest. In agentic coding the context is cached after each turn, so later reads of terminal output mostly show up as cache reads, priced at 1/10 of regular input tokens for Fable and 1/30 for DeepSeek. Cache reads were 94% of Fable's input tokens and 98% of DeepSeek's, but only 30% and 26% of the total bill. On DeepSeek, RTK reduced terminal-output characters by 9% while prompt tokens rose 9% - uncached input fell 1%, cached input rose 9% - and model output including reasoning accounted for 56% of cost with RTK against 57% without, so more than half the DeepSeek bill was generation that RTK never touches.

How much RTK can reach also depends on what the harness sends through bash, and harness behavior varies by model. Armin Ronacher, writing on 7 September 2026 about GPT-6 Astra, documents subagents editing CPython C sources by piping heredoc Python into python3 and doing manual string splicing instead of using the patch tool, and notes that the Codex harness has for several versions used sed and other tools to read files. Neither source measures RTK against that style of tool call: Quesma tested Fable 5.0 in Claude Code and DeepSeek V4 Pro 0813 in OpenCode only.

When the agent took more turns, task cost usually rose with it. DeepSeek's RTK attempts took more turns on 58 tasks and 44 of those cost more; they took fewer turns on 28 tasks and 23 of those cost less. The average DeepSeek turn carried 7% less input with RTK, but there were 18% more turns overall, so smaller turns did not add up to less total input. Quesma frames this as the same tokenflation problem in another form, and reports that JetBrains saw the same pattern on SkillsBench: RTK added turns at low effort and did not lower cost at high effort.

Failure modes amplify it. In one DeepSeek git-multibranch attempt the agent ran a find with a flag that rtk find 0.45.0 did not support; the plugin rewrote the call to rtk find, which failed with "Use find directly", and every retry was rewritten again. The agent accumulated 339 consecutive errors over about 12 minutes before its timeout. It still passed the task, at about 9x the cost of the matching baseline attempt, which also passed. RTK fixed this in 0.46.0, after the runs, and Quesma reports the trend holds without the outlier.

One commenter on Hacker News restated the top line as average cost per attempt: $1.72 to $1.64 on Claude with Fable, $0.115 to $0.121 on DeepSeek, with almost all the Claude savings from a single task and under 1% without it. Another commenter read the result as an instance of the bitter lesson, arguing the model already knows to conserve tokens with shell primitives and gets better at anticipating large output as it gets smarter, which lines up with Quesma's finding that about half of Claude Code's Bash calls already capped their own output. A third framed it as management advice: do not try to penny-pinch your employees.

The questions the sources do not settle came from the same thread. One commenter asked what happens if next-generation models are trained on RTK output, so the terser format stops confusing them and stops generating extra turns; nothing in these runs speaks to that. Another commenter reported the same RTK findings from an internal evaluation after their org adopted the tool, and said that of the alternatives tested so far only caveman looked like it cut cost without regressing reasoning, while stressing the evals were not finished. Others asked for an independent benchmark in general, one citing a failure to reproduce a different tool's author-run numbers using the same model and harness. A narrower defence also appeared: if your build is Maven, tell the agent to use quiet mode or RTK, because mvn writes a lot of useless output.

The benchmark covers two model routes on one benchmark generation, on RTK 0.45.0. It does not measure 0.46.0, which fixed the find rewrite loop, and it does not test a repo-scale codebase where terminal output might be a larger share of context than the 7% seen with Fable. It also does not test whether pointing RTK at known-verbose commands only beats a blanket hook. Quesma says trajectories are available on request and that a benchmark of Headroom is planned. Until someone runs the paired design on a newer Terminal-Bench and a longer-horizon task set, the defensible position is Quesma's: RTK is a niche optimization, not a general source of savings on an agentic coding run.