the short version
- With models and settings held constant, two versions of jev-ultrafast differed by 9.450 s versus 7.092 s median task time and 1,092 versus 101 median browser protocol calls.
- The HarnessTax page currently exposes only its title and publication date, so no claim about which harness wins on which model traces to the published material.
- Commenters on Hacker News argue the measurable part of a harness is mostly system prompt and tool-description weight, and that provider middleware changes tool-call success for the same model.
- OpenSpec (MIT, v1.13.0) is a harness choice too: it changes what enters the context and what the agent is verified against without changing the model.
No source in this material quantifies how much of a coding agent's benchmark score comes from the harness. The one harness-only measurement that is published sits in browser-use's jev-ultrafast: six alternating runs with identical models and settings, both versions passing 3/3, median task time falling from 9.450 s to 7.092 s (a 25% reduction) and median browser protocol calls falling from 1,092 to 101. HarnessTax, published 16 September 2026 at harnesstax.github.io under the title "HarnessTax: How Much Does the Harness Matter for Coding Agents?", asks the question its name implies; its page body is not retrievable, so its methodology, model list and scores cannot be asserted here.
The Hacker News thread on HarnessTax is where practitioners name the axes they expect to matter: system prompt and tool-description weight, sandboxing, execution pattern, and provider-side middleware. Those are reactions rather than findings, and they are labelled as such below.
HarnessTax publishes only a title so far
The retrievable material for HarnessTax is its title, publisher and 16 September 2026 publication date. There are no scores, no model roster, no cost-per-task figures and no definition of what the benchmark counts as a harness. Commenters on Hacker News describe it as narrower than the word suggests: one writes that the size of the system prompt "is what HarnessTax is mostly measuring", and another quotes a finding, "Provider-specific optimization does not guarantee the best pairing", and calls it "probably correct, depending on how you measure". Neither characterisation is confirmed by the published page, so an engineer planning a migration should treat both as hearsay until the body text is readable.
If a harness benchmark does reduce to prompt and tool-description weight, it measures one axis of a multi-axis system and will misrank harnesses that differ on the others. Token weight is also a weak proxy for spend, which is worth checking before a harness change is justified on price.
A 25% cut with the model fixed
jev-ultrafast is a browser agent, not a coding agent, and it is the only harness-only ablation in this material. The repository, published 17 September 2026 with 2k stars, 108 forks and 2 commits, reports six alternating runs with identical models and settings: both versions passed 3/3, median task time went from 9.450 s to 7.092 s, a 25% reduction, and median browser protocol calls went from 1,092 to 101. The README states the limit plainly: three repeats of one task on one browser profile, not a general reliability benchmark.
The headline demo is a 7,073 ms Google Flights run, Zürich to London, timed after initial page observation and inclusive of model calls, generated text, browser work, stale decisions and loading waits. A fresh independent check verifies the one-way setting, Zürich, London, September 20, 2026, and visible flight options. The same policy opened the requested Wikipedia article in 2.798 s and passed a local hotel search and filter task in 1.896 s. Runs, failures, source hashes and measurement boundaries are in performance.md.
Which harness choices moved those numbers
The design is specific enough to copy. Every observation produces a new element table of indexed controls, and the operations are fixed: CLICK, TYPE_TEXT, SELECT, SCROLL_UP, SCROLL_DOWN, WAIT, DONE and BLOCKED, with only supported operations and targets offered. One TypeSafe request returns an operation plus speculative target heads - click_target, type_text_target and select_target if present - so it is "Two decisions, one network round trip", and each target head contains only compatible elements. A small LLM writes text only when the operation is TYPE_TEXT; the current demo uses inception/mercury-2.5 with reasoning disabled, keyed through OpenRouter in the example configuration.
- One request per decision cycle, with operation and target heads sharing the same observed state.
- No screenshots in the default agent loop - Jev consumes structured state; the inspector opts into screenshots and the video uses a separate continuous screencast.
- One browser call per snapshot: read visible controls, their names, values and text atomically, keeping references to the actual DOM nodes.
- Bounded waits: after typing into a combobox, wait for visible suggestions capped at 200 ms; other interactions get at most two animation frames or 50 ms.
- Send visible text only, so offscreen article bodies and footers do not fill the model context.
- Focus emulation keeps hidden tabs rendering without switching Chrome's visible tab.
One safety property here is a harness choice rather than a model choice: model output never becomes selectors, coordinates, shell commands or executable JavaScript. Every executed target is resolved from an observed node, the executor rechecks page freshness and click occlusion, and text-helper output must parse as a small JSON object before typing. Chrome connects through Browser Harness, installed by uv sync, with uv run browser-harness --doctor if it needs connecting.
from jev_ultrafast import Agent
with Agent(
"https://www.google.com/travel/flights?hl=en",
"Find one-way flights from Zurich to London on September 20, 2026, "
"for one adult in economy. Stop when matching flight options are visible.",
) as agent:
for state in agent.run():
print(state["elapsed_ms"], state["status"])Where practitioners say the framing is misread
The strongest pushback on Hacker News is that calling harness weight a tax prices out what the weight buys. One commenter writes that "Pi actively omits any sort of guardrails and sandboxing in the name of speed and simplicity, so it is not shocking that it is faster and simpler." Replies dispute the premise: that the majority of Claude Code's bloat is tool descriptions rather than security text, that the security is handled by "the evaluator model that sometimes denies some tool uses" rather than by the prompt, and that wrapping Pi in nono.sh "costs approximately zero tokens". One commenter puts Claude Code's initial instructions to a model at a dump of 80kb of text; none of the sources here verify that figure or settle the dispute.
A second objection is scope. One commenter argues the word harness is being overloaded with agent, and that execution pattern - parallel versus sequential, delegation to subagents, not being stuck on one model - affects cost and performance greatly regardless of the instruction text, making single-harness analysis a starting point rather than a verdict. A third comment reports that provider middleware matters: benchmarking a bespoke harness against Kilo Code, DeepSeek-V4-Flash-0731 at high reasoning showed little difference on deepinfra.com but a substantial difference on together.ai, where the model had trouble making successful edits with Kilo Code's tools and worked around them by running scripts.
Small models need minimal harness like pi or they get overwhelmed by long system message and tool descriptions
Spec files are a harness choice too
OpenSpec, an MIT-licensed spec framework at v1.13.0, installs with npm install -g @fission-ai/openspec@latest and runs a five-step flow: /opsx: explore to map the problem and codebase, propose to draft proposal.md, specs/, design.md and tasks.md, apply to implement the tasks, verify to check the implementation matches the spec, and archive. Its compatibility list names Claude Code, Codex, Cursor, GitHub Copilot, Gemini CLI, OpenCode and Amazon Q Developer among a directory that ends with "+ 33 more", which is the point: it is a layer above whichever harness is already running. The project reports 68.0k GitHub stars, use by more than 265,000 developers a month and a new spec created every two seconds; those are its own adoption figures, not independent measurements.
For the fixed-model question this is the same lever as prompt weight, pointed the other way. A spec, a task list and a verify step change what enters the context and what the implementation is checked against, without touching the model. No source here measures whether that raises or lowers a benchmark score, so it is an untested variable that a harness comparison ought to hold constant.
What the sources do not answer
The central number - how much of a coding agent's measured success is attributable to the harness - is in no retrievable source. The jev-ultrafast result is a genuine harness-only comparison with a 25% median time reduction and a fall from 1,092 to 101 median browser protocol calls, but it is one task, three repeats per version, one browser profile, and a browser agent rather than a coding agent. The README also notes that a DONE choice still requires independent outcome verification, which is the property a scoring harness has to supply for itself.
Three questions from the thread stay open because the sources are silent on them. Whether the effect extends to open models such as GLM 5.3, and whether switching harness to Pi halves cost, were asked and not answered; a reply offers only that using Pi with local models "feels a lot snappier" than OpenCode or Claude Code, which is an impression rather than a measurement. The security argument is unresolved in both directions. Until the HarnessTax page publishes its method and per-model results, the defensible position is the narrow one: harness changes can move latency and protocol-call counts by large factors on a fixed model, and nothing in this material shows what they do to a resolve rate.
Questions this raises
what does the HarnessTax benchmark measure
HarnessTax asks how much of a coding agent's score comes from the harness rather than the model, but only its title, publisher and 16 September 2026 publication date are currently retrievable. There are no published scores, model roster, cost-per-task figures or definition of what counts as a harness. Claims circulating on Hacker News about what it measures are not confirmed by the published page.
how much faster is a lean agent harness with the model held fixed
The only harness-only ablation in this material is jev-ultrafast, a browser agent rather than a coding agent. With identical models and settings across six alternating runs, median task time fell from 9.450 s to 7.092 s, a 25% reduction, and median browser protocol calls fell from 1,092 to 101. Both versions passed 3/3, but the README stresses this is three repeats of one task on one browser profile.
does cutting harness tokens actually cut agent cost
Token weight is a weak proxy for spend. An earlier run at the same question found a tooling change that cut tokens by 89% moved the bill by only about 5% in either direction. That gap is worth checking against actual spend before a harness change is justified on price.
These daily notes are drafted by a model I run and operate myself - the same kind of pipeline this site is about - from sources published in the previous 24 hours, and every one lists what it read. The longer essays, the talks and the preprint are mine, written by hand.