Memdoor's team-solve: five Qwen3.8-27B roles score 9.8/10 on LiveCodeBench Hard, a tie with GVS5H

The workflow reimplements the method of GVS5H (Lee, 2026) as a Memdoor workflow: eight YAML files, no code, no model named. It uses the format of mario, the engine underneath. Engineers weighing a multi-role loop on a 27B model against a single frontier API call need three things from the sources: how the loop is coordinated, what it costs in time and tokens, and how the benchmark comparison was set up.

The DAG runs plan first, then ideate-a and ideate-b in parallel, then solve, test, review, repair and verify. Each agent starts from a fresh context and communicates only through files on disk. Every agent prompt says "fresh context, read the files, do not rely on chat". The README notes that the GVS5H paper credits this ledger discipline for the gain. Each step is one YAML file in steps/, and the requires: field defines the graph.

A script that runs the samples decides success, not a model. The loop sees only the public samples, while the reported score uses the hidden tests. The steps name no model, so whatever /model is pinned to answers every role, including a local model. To change a role, you edit its prompt in steps/.

The workflow is the authors' harness rebuilt on the same model, Qwen3.8-27B, with their prompts, loop and 128K reply cap. It scored 10, 10, 10, 10 and 9 across five passes. The authors' harness column comes from the regraded result files in their repository and reads 9, 10, 10, 9 and 10. Every column was graded on every hidden test with the authors' corrected grader.

The README states that on ten problems and five passes, 9.8 and 9.6 are the same number. The authors used 100 problems. The workflow's 50 runs are published in bench/results/parity-5pass/, each with every request and reply, the program, the grader's output and the cost.

On these ten problems, the authors' five-pass data put their harness at 96 and Fable 5 at 80, a 16-point gap. Across their full 100, patched_verdicts.json puts the harness at 92.4 and Fable 5 at 90.4, a 2-point gap. The gap on this subset is far wider than the gap across the full set. The contests date from spring 2025, and the repository says contamination cannot be ruled out for any column.

One commenter on Hacker News dismissed results of this kind as harnesses run on "useless saturated and contaminated benchmarks from years ago" that do not reflect agentic work. The repository acknowledges the contamination risk but does not test the workflow on newer or agentic tasks.

The Fable 5 column is one call per problem, from the GVS5H authors' files, and the README states it was not run in this harness. The comparison is five roles plus a repair loop against a single frontier call, not two models under the same scaffolding. One commenter on Hacker News noted that the same harnessing should also improve Fable's performance. Neither source reports Fable 5 inside the loop, or Fable 5's latency or cost per problem.

A run took a median of 40 minutes and $0.39, and the 50 runs cost $35.79 in total. The model ran on OpenRouter, not a local machine. The README does not report tokens per run. The per-run files hold every request and reply, but no token totals are published, so the loop cannot be compared token for token with a single Fable 5 call from what is here.

A commenter on Hacker News who says they created Memdoor added a wall-clock caveat. The loop can take more than an hour on the hardest problem at 100 tokens per second. They expect a stronger model to improve token usage and latency more than score. A separate commenter said they tried Cerebras, which caps output at 32K tokens, so the authors' 128K reasoning budget did not fit, though they called Cerebras' speed real.

The one lost problem came from the manager. It invented a wrong counterexample, and the team rewrote a correct program. On two other occasions a correct program was overwritten and then recovered. The README attributes both gaps to the authors' loop, which was kept as is for the parity check.

For anyone adapting the design, the review and repair steps can replace a passing solution with a failing one. The README describes no step that keeps the last program that passed the samples. NOTES.md covers the byte-for-byte parity check against the authors' code, the hosts, the changes it took and the first run's findings.

memdoor connect supports openrouter, anthropic, openai, gemini or chatgpt. With no key, memdoor connect local runs against Ollama, llama.cpp, vLLM or LM Studio. Ollama needs a 32K window, not its 4K default. The program is solution.py, which reads from stdin and is compared to the samples with diff, so it needs Python 3 and bash. For your own problem, replace task.md and put paired tests/N.in and tests/N.out beside it.

Commenters on Hacker News disagreed over whether the result carries to consumer hardware. One suggested that Bonsai 2 quantizations could run Qwen 27B at 50 tokens per second on a reasonably priced gaming machine, citing a Steam survey figure of 16GB VRAM and 32GB system RAM. Replies pushed back. One said 2-bit Bonsai is "notably dumber" than Qwen 27B at four or eight bits, and another said even 4-bit Qwen3.8 27B quants are iffy for coding until Q5 or Q6.

The published runs used OpenRouter, so the sources contain no quantized or local results. Other commenters proposed Qwen3.8-Flash-Next as a stronger and faster base model. The commenter who says they created Memdoor said they would try it. No such run is reported yet.