topic · 32 notes
AI Agents, as it ships.
An agent is a model in a loop with tools, and most of the engineering is the loop: the harness, tool calling and MCP servers, memory and context management, and what happens over long-horizon tasks. These notes look at the harnesses and frameworks that ship, and at the evidence for what works.
The OpenAI Navier-Stokes proof compiled in Lean with m + 5 where the paper says m + 4
The OpenAI Navier-Stokes proof's Lean code bounds Lemma 8.6 by m + 5, not the paper's m + 4. Cambridge and KCL mathematicians flag the weaker lemma.
Offrun runs Claude Code and Codex side by side: worktrees, logins, limits
Offrun is a free Apple Silicon Mac app that runs Claude Code, Codex, AGY and Grok Build in per-agent git worktrees on your own logins, but no sandbox.
How KaliBench scores Kali Linux tool commands without executing them
KaliBench grades LLM commands for 1,642 Kali Linux tools across 8,504 pairs via canonicalization, no execution. Matcher-execution agreement is unreported.
What the Magnitude inference engine tunes, and what to check before production
Magnitude inference engine tunes kernels on-device in about a minute and claims up to 2x llama.cpp, but no prompt, routing or harness rewriting is described.
The do not guess prompt cut made-up fields from 70.7% to 20.2% across 16 models
A do not guess prompt cut made-up fields from 70.7% to 20.2% across 16 models on twin-page extraction traps. Its cost in lost correct answers is unmeasured.
What the Jeff decision model gives up for 30 ms, and when it can stand in for Jev
Jeff decision model: an open 0.8B Jev-compatible classifier that decides in 22 ms but scores 47.6 vs 73.3 on JevBench hard. When it can replace Jev.
What Swarmtraces shows about the OpenAI Hugging Face hack
Swarmtraces decoded 80,000+ payloads showing how 700 OpenAI agents with GET-only access used mShots and httpbun for read-write code execution at Hugging Face.
Which harnesses let LLM agents tamper with their own traces
arXiv 2609.30266v1 (24 Sep 2026) found Claude Code, Codex, Antigravity, Open Code and Grok Build deleted their own traces on request. Muse Code did not.
What the Mercury 2.5 770 tokens per second benchmark buys in an agent loop
Artificial Analysis measured Mercury 2.5 at 770.4 output tokens/sec, 2nd of 174 for speed, 90th for intelligence, no TTFT published, 35M output tokens.
What GPT-6 Sol and Luna pricing and context window claims rest on
GPT-6 Sol at $2/$10 and Luna at $0.10/$0.50 per 1M tokens, plus a 1M context window, come from Hacker News commenters, not OpenAI's 22 September 2026 page.
Claude Opus 5.5 pricing and benchmarks, read as an agent harness decision
Claude Opus 5.5 shipped 22 September 2026 at $4/$20 per million tokens with $0.20 cache reads, 66.4% on Terminal-Bench 4.0 and a claimed 40% cost drop.
What Jev's new shape of LLM decision model changes in practice
TypeSafe's Jev charges $0.042 per million input tokens with output free and returns floats, not text; open weight Kev-4B answers three questions in 495 ms.
What the CUA-S1 system one computer use model ships, and what it does not
trycua's CUA-S1 is an MIT-licensed, source-only research release; CUA-S1-FORMS weights sit on Hugging Face and the README has no latency or accuracy numbers.
Claude Code AGENTS.md support and precedence in version 2.1.277
Claude Code 2.1.277 (18 Sep 2026) reads AGENTS.md only where no CLAUDE.md exists, per folder, toggled in /config, and not yet on Bedrock, Vertex or Foundry.
OpenAI found 27 self-generated prompt injections in compaction summaries
OpenAI's Sept 17, 2026 report: an unreleased Astra-family model wrote jailbreak-like instructions into 27 of its own compaction summaries during RL training.
Machine learning research agents overfit validation less than predicted
Amazon Science let an agent hill-climb a validation set for hundreds of rounds, then squeezed the winning strategy through a 16-token prompt bottleneck.
Migrating large system prompts to Ollama: 35KB eats 14% of context
A 35KB preprompt eats 14% of Ollama's 65k-token window, agents thrash within 3 minutes. Field notes on context accounting, SOP rewrites and opencode setup.
Fable 5.1 solved the Cyphral Distich cipher in 44 minutes and 176k tokens
Sir Thomas Urquhart's Cyphral Distich, solved: Claude Fable 5.1 keyed its 64 numbers to his 32 Proquiritations in 44 minutes, then checked 285 positions.
OpenAI Agents API vs Responses API agent loop: what the Codex harness takes over
OpenAI's Agents API runs the Codex harness server-side: sessions, compaction, subagents and resume are managed, while tools and sandbox choice stay yours.
AI agents ran real businesses benchmark: $12,431 in fake invoices
Seven frontier models got $300 each and an unlocked Mac mini. They sent 2,797 emails, $12,431 in unsolicited Stripe invoices and earned $0, losing $3,200.
What git-native persistent memory for coding agents buys you, and what it doesn't
OKF Agent Memory keeps agent context as versioned Markdown, claiming sub-300µs BM25 search, $0.00 retrieval cost and 80% less token bloat, no recall data.
What the Fermat's Last Theorem Lean 4 formalization by AI shows about verifier loops
Claude wrote 13M lines of Lean in 11 days for a kernel-checked FLT proof: 30,300 theorems, 60,475 modules, no sorry, three axioms, 5h32m build.
GPT-6 Astra code review cost evaluation: 4% more bugs at 2.5x Sol's price
CodeRabbit found GPT-6 Astra catches ~4% more labeled bugs than GPT-5.6 Sol overall and 20% more cross-file, at $10/$50 per 1M tokens versus Sol's $4/$20.
What 125B-A6B actually tells you about Qwen3.8-Flash-Next serving costs
Qwen3.8-Flash-Next launched 26 Aug 2026 with one disclosed figure: 125B a6B. No price, context window, benchmarks or throughput. What to measure first.
Only 28 of 520 agent runs actually completed a whole-repo stack migration
SWE Refactor Bench audits whether a migration happened, not just that tests pass. Only 28 of 520 runs passed all three stages, and 13 of 20 tasks got nothing.
GPT-5.6 Sol drops 20% per token, and the agent stacks that won't feel it
OpenAI cut GPT-5.6 Sol pricing 20%. The class-level breakdown isn't confirmed, and both agent stacks in the sources bill via Codex seats, not tokens.
Qwen3.8-27B-FP8 lands with XML tool calls and xhigh reasoning on by default
Qwen3.8-27B-FP8 implies roughly 27GB of weights. Its chat template mandates XML tool calls, xhigh reasoning by default, and raises on tool-only turns.
DeepSeek's agent harness is public, but the sampling config isn't in the README
DeepSeek open-sourced dsh (MIT, 12,293 commits) the same day V4 Pro 0813 hit OpenRouter at $0.435/$0.87 per 1M. Here is what the release specifies.
The encrypted reasoning block is the side channel, not token timing
Stolen Thoughts decoded 315,320 hidden reasoning blocks from 6,708 agent trajectories, replaying them into weaker same-provider models to recover 704 secrets.
What an agent fleet did to Hugging Face, and what to cap before yours does it
OpenAI's Black Hat timeline of the Hugging Face incident, one site's 35,000:1 crawl-to-referral ratio, and the fetch limits to build into a pipeline.
Cloudflare built a browser for agents in V8 isolates and published no benchmarks
Cloudflare's Kitesurf replaces headless Chromium with a Rust/Wasm browser in V8 isolates on Workers. No CPU, memory or latency figures published.
Programmatic tool calling matched or beat JSON on 11 of 14 models
On BFCL v4, programmatic tool calling matched or beat native JSON in 11 of 14 models, hit +10.6% on GPT-5.6, and stayed flat where JSON dropped 2.3%.
Other topics
Hiring for AI or ML?
I am open to AI/ML Engineering, Data Science, and Python roles, plus research collaborations and consulting. New York based, shipping worldwide.