GPT-6 Astra code review cost evaluation: 4% more bugs at 2.5x Sol's price

The metric is actionable bug coverage: how many labeled bugs a model catches through findings a developer can act on. CodeRabbit's own explanation for the small overall number is that the full evaluation includes simpler reviews, where there is less room for a stronger model to differentiate itself. That is also the shape of most review queues, which is why the routing decision matters more than the headline.

CodeRabbit calls this an early, directional result. The findings do not establish an overall ranking of review quality, predict a team's defect rate, or promise the same gain on every pull request. Anyone porting the decision into their own review or agent harness should treat it as a hypothesis to re-test locally.

Coverage is rounded to one decimal and relative gains are computed on unrounded values. Two comparisons were published: an overall set and a harder cross-file subset. CodeRabbit states that models should be compared within each chart because review difficulty differs between them, so the 4% and the 20% are not two views of the same population.

The cross-file result carries the engineering claim. CodeRabbit's interpretation is that Astra's advance lies in connecting the right information across a codebase, identifying which pieces of a large context matter and reaching a conclusion supported by evidence. The post is explicit that the result does not isolate the cause of that progress or prove that more context alone improves performance.

CodeRabbit modeled an illustrative task at 100,000 uncached input tokens and 10,000 billable output tokens including reasoning tokens, using publicly listed standard API prices checked September 4, 2026 and OpenAI short-context rates. The example excludes caching, cache writes, tools, retries, regional uplifts and service-tier adjustments.

At that fixed usage Astra costs 2.5 times Sol, about 4.7 times Terra, and about 47 times Luna. Fable 5.1 shares Astra's base input and output rates, though caching prices differ. CodeRabbit notes that Sol's public promotional pricing is available at least through November 21, 2026; the source does not say what the rate becomes after that date, so the 2.5x multiple is anchored to a price that is explicitly promotional.

Fixed-usage arithmetic is not cost per completed task, and CodeRabbit says so directly: a model that needs fewer tokens or fewer attempts could narrow the gap. It adds that OpenAI reports lower estimated task costs for Astra in some of its own evaluations despite higher token prices. Artificial Analysis reports the same tension from the other side, finding Astra more token efficient than almost every other model near the intelligence frontier and using fewer tokens than GPT-5.6 Sol for similar Intelligence Index performance, but concluding that the efficiency is outweighed by higher prices.

OpenRouter's provider table lists five endpoints for GPT-6 Astra, although its FAQ describes the model as served by two providers, OpenAI and Azure (US). OpenAI's own endpoint posts $10/$50 with $1.00 per million cache read, 3.10 s P50 latency, 25 tps and 99.59% uptime. OpenAI Flex is half price at $5/$25 with $0.50 cache read, 4.00 s latency and 48 tps. OpenAI Fast doubles the rate to $20/$100 at 3.70 s and 41 tps; Azure sits at $10/$50 with 8.51 s and 23 tps, and Azure (US) at $11/$55 with 7.63 s and 29 tps.

For a review harness that fans out many calls per pull request, endpoint choice moves the posted rate by 4x and P50 round-trip latency by about 2.7x on the same model. OpenRouter defines time-to-first-token on the page but does not list a TTFT figure for Astra, so the latency column is total round-trip time. Over the past three days it reports 100.00% uptime for the best provider and 99.11% availability; over the past 24 hours, availability is 99.14% with routing against 96.26% without, the gap being what automatic failover recovers.

The model has a 1,050,000 token context window with up to 128,000 completion tokens, accepts PDFs, images and text, returns text, and supports tool calling and structured outputs via a JSON schema in response_format. Cache write is $12.50 per million and web search $10.00 per 1,000 calls. The top apps by token volume are Codex at 12.2B tokens, Hermes Agent at 8.04B, Cursor at 1.85B, omp at 1.81B and Claude Code at 1.63B, which is a reasonable proxy for the agent harnesses this model is already wired into.

Artificial Analysis published Intelligence Index v4.2 on September 4, 2026, adding AA-Briefcase and Surge AI's GDP.pdf and removing GPQA Diamond as saturated. Claude Fable 5.1 leads the Index, with GPT-6 Astra second and showing a 4pt gain over GPT-5.6 Sol. Held-out private test sets now carry 40% of the Index weighting, double the figure from v4.1.

On AA-Briefcase, which grades multi-week agentic knowledge work projects with many linked tasks and thousands of input source files using rubric and pairwise grading, Fable 5.1 and Opus 5 lead, followed by Astra and Muse Spark 1.3. Astra shows a gain of roughly 85 Elo points over Sol. On GDP.pdf, which synthesizes evidence across 4,592 PDF pages against 1,275 expert-authored atomic criteria and credits a task only when every criterion is satisfied, OpenAI leads with Astra at 33.2%, Sol at 28.2% and Fable 5.1 at 26.2%.

Separately, Artificial Analysis reports that Astra makes significant gains in the Coding Agent Index, scoring equal to Fable 5 at lower cost. That comparison is against Fable 5, not Fable 5.1. Anthropic, OpenAI, Meta and Z.AI share the updated Cost per Task Pareto frontier.

Commenters on Hacker News read the release as a pattern: both OpenAI and Anthropic shipping a model that is slightly better at roughly double the previous price. One commenter argued that Astra is costlier for the same intelligence tasks as Sol while Fable is not in the same position. Another noted that labs do not publish model sizes, so the scaling-law explanation cannot be checked from outside.

The evaluation's setting drew the sharpest criticism. One commenter, describing themselves as a software engineer and former security researcher who did manual code audit, said scoping the comparison to CodeRabbit's own review tool makes it practically irrelevant to them and characterized the tool's output as noise and friction in CI review. Another argued that a dedicated review layer bolted on top of a model is the wrong shape, and that review should consume test results, the original task and the open spec. The sources do not settle this: CodeRabbit reports coverage against labeled bugs and says nothing about false-positive rate or comment volume.

Latency reports conflict. One commenter found Astra definitely slower than 5.6 Sol on the same tasks and speculated that it reads more context. Another found Astra faster than Sol with both on medium reasoning for simple agentic coding, while adding that comparison is hard because Sol may miss something on open-ended tasks and yield a cheaper but worse outcome. The 3.10 s to 8.51 s P50 spread across OpenRouter providers suggests some of that disagreement is endpoint rather than model.

One question the thread raised and the sources do not answer is whether Astra should be benchmarked against Mythos rather than Opus 5, on the argument that Fable is a Sol-class model. CodeRabbit's comparison set is Sol and Opus 5; neither Fable 5.1 nor Mythos appears in its coverage charts.

CodeRabbit states that neither it nor its model providers train AI models on customers' proprietary code or personal information collected during private code reviews, and that any model used for customer reviews must meet its data-protection requirements. The two providers reach that bar by different routes.

For a review harness that touches customer repositories, that asymmetry is a procurement item before it is a benchmark item. Astra's ZDR path is available to eligible API customers directly; the Fable equivalent for a business-to-business product runs through negotiated terms and a programme still being introduced.

CodeRabbit's post reports relative gains only, with coverage rounded to one decimal; absolute coverage figures appear in charts rather than in the text. There is no false-positive rate, no per-review latency and no observed cost per completed review - the $1.50 figure is a fixed-usage calculation, not a bill. OpenAI's own announcement, titled "GPT-6 Astra: A new generation of intelligence", was not retrievable beyond its title, so nothing in it is asserted here.

CodeRabbit also used Astra to build NIGHTSHIFT, an action RPG made with Godot and GDScript with seven character classes, a 988-node passive skill tree, and 40 zones across 10 acts, plus native PS5 and Xbox controller support, native macOS, web and Linux builds, and co-op requiring a new Xcode project, App Store Connect account, certificates, entitlements and notarization. That is an anecdote about long-horizon agentic work, not a measurement, and CodeRabbit does not claim otherwise.

The test CodeRabbit recommends is the one worth running: put Astra alongside your current model on the same tasks and compare answer quality, verification time and total cost. Two dates bound the arithmetic. Sol's promotional pricing runs at least through November 21, 2026, and Artificial Analysis says Index v5 is in progress with more incremental releases planned and the held-out share rising above 40%.