topic · 14 notes
Decision Models, as it ships.
Decision models answer the small, frequent questions inside an AI system - which route, which label, is this safe, is this step done - in milliseconds instead of a full LLM call. These notes compare the releases on accuracy, latency and calibration. Disclosure: I build Tron-1B, an open-weight decision model, so I have a stake in this category.
Neural Thickets gains reappear with a frozen model and a random logit bias
Neural Thickets gains reproduced: a frozen model with a random logit bias matches or beats weight perturbation on most reasoning tasks (arXiv 2610.40649).
DecisionRAG drops the first-stage retriever and beats dense retrieval plus an 8B reranker by 13.8 R@1
DecisionRAG drops the first-stage retriever and beats dense retrieval plus an 8B reranker by 13.8 R@1. Runs 0.7-1.8 s per query; no baseline latency or cost.
Inception v3 scores 62% to frontier LLMs' 42% on an LLM skin cancer benchmark
LLM skin cancer benchmark: Inception v3 got 62% of 200 MILK10k lesions right vs 42% for GPT-6 Astra and Claude Fable 5.1, but discharged 18/30 melanomas.
Cloudflare's Clairic extraction model returns typed fields in one pass on Workers AI
Clairic extraction model on Workers AI returns 9 typed fields with p_null and confidence in one pass, bills input only, but lists no price, licence or size.
What the OpenAI Decisions API beta exposes, and when a local decision model wins
OpenAI Decisions API beta returns typed answers 10x faster than Responses on gpt-6-luna, but OpenAI lists no price or latency. Decider 2B: 115 ms median.
Strands Decider 2B makes local decisions in a median of about 115 ms on an RTX 3090
Strands Decider 2B is an open 2B decision model: 115 ms median on an RTX 3090, 3rd of 33 on JevBench, but no named licence or p99 latency in the release post.
Clef decision models ship Apache 2.0 weights, a 38.8 ms flash tier and RL fine-tuning
Clef decision models from Cloudflare ship Apache 2.0 weights on Qwen backbones, a 38.8 ms Clef-flash tier vs Jev's 524.1 ms, and FDE-run RL fine-tuning.
What the Jeeves decision model's reasoning buys: 5 points for 11x the latency
The Jeeves decision model's reasoning lifts dev accuracy from 0.775 to 0.825, but median latency rises from 0.3 s to 3.3 s and p90 hits 17.1 s on one H100.
Jevstiller's disagreement bound: what distilling Jev into a local model guarantees
Jevstiller answers confident Jev calls locally in about 15 ms and bounds disagreement at 2% of requests at a 98% target, costing 4 to 8 points of coverage.
What the Jeff decision model gives up for 30 ms, and when it can stand in for Jev
Jeff decision model: an open 0.8B Jev-compatible classifier that decides in 22 ms but scores 47.6 vs 73.3 on JevBench hard. When it can replace Jev.
What Ollaya runs locally for Jev-style decision models, and what it needs
Ollaya serves open decision models on TypeSafe's /v1/systemone API. Laya answers five questions in 8 to 10 ms on an RTX 4090 but trails Jev on hard queries.
JevOut flips Jev on 61.4% of correct decisions with ordinary context
A 24 September 2026 arXiv paper (2609.30243v1) redirects Jev on 312 of 508 correct decisions using fluent context additions, with 229 flips above 0.7.
What Jev's new shape of LLM decision model changes in practice
TypeSafe's Jev charges $0.042 per million input tokens with output free and returns floats, not text; open weight Kev-4B answers three questions in 495 ms.
When LLM judges agree, evaluation reliability needs a correlation discount
Amazon's ICML 2026 dependence-aware aggregator models judge panels as an Ising network, hitting 0.912 on relevance vs 0.820 for weighted majority vote.
Other topics
Hiring for AI or ML?
I am open to AI/ML Engineering, Data Science, and Python roles, plus research collaborations and consulting. New York based, shipping worldwide.