What Jev's new shape of LLM decision model changes in practice

Willison covered the model on 21 September 2026. TypeSafe calls the category "System One models"; Willison sides with Maggie Appleton in preferring "decision models", which is the more useful framing for deciding where to put it. Arcturus Labs quotes Vercel saying Jev "was adopted faster than any other model in AI Gateway history". Within days there was an open weight recreation, jaredpalmer/kev, built on Qwen3.5 in 0.8B, 4B and 9B sizes, plus a JevBench benchmark comparing "Jev-class decision models".

The request shape matters as much as the output shape. You compose a "state" object containing a string, an array of strings, or a set of name-value pairs, then attach as many questions as you can cram into the context window; Willison's write-up does not state what that window is. Kev's README states the constraint on the questions plainly: they "share the input text but can't read each other", so there is no cross-question reasoning and no chaining inside one call. Willison frames the fit as anything expressible as a classification task, naming spam detection, suggesting labels, prioritization and ranking, and reports experimenting with search reranking by fetching 100 likely matches with BM25 and having Jev score those 100 candidates for relevance against the original query.

Arcturus Labs argues Jev is using something quite close to a conventional large language model, generating the probability distribution over all possible next tokens at a single step and massaging those numbers into the return format. For a noul question the model looks at just the true and false tokens, ignores everything else, and normalizes the two into one probability; for a choice question it reads the relative probabilities of the option tokens and takes the highest as the winner. The same author's 2025 post, Supercharging LLM Classifications with Logprobs, described the choice pattern before Jev existed, and he writes that he has not thought hard about the score primitive but suspects it is a variant of the same pattern.

That reconstruction is not confirmed by TypeSafe, which has not published Jev's architecture. Arcturus cites Latent Space reporting that many of the early clones are LLM-based. One commenter on Hacker News reduced the idea further, to an LLM with a chat template, a cached options prefix and generation constrained to a few special tokens indicating the possibilities; another commenter said TypeSafe has been adamant this is a new class of model, while conceding nobody outside the company knows enough about the architecture to say.

Kev makes the serving profile concrete because you can run it yourself with Python 3.12+ and uv. The repo starts a server on port 8009 that answers POST /v1/systemone, and its published example, Kev-4B running in bf16 on an Apple M5, returns a three-question support ticket classification with latency_ms 495, 101 input tokens and 161 output tokens. The response still reports output tokens even though the hosted product charges for input only. The 4B and 9B models fit a 32 GB Mac in bf16, the code runs on CUDA, ROCm and Apple Silicon, and setting KEV_DTYPE=fp32 reproduces the exact path the evaluations use.

Because the API matches TypeSafe's System One, the same Python SDK points at either endpoint, and the SDK ships with uv sync --extra serve. There is a web playground, run with Node 20.9+, whose presets include "Packed vs separate" for comparing all questions in one request against one at a time, "Permute" for running a Choice question through six option orders, and presets for question isolation and fake delimiter tokens; Kev-4B and Kev-0.8B can also be tried in the browser at huggingface.co/spaces/jaredpalmer/kev. Neither source reports a latency figure, a context window size or rate limits for hosted Jev, so the 495 ms number describes local Kev only. A commenter on Hacker News notes the hosted product is API-only and that you have to shape your calling software to the way it communicates.

Kev ships pretrained weights plus training code, evaluation data and a kev-finetune skill, so training your own is the documented path. Its results table separates held-out examples from the datasets used to train Kev ("trained sources") from datasets and policy rule types Kev was not trained on ("new sources"), reporting development and test numbers for each. That split is the useful one if you are deciding whether to fine-tune, because it measures what happens when your production data drifts away from your training set.

Kev-4B scores 0.872 / 0.871 on trained sources and 0.797 / 0.837 on new sources, with Brier 0.299 / 0.255. Kev-9B scores 0.872 / 0.874 and 0.822 / 0.852, with Brier 0.286 / 0.237. Kev-0.8B drops to 0.825 / 0.834 on trained sources and 0.652 / 0.684 on new sources, with Brier 0.499 / 0.460. Every cell is development / test, and lower Brier is better.

Hosted Jev is listed at 0.845 on trained sources and 0.857 on the new-source development set, with Brier 0.211; the test column for Jev is blank. Kev's README states that Kev-9B trails Jev by 3.5 points on the new-source development set, 0.822 against 0.857, and its Brier of 0.286 is worse than Jev's 0.211. All three Kev models use Qwen3.5 bases with the same training data and settings, evaluated on the same development sets, decision-v7 and transfer-v4, with the test sets read once per released checkpoint after model selection. The README recommends starting with Kev-4B, using Kev-9B when accuracy and calibration matter more than memory, and Kev-0.8B when you need the smallest model.

Arcturus Labs points out that OpenAI has been using large language models implicitly as specialized classifiers since at least the introduction of tool calling. In the 2024 post Tool Invocation - Demonstrating the Marvel of GPT's Flexibility, the author shows that right after <|im_start|>assistant the first token the model predicts is either a newline or to=function., which decides whether a tool is invoked at all; the next handful of tokens identify which tool; and <|im_end|> is itself a classifier reading true when the model believes the message is complete. His argument is that those micro-classifiers are specialists while Jev's are general, which is why he expects OpenAI to fold the capability into model selection, more efficient thinking and security guardrails rather than sell it as a separate product.

In agent code the practical difference is routing on a distribution rather than a label. Kev's published example returns department probabilities of returns 0.47, shipping 0.28 and billing 0.25 for a ticket that mentions a return, a late delivery and a billing problem, with a choice confidence of 0.21, and the README says that is the point of getting probabilities back instead of a single label. One commenter on Hacker News is pairing Deepgram with Jev to talk to an application and have it respond in real time, and says it is not yet clear whether what is missing is a new programming language or a harness over the capability.

Willison's objection is that Jev is a regression further towards black box machine learning. An LLM can be asked to justify its decisions, with no guarantee the justification is accurate; Jev returns a floating point number and nothing else, so if it marks something as spam you cannot see which content signals tipped it off. He writes that he really hopes nobody uses Jev to rank job applicants, and reports scoring every city in the San Francisco Bay Area on a yes/no "Good city?" question, which rated Cupertino top and East Palo Alto bottom. His mitigation is that evals and structured experiments matter more here than for regular LLM projects, and that Jev is cheap enough that hundreds or thousands of experimental prompts cost just a few cents.

The architecture is unpublished, so the logprob account is an informed reconstruction rather than a fact. Arcturus is explicit that if Jev is close to a conventional LLM there is no architectural moat, and that the biggest moat he sees is TypeSafe's training data and training processes, specifically the technique for turning raw data into training data. Outside that, no calibration detail for hosted Jev exists beyond the Brier of 0.211 in Kev's comparison table, and no source states Jev's licence terms, context window or rate limits.

Three things are worth watching concretely. Whether JevBench stabilizes into a shared score for Jev-class decision models, whether the 3.5-point new-source gap between Kev-9B and Jev closes, and how far option ordering moves answers. Kev already ships the third experiment as the "Permute" playground preset, which runs a Choice question through six option orders, and that is the cheapest check to run before any of this reaches a user.