LLM Serving

Qwen3.8-27B-FP8 lands with XML tool calls and xhigh reasoning on by default

The Qwen3.8-27B-FP8 weights are on Hugging Face, and the card's chat template is the part that decides whether your agent harness works.

Published
August 14, 2026
Read
5 min
Published by
Samir Sengupta
Qwen3.8-27B-FP8 lands with XML tool calls and xhigh reasoning on by default
Note 011 / 147Daily note · Written from 3 sources

How this was madeDrafted with Claude, Anthropic’s model, from 3 cited sources, then checked against them in a second editing pass. Editorial policy

the short version

  • A 27B dense model in FP8 is roughly 27GB of weights by arithmetic alone, which fits a single 48GB card with room left over.
  • The template emits tool calls as nested XML (<tool_call><function=...><parameter=...>), not JSON, so any harness that parses OpenAI-style function-call JSON needs a new parser.
  • Reasoning is on unless you pass enable_thinking=false, and the default effort is xhigh, which is injected as text into the system message.
  • Nothing covered here supports a dense-versus-MoE throughput or quality comparison; the GLM-5.3 announcement is cited by title only, and Netlify reports cost in platform credits, not tokens per second.

Qwen/Qwen3.8-27B-FP8 went up on Hugging Face on 14 August 2026, under the headline "Qwen 3.8 27B is out: open weights, best local dense model yet". For anyone planning to serve this locally behind a coding agent, the card's Jinja chat template is arguably the most consequential part, because it is the integration contract your harness has to satisfy on day one.

The quickest sizing number is the one in the repo name. Twenty-seven billion parameters at FP8 is approximately one byte per parameter, so roughly 27GB of weights. That is arithmetic from the name and the quantization format, not a figure the card reports. Weights alone leave headroom on a 48GB card and a lot of headroom on an 80GB one, but weights are not the working set: KV cache, activations, CUDA graph buffers and framework overhead all sit on top, and computing them needs the context length, head configuration and attention layout, not just the parameter count.

So the honest position on the throughput question is that it is unanswered. There is no tokens-per-second figure here, no batch-size curve, no comparison against a sparse mixture-of-experts alternative at equal memory. What there is, in detail, is how the model expects to be prompted.

The template is the integration contract

Reasoning is on by default, at xhigh

If enable_thinking is undefined or true, the template resolves reasoning_effort with a default of xhigh. Only three values are accepted - xhigh, medium and low - and anything else raises an exception rather than degrading quietly. The effort setting is not a sampling parameter or an API field; it is rendered as English text and prepended to the system message. At xhigh the injected string tells the model to "think carefully through the task, validate key assumptions, consider plausible alternatives"; at low it tells the model to keep thinking brief and move directly to the conclusion. Medium injects nothing at all.

The operational consequence is that the expensive setting is the default. Anyone wiring this into an agent loop that fires dozens of small tool-mediated turns should decide deliberately whether every one of those turns needs xhigh, because the template will not decide for them. Netlify made the same call in the other direction on hosted models, noting that it runs GPT 5.6 Sol on low effort by default "giving you a more economical alternative to Opus that still provides pretty darn good results", with the effort setting now exposed to the user.

Tool calls are XML, not JSON

Tools are serialized as JSON into a <tools> block inside the system message, but the model is instructed to reply in a nested XML form, and the instruction block is emphatic that an inner function element must sit inside tool_call tags, that required parameters must be specified, and that optional natural-language reasoning may appear before the call but never after.

text
<tool_call>
<function=example_function_name>
<parameter=example_parameter_1>
value_1
</parameter>
<parameter=example_parameter_2>
This is the value for the second parameter
that can span
multiple lines
</parameter>
</function>
</tool_call>

Parameter values are free-form text spanning multiple lines, which is friendly to file diffs and shell commands and hostile to anything expecting strict JSON schema validation on the way out. If your harness assumes OpenAI-style function-call objects, you are writing a parser. The upside is that multi-line code payloads do not have to survive JSON string escaping, which is a recurring source of corruption in coding agents.

Two failure modes to guard

  • The template walks messages in reverse to find the last real user query, and explicitly skips user messages whose content starts with <tool_response> and ends with </tool_response>. If every user-role message in your history is a tool result, it raises "No user query found in messages" and your request dies at render time.
  • System messages must be first and cannot contain videos; both conditions raise rather than warn. Multimodal content is handled through vision_start, image_pad and video_pad markers, with optional numbering when add_vision_id is set.
  • preserve_thinking controls whether reasoning content from earlier assistant turns is replayed. Leave it at the default and every prior <think> block stays in context; set it false and turns at or before the last user query are rendered without reasoning. On long agent trajectories at xhigh, that flag is a direct lever on prompt length and therefore on prefill cost.

The MoE comparison these sources do not support

The obvious next question is whether a 27B dense model at FP8 beats a sparse MoE of similar or larger total size on a single card, where the MoE wins on active parameters per token but loses on resident memory. Nothing covered here answers it. The GLM-5.3 announcement from z.ai, published the same day under the title "Frontier coding with emergent cyber capabilities", is cited by title only; its architecture, size and results are not covered.

Netlify's comparison of eleven models, published 13 August, is the closest thing to a competitive read, and it is measuring something different. Netlify tested via OpenRouter through its AI Gateway and inside Agent Runners, using the open-source OpenCode agent to drive open models including Kimi K3, GLM 5.2 and DeepSeek V4. Scoring came from AXIS, its internally built and now open-sourced evaluation tool, with checks focused on functional correctness - does the generated site use a database when the requirement calls for one, and avoid one when a static page will do. Cost is reported in Netlify credits per run across three runs per model. That is a useful signal about task success in a hosted agent, and no signal at all about what a 27B dense model does on your own GPU.

What to watch

Three gaps decide whether this model is worth a slot on a local box. Context length and attention configuration, because they set how much of your 48GB or 80GB is left for KV cache after the roughly 27GB of weights. Serving-stack support for the FP8 checkpoint, since FP8 weight loading and FP8 compute are not the same thing. And any published coding or agentic benchmark result, because the repo headline claims "best local dense model yet".

Until those land, the defensible engineering move is narrow: pull the weights, implement the XML tool-call parser, decide the reasoning_effort default for your loop rather than inheriting xhigh, and measure prefill and decode on your own hardware at your own context length. Everything else on offer today is a template.

Spotted an error? Email samir.s.sengupta@gmail.com and I will correct it and date the change at the top of the note. How these notes are researched, drafted and checked is in the editorial policy.

More notes

Building something on this?

I ship production LLM, RAG and agentic systems for a living - the infrastructure behind the things these notes are about. Open to roles, contract work and research collaboration.