topic · 24 notes
Open-Weight Models, as it ships.
When a lab publishes weights, the questions an engineer has are concrete: how big is it, how many parameters are active, what licence does it ship under, what do its own benchmarks claim and what does it take to serve. These notes answer them release by release, from the model card and the paper.
LittleBit compression stores LLM weights at 0.1 bits via binarized latent factors
LittleBit compression from Samsung Labs targets 1.0 to 0.1 bits per weight via binarized low-rank factors. Code ships QAT and eval, no serving path.
Strands Decider 2B makes local decisions in a median of about 115 ms on an RTX 3090
Strands Decider 2B is an open 2B decision model: 115 ms median on an RTX 3090, 3rd of 33 on JevBench, but no named licence or p99 latency in the release post.
GLM 5.3 Flash after a month on Wagtail: 1B of 2B tokens, $68, 4 kWh
GLM 5.3 Flash took 1B of Wagtail's 2B September tokens for $68 and 4 kWh. A wrong-model run, provider limits and R&D, not the model, took the rest.
Kolibri model: 78B MoE, 3.46B active, Apache 2.0 weights for German and English
Kolibri model from Aleph Alpha: 78.1B-total, 3.46B-active German-English MoE, Apache 2.0, 78 GB in FP8. Beats Qwen3.6-35B-A3B on AIME, lags on long context.
Clef decision models ship Apache 2.0 weights, a 38.8 ms flash tier and RL fine-tuning
Clef decision models from Cloudflare ship Apache 2.0 weights on Qwen backbones, a 38.8 ms Clef-flash tier vs Jev's 524.1 ms, and FDE-run RL fine-tuning.
Fireworks' Ember-1: Kimi K3 with 35-50% shorter reasoning, no weights or price
Ember-1, Fireworks' post-trained Kimi K3, reports K3-level scores with about 40% fewer tokens. Two-week serverless preview, no weights, licence or price.
What Heretic abliteration changes inside open weight models
Heretic edits open weight model weights to remove refusals and reports two numbers: refusal count and KL divergence. HN commenters dispute what they prove.
M5 Ultra Mac Studio local LLM benchmarks: prefill, bandwidth and the 512 GB ceiling
M5 Ultra Mac Studio prompt processing is up 150% over M3 Ultra, generation about 70%, bandwidth 819 GB/s to 1.2 TB/s, memory ceiling still 512 GB.
The Qwen-Image-2.1 license is clear, the weights and VRAM numbers are not
Qwen-Image-2.1 shipped 20 September 2026. Its LICENSE bars commercial use without a separate Qwen license; the 7B figure is a commenter claim, VRAM unknown.
What the CUA-S1 system one computer use model ships, and what it does not
trycua's CUA-S1 is an MIT-licensed, source-only research release; CUA-S1-FORMS weights sit on Hugging Face and the README has no latency or accuracy numbers.
The ZCode GLM agent uploads git history, and only Z.ai can decrypt it
ZCode packs a 345MB workspace into a 313MB encrypted archive and POSTs it to Aliyun OSS at login. The RSA unwrap key lives only in Z.ai's cloud.
DeepSeek v4.1 Flash pricing, benchmarks, and 890 bytes per token
DeepSeek V4.1-Flash is a 552B MoE with 8B active params at prefill, 16B at decode, 890 bytes per token KV cache, and off-peak rates at 50% of peak.
Deltafin Kimi K3 SSD weight streaming on a MacBook: 1 tok/s, 376s first token
A deltafin fork streams Kimi K3's 1.45 TB expert bank off four SSDs on an M5 Max MacBook Pro at 1.00 tok/s, with 376s to first token on 512 tokens.
Run 104GB Qwen3.8-Flash-Next on a 48GB Mac via slotstream at ~12 tok/s
slotstream keeps a 3.8GB dense trunk resident and streams 68GB of experts from SSD into a 20.1GB slot pool for ~12 tok/s decode on a 48GB M5 Pro.
vLLM v0.28.0 doubles the default batched-token budget and drops bitsandbytes in-tree
vLLM v0.28.0 raises max_num_batched_tokens from 8192 to 16384, enables prefix caching for Mamba, and drops in-tree bitsandbytes and calculate_kv_scales.
GLM-5.3 ships open weights with no published parameter count
Z.ai released GLM-5.3 on Hugging Face on 28 August 2026 with no parameter count, context length, or license name stated, plus a reported move off MIT.
What 125B-A6B actually tells you about Qwen3.8-Flash-Next serving costs
Qwen3.8-Flash-Next launched 26 Aug 2026 with one disclosed figure: 125B a6B. No price, context window, benchmarks or throughput. What to measure first.
Qwen 3.8 27B Reverse-Engineered a License Check in 30 Minutes, Locally
Qwen 3.8 27B ran on one Nvidia GB10 workstation, statically reverse-engineered a paid app's license scheme and built a bypass in about 30 minutes.
Mojo's compiler goes Apache 2.0 while MAX stays source-available
Mojo 1.0's compiler and toolchain are now Apache 2.0 with LLVM exceptions. MAX is source-available, but kernel work still needs a prebuilt compiler.
Qwen3.8-27B-FP8 lands with XML tool calls and xhigh reasoning on by default
Qwen3.8-27B-FP8 implies roughly 27GB of weights. Its chat template mandates XML tool calls, xhigh reasoning by default, and raises on tool-only turns.
DeepSeek's agent harness is public, but the sampling config isn't in the README
DeepSeek open-sourced dsh (MIT, 12,293 commits) the same day V4 Pro 0813 hit OpenRouter at $0.435/$0.87 per 1M. Here is what the release specifies.
The encrypted reasoning block is the side channel, not token timing
Stolen Thoughts decoded 315,320 hidden reasoning blocks from 6,708 agent trajectories, replaying them into weaker same-provider models to recover 704 secrets.
Meta's 30B Muse Glimmer fits a local coding agent under 20GB, Apache 2.0
Meta's Muse Glimmer is a 30B open-weights agentic model under Apache 2.0, quantized to ~4-bit to fit under 20GB with room for KV cache and a drafter.
DeepMind open sources WeatherNext Cyclones, a 1,000-member ensemble at 28km
WeatherNext Cyclones runs 1,000-member ensembles at 28x28km and produces a 15-day forecast in under a minute on a TPU. It is now open source.
Other topics
Hiring for AI or ML?
I am open to AI/ML Engineering, Data Science, and Python roles, plus research collaborations and consulting. New York based, shipping worldwide.