Deltafin Kimi K3 SSD weight streaming on a MacBook: 1 tok/s, 376s first token

The benchmarks were measured on 2026-09-08 against the fork's configuration of record in k3-public-bench/env.sh; every number is one cold run with the exact prompt, and the per-run logs are committed under k3-public-bench/results/. With the drafter on: 1.0015 tok/s over 512 generated tokens, 1.1252 over 128, and 0.9631 median of three on the 17-token prompt from upstream issue #15, where upstream reported 0.684. With the drafter off: 0.9232 over 512 and 0.9261 over 128. Time to first token on a 512-token prompt is approximately 375 s with the drafter and 376 s without.

This is a fork of gavamedia/deltafin, which built the engine and whose tracked project code is MIT; the fork adds the ARGODRIVE storage work and the benchmark package. Upstream's own reference numbers come from an M1 Max laptop and stand at 0.2901 tok/s, or 3.447 s per token, up from 0.0141 tok/s on 27 July 2026. The project's rule is that speed never comes from reducing quality: all 16 routed experts on every token, with K3 as the sole authority and small draft models allowed only to propose.

Expert weights are untouched at their released MXFP4 precision, while the resident attention trunk is int8, which upstream labels non-weight-exact. In the Hacker News thread the author therefore makes no claim of bit-exactness against BF16, only token-identical output against his own reference on the prompt of record, checked on every promotion. The README contrasts this with other projects that re-encode K3's expert bank down to roughly 3 bits: those weights are no longer the ones Moonshot released, and nobody, including the people who produced them, has measured what those compromises cost.

Storage sizing follows from the same design. setup --full downloads the full 1.7 TB K3 model to disk; setup --stream installs the resident model at 215 GB to start and fetches exact experts on demand, running far more slowly until routes build up a local cache. The speculative components are small next to that: Inferact's Kimi-K3-DSpark takes 6.635 GiB on disk and about 4.49 GiB when admitted at runtime, and the optional Qwen 0.6B/1.7B proposal-only add-on is 4.337 GiB and helps raw completion only, where one measured 17-token completion ran 2.7x faster with the same output IDs.

The drive-count ladder in results/SCALING.md is the most transferable finding in the repository. One drive delivers about 52% of four-drive decode speed, two full mirrors about 73%, three about 90%. Going from one drive to four raises combined ceilings from 13.6 GB/s to roughly 33 GB/s, per the author's account in the thread, and takes decode from 52% to 100% rather than 4x, with every drive already running at 90 to 100% of its own ceiling. What moves the number is per-read latency and read scheduling, not bandwidth you can buy more of.

RAID-0 was one of the first things tried and it lost. Striping makes every read touch every drive, so the slowest drive sets every barrier rather than only the barrier for the expert it holds. The design constraint for anyone building a similar rig is that you are scheduling 16 concurrent 17.5 MB reads against a max-latency barrier, which makes tail latency per device the hardware question rather than sequential throughput per device.

The rig is three Thunderbolt 5 enclosures plus the internal SSD, with one Gen4 drive and three Gen5 drives inside the enclosures, per the author's comment. Drive specs are published in k3-public-bench alongside the placement manifests and per-run logs.

The README lists the honest limit in its TL;DR: a 512-token prompt takes about 6.3 minutes to its first token. The diagnosed cause is that prefill re-reads each layer's experts eight times, and the fix is planned but not built; results/PREFILL.md is the write-up of why prefill is slow and what fixes it. Until that lands, prompt length is a far more expensive variable than output length in this system.

Long conversations compound the problem. The README warns that they are far slower than short completions because prefill and cache grow with history, and that startup prints the actual usable context bound. The native server implements exact conversation-state reuse, draft-verified DSpark speedups and an exact-response memo, which reduce repeat cost but do not remove the price of a long first prompt.

One of the top comments in the Hacker News thread objects that a model this slow cannot be useful and costs a lot to perform that badly. Several commenters pushed back on the objection rather than the benchmark, arguing that getting the full model to run at all is the result. One commenter estimated the author's opening comment at about 1000 tokens and noted that at 1 tok/s the machine would take almost 17 minutes to generate it. Another measured a wall of text at 843 tokens using token-calculator.net, to give a feel for 512 tokens being roughly one question and one long answer.

One commenter proposed scaling by brute force: 40x the RAID-0 SSDs, or 40 MacBooks with four SSDs each, for 40 tok/s. The author's reply is the measured refusal above, that a layer costs the max over its 16 reads rather than the sum, so bandwidth does not multiply. A follow-up commenter added that PCIe lanes cap this regardless of RAID configuration. Three questions in the thread the sources do not answer: whether an x86 machine with a large GPU could use NVMe as virtual VRAM, how the same model performs with zero external drives on a single 4 TB internal SSD, and whether used Optane SSDs would help. A commenter asked for that zero-external-SSD column on the scaling graphs and it does not exist.

The author's stated use case is the crossover: scheduled, unattended jobs where nobody is waiting on the cursor. Specifically day, week and month end review, going through the numbers, flagging what does not reconcile, and drafting the report. Two conditions have to hold at once for that to make sense: the model has to be good enough to trust with the judgement, which the author argues means the full 2.8T K3 rather than a cut-down one, and the data has to never leave the machine.

If either condition fails the case collapses, because a smaller resident model wins on any interactive path and a hosted API wins wherever the data is allowed to leave. What the configuration buys is a frontier-scale model on what the README frames as a $15,000 home setup instead of the $2,000,000 infrastructure Kimi recommends, which is 16 nodes and roughly 4.8 TB of aggregate VRAM for the full 2.8T parameters and the 1M-token context window. The sources give no cost-per-token comparison against any hosted API, so the crossover here is argued in privacy and latency-tolerance terms rather than in dollars. One commenter suggested a narrower fit, classification, where you feed in a large input and ask for a single token of output; that is viable on decode at 1 tok/s, though it is exactly the shape that pays the prefill cost hardest.

Every published figure is one cold run with the exact prompt, so there is no variance and no percentiles, and only the 17-token benchmark is a median of three. There is no quality evaluation against the BF16 model, only token-identical output against the project's own reference on the prompt of record. The prefill fix is diagnosed but unbuilt, so the 376-second first token is the current ceiling on prompt length, and nobody has measured what a single fast internal SSD does relative to the four-drive rig. The server subset is strictly greedy, one generation at a time and text-only, with a default response ceiling of one million tokens and a 128 MiB request bound, which is worth knowing before pointing an OpenAI-compatible client at port 8000 and raising its timeouts.