# Next-concept modeling is 10x to 31x faster than diffusion LMs, with no autoregressive speed figure

> NCM generates text level by level from a residual VQ-VAE code hierarchy. It beats the strongest diffusion LM baseline on speed, but the abstract compares it with next-token decoding only on Sudoku.

- Published: 2026-11-09
- By: Samir Sengupta (https://www.samcodeman.com/about)
- Canonical: https://www.samcodeman.com/writing/next-concept-modeling-10-31x-faster-than-diffusion-lms
- How this was made: Drafted with Claude, Anthropic's model from 1 cited source, then checked against it in a second editing pass. Editorial policy: https://www.samcodeman.com/editorial-policy

## The short version

- NCM's reported 10x to 31x speedups are measured against the strongest diffusion language model baseline, not against standard autoregressive next-token decoding.
- On LM1B, at matched parameter scales, NCM reports better generative perplexity than the strongest diffusion baseline while running 21x faster. It is competitive on OpenWebText at 10x and matches ROUGE-L on XSum at 31x.
- The abstract compares NCM with autoregressive models only on Sudoku, where it outperforms them at every difficulty level using fewer function evaluations.
- NCM requires a trained multiscale residual VQ-VAE and a transformer that generates its codes. The abstract describes no conversion path from existing autoregressive checkpoints.

Next-Concept Modeling (NCM) is an arXiv paper from Hermann Kumbong, Christopher Ré and seven co-authors that generates text coarse-to-fine. A multiscale residual VQ-VAE encodes text into a hierarchy of discrete codes. A transformer then generates one level at a time, predicting every code within a level in parallel. The reported speedups are 21x on LM1B, 10x on OpenWebText and 31x on XSum, all measured against the strongest diffusion language model baseline. Against standard next-token decoding, the abstract reports no speed figure and no text-quality figure; its only autoregressive comparison is on Sudoku.

The paper was submitted on 9 October 2026 under cs.CL and builds on diffusion language models (DLMs). The abstract opens by saying DLMs offer parallel sampling for faster generation than autoregressive models, which matters more as inference becomes a dominant computational workload. It then lists three problems with DLMs. Quality degrades quickly with fewer sampling steps, every step processes the full sequence, and training needs many passes over the dataset to reach good quality. NCM is introduced after that list, though the abstract does not show how NCM addresses each problem.

## How next-concept modeling decodes coarse-to-fine

NCM is a two-stage framework. The first stage is a multiscale residual VQ-VAE that turns text into a hierarchy of discrete codes. The second stage is a transformer that generates those codes level by level, from coarse to fine. The abstract states the core mechanism in a single sentence:

> a multiscale residual VQ-VAE encodes text into a hierarchy of discrete codes, and a transformer generates one level at a time, predicting all codes within a level in parallel
>
> - Kumbong et al., Next-Concept Modeling, arXiv:2610.14162

Sequential generation steps therefore correspond to levels of the hierarchy, not to individual tokens. An autoregressive model takes one step per token. A DLM, by the authors' account, processes the full sequence at every sampling step.

The abstract does not give the number of levels, the number of codes per level, or whether either grows with output length. It also does not say how the VQ-VAE decoder maps codes back to text. Those details determine the real step count and per-step cost, so estimating cost for a given output length requires the full paper.

One commenter on Hacker News called NCM basically a hierarchical diffusion transformer, in which later layers take one big token and generate two smaller ones. On that reading, it is closer to an evaluation strategy that could apply to any model architecture. The abstract does not mention a two-way split per level, so that detail cannot be confirmed from it. The same commenter objected to clickbait videos claiming NCM obsoletes LLMs. The abstract supports a narrower point: NCM still uses a transformer to model language, and what changes is the representation and the generation order.

## The 10x to 31x speedups are against diffusion

Every headline speed number in the abstract compares NCM with the strongest DLM baseline:

- LM1B: better generative perplexity than the strongest DLM baseline at matched parameter scales, while running 21x faster.
- OpenWebText: competitive with the strongest DLM, while running 10x faster.
- XSum: matches the strongest DLM's ROUGE-L, while running 31x faster.

The quality claims differ in strength. LM1B is the only quality win, OpenWebText is described as competitive, and XSum as a match on ROUGE-L. The abstract does not name the DLM baseline or give parameter counts. It also does not say whether faster means wall-clock latency, throughput or function evaluations, and it states no hardware or batch size. Anyone planning serving capacity needs those details to judge whether a 10x figure holds on their own workload.

## Next-token decoding is compared only on Sudoku

The abstract's one autoregressive comparison is on Sudoku, which it describes as a reasoning task. There, NCM outperforms both autoregressive and diffusion baselines across all difficulty levels, using fewer function evaluations. For LM1B, OpenWebText and XSum, the abstract reports no autoregressive perplexity, ROUGE-L or speed figures. Its premise that DLMs are faster than autoregressive models suggests NCM would be faster still, but the abstract gives no number for that comparison.

Commenters on Hacker News raised this directly. One said the paper hardly compares against conventional next-token prediction, which suggests NCM does not beat it except on highly non-linear tasks like Sudoku. The abstract supports the first part, since autoregressive baselines appear only in the Sudoku result. It neither confirms nor rules out the second. Without autoregressive language-modeling numbers, the source gives no basis for estimating the speed and quality trade-off against a standard decoder.

Another commenter suggested that if NCM turns out weaker than next-token prediction, it could still serve as a drafting model, similar to how DFlash works. The abstract does not mention evaluating NCM as a draft model, so that use remains speculation.

## Adopting NCM means a new VQ-VAE and retraining

NCM needs a trained multiscale residual VQ-VAE to define the code hierarchy, plus a transformer trained to generate that hierarchy. The abstract describes no way to convert an existing autoregressive checkpoint. Commenters on Hacker News described this as a brand-new tokenizer and training method that requires significant retraining effort. One said the overhead of essentially training from scratch had made custom tokenizers for domain-specific models much more expensive than fine-tuning. Another asked who would train a large coding model this way, because they lacked the GPUs to do it.

The authors criticise DLMs for needing many passes over the dataset to reach good quality. One could infer that NCM is meant to train more efficiently, but the abstract does not make that claim. It gives no training-compute, token-count or epoch figures for NCM, so training cost cannot be compared from the abstract. The abstract also does not mention a code or weights release.

## What the LM1B, OpenWebText and XSum results support

Taken together, the results support a narrow claim. Against the strongest DLM, NCM's coarse-to-fine hierarchy runs 10x to 31x faster on three text benchmarks. Its quality is better on LM1B, competitive on OpenWebText and matching on XSum. On Sudoku it outperforms both autoregressive and diffusion baselines. Teams already evaluating diffusion models for parallel generation have a concrete reason to test NCM alongside them.

Teams serving autoregressive models today have less to go on. Their decision depends on the quality and latency trade-off against next-token decoding, and the abstract does not report it outside Sudoku. One commenter said wide adoption depends on the performance NCM can reach at frontier scale, and that question is also open. The reported experiments cover LM1B, OpenWebText, XSum and Sudoku, and the abstract gives no parameter counts.

## Questions this raises

### Is next-concept modeling faster than standard next-token decoding?

The abstract does not say. All of its text speed figures (10x to 31x) are measured against the strongest diffusion LM baseline. The only autoregressive comparison is on Sudoku.

### How does next-concept modeling generate text?

A multiscale residual VQ-VAE encodes text into a hierarchy of discrete codes. A transformer then generates one level at a time, coarse to fine, predicting every code within a level in parallel.

### Can an existing autoregressive model be converted to NCM?

The abstract describes no conversion path. NCM needs a trained multiscale residual VQ-VAE and a transformer trained on its code hierarchy. Hacker News commenters said this amounts to significant retraining effort.

## Key facts

- NCM runs 21x faster than the strongest diffusion LM baseline on LM1B, with better generative perplexity at matched parameter scales.
- On OpenWebText NCM is competitive with the strongest diffusion LM while running 10x faster.
- On XSum NCM matches the strongest diffusion LM's ROUGE-L while running 31x faster.
- The arXiv paper by Hermann Kumbong, Christopher Ré and seven co-authors was submitted on 9 October 2026 under cs.CL.
- The abstract's only autoregressive comparison is Sudoku, where NCM beats both autoregressive and diffusion baselines with fewer function evaluations.

## Sources

1. [Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation](https://arxiv.org/abs/2610.14162) - arxiv.org

Discussion read (practitioners' reactions, not a source): https://news.ycombinator.com/item?id=50422168