AdvancedAI
Next-Concept Modeling Beats Diffusion LMs at 10-31x Speed
- architecture
- autoregressive
- benchmarks
- llm
Stanford’s Hazy Research group has published Next-Concept Modeling (NCM), a language model that writes text as a hierarchy of codes, coarse structure first and fine detail last, in eight to 32 steps rather than one token at a time. In a preprint posted to arXiv on October 9, 2026, and a blog post three days later, the team reports that NCM matches or beats strong diffusion language models while generating 10 to 31 times faster, and trains on four to five times fewer tokens. The work is a deliberate attempt to fix the three weaknesses the authors attribute to diffusion language models: too many sampling steps, a full-sequence pass at every step, and expensive training.
Two stages, one hierarchy
The first stage is a tokenizer, not a model of text. NCM trains a multiscale residual VQ-VAE that compresses a passage into a stack of discrete code levels: the earliest level holds a handful of codes that capture the broad shape of the sentence, and each later level is longer and adds finer detail. The design follows VAR, a 2024 image-generation method that predicts images scale by scale, with three changes for text. Each level gets its own codebook; nested dropout pushes earlier levels to carry meaning on their own rather than leaning on later ones; and a token-level reconstruction objective keeps the decoded text accurate. Depending on the dataset, NCM uses 8 to 32 levels with codebooks of 4,096 codes, against a text vocabulary of 8,192 to 151,680 tokens.
The second stage freezes that tokenizer and trains a transformer to predict one level at a time, conditioned on the cumulative reconstruction of everything before it. All codes inside a level are predicted in a single parallel pass. Training covers every level in one forward pass, with a fraction of the conditioning codes deliberately corrupted so the model does not fall apart when its own earlier predictions are wrong. At inference the levels are generated in order, and the tokenizer’s decoder converts the finished hierarchy into text.
Where the speed comes from
Diffusion language models win their parallelism by denoising the whole sequence repeatedly, but the number of passes is what limits them, and cutting passes degrades quality quickly. Distillation can reduce that count, at the cost of an extra training stage and lost sample diversity. NCM takes a different route: the sequence of steps is short because there are only as many steps as levels, and the early steps are cheap because early levels are short. In the 12-level configurations used for LM1B, TinyStories and Sudoku, generation takes 12 function evaluations, with most of the compute spent in the last few levels.
The headline numbers, all from the paper: on LM1B, NCM reaches better generative perplexity than the strongest diffusion baseline while running 21 times faster; on OpenWebText it is competitive while running 10 times faster; on XSum summarization it matches the strongest baseline’s ROUGE-L while running 31 times faster. TinyStories and LM1B reach comparable quality with about 11 times fewer function evaluations. Counting both tokenizer and generator training, the team says NCM needs about 4.9 times fewer tokens than strong diffusion baselines on LM1B and about 4 times fewer on OpenWebText.
The results, and their limits
The reasoning result is the most interesting, because it tests the paper’s argument that generating an overall structure first helps tasks with global constraints. On Sudoku the baselines come from a prior comparison paper rather than being retrained here, and the parameter counts are close enough that size does not explain the gap:
| Method | Params | Function evaluations | Easy (40 clues) | Medium (35) | Hard (30) |
|---|---|---|---|---|---|
| Autoregressive (greedy) | 25M | 128 | 14.6% | 5.1% | 1.0% |
| MDLM (diffusion) | 29M | 128 | 92.0% | 77.1% | 30.2% |
| FLM | 29M | 128 | 94.2% | 82.7% | 44.5% |
| FMLM+ | 29M | 16 | 97.7% | 92.8% | 81.3% |
| NCM | 25M | 12 | 99.5% | 96.5% | 86.1% |
Read the table with the paper’s own scope in mind. These are tens-of-millions-parameter models on benchmark datasets built for diffusion research, not frontier-scale language models; the largest NCM model in the scaling study is 725 million parameters, and the authors call those curves preliminary. The 21- and 31-fold speedups are measured against diffusion baselines on one H100, not against a production autoregressive serving stack with paged attention and batching. The VQ-VAE tokenizer is a second model with its own 15 to 144 million parameters in the paper’s configurations, and the authors note that tokenizers of this kind have failure cases in reconstruction. Nor has the work been peer-reviewed.
The authors are explicit about what they have not done: they have not tested how NCM scales in parameters, FLOPs and tokens; they trained the tokenizer and generator separately rather than jointly; and they use a fixed hierarchy for every input instead of learning when to stop or spend more computation. Each of those is a plausible source of further gains, and each is also a reason the current numbers are an early result rather than a settled one.
What This Means
Most work on faster generation tries to squeeze the number of denoising passes or to distill a big sampler into a small one. NCM changes the axis: it makes the representation itself hierarchical, so a short sequence of steps covers the whole output. If that holds up at larger scale, the practical argument is not that diffusion language models are beaten on quality, but that parallel text generation may not need a diffusion process at all. The test-time scaling result points the same way: because each sample is cheap, the model can draw more candidates within the same time budget, and minimum Bayes risk decoding over those candidates raised BLEU on WMT14 and ROUGE on XSum in the reported experiments.
For now, the honest summary is that a small research team has shown a genuinely different generation paradigm working at small scale, with released code and an unusually clear list of open questions. Whether it survives the jump to billions of parameters is the next paper’s problem.
This post was drafted with AI assistance and reviewed by RITS staff.
Sources
- Kumbong et al., “Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation,” arXiv:2610.14162, submitted October 9, 2026
- Hazy Research, “Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation,” October 12, 2026
- HazyResearch/ncm on GitHub — code release
- Stanford Scaling Intelligence Lab, NCM blog post






