Every few months a headline says a lab has just "killed ChatGPT." This week it was Stanford's turn. A chart of hard Sudoku puzzles is going around: one small model solves about 1 in 100, and a new one called NCM solves about 86 in 100.
The chart is real. The conclusion drawn from it is not. This guide explains what Next-Concept Modeling actually is, what the numbers measure, and what they leave out. Everything below comes from the paper, the authors' own blog, the code repository and independent write-ups, as read on 11 November 2026.
What did Stanford actually release?#
On 9 October 2026 a team from Stanford posted the paper Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation. Nine authors are listed, including Hermann Kumbong and Ishan Khare as lead authors, with Simran Arora, Azalia Mirhoseini and Christopher Ré as senior authors. The Stanford lab page lists every author under Stanford University; some paper listings also show Together AI, an AI infrastructure company. The lab published its own explainer on its Hazy Research blog on 12 October.
Three things were released:
- the paper on arXiv,
- the code on GitHub (MIT licence),
- checkpoints on Hugging Face, one per task: XSum summaries, German to English translation, Sudoku, and small text sets such as TinyStories, LM1B and OpenWebText.
What was not released: a chat model, an app, or a hosted API. That is a research release, not a product.
How does ChatGPT write, and how does NCM differ?#
ChatGPT and the models like it are autoregressive. They write one token (roughly a word or part of a word) at a time, left to right. Token 200 cannot start until token 199 exists. Long answers are slow because those steps queue up one behind another.
Diffusion language models are the main alternative. They start with a blanked-out text and fill in many positions per pass, over many passes. That removes the queue, but every pass reworks the whole text, and quality falls fast if you cut the number of passes.
NCM is a third route, borrowed from image generation. A picture can be drawn as a blurry thumbnail first and then sharpened. NCM does that for text:
- Learn a hierarchy. A trained tokenizer turns text into stacked levels of discrete codes. The top level has a very small number of codes and holds the broad shape. Each lower level is longer and adds only what the levels above missed. The last level has one code per token.
- Generate it top-down. A transformer predicts one whole level at a time, with every code in that level predicted in parallel. A decoder then turns the finished stack back into text.
The practical result is a dozen or so steps instead of one per token, and the early steps are cheap because the top levels are short. In the paper's translation example the first decoded draft is nearly unreadable, and by step 11 it reads "However, there is concern that small and medium-sized enterprises will remain vulnerable to hacking and surveillance." Quality does not rise smoothly: the paper notes it rises and dips on the way.
One note on the word "concept". It means the information held in the coarse levels. It does not mean the model thinks in human ideas, and the authors do not claim that.
What do the speed numbers really say?#
The headline numbers are three separate experiments. They should not be merged into one claim.
| Test | NCM vs strongest diffusion baseline | Steps (NCM vs rival) |
|---|---|---|
| LM1B (unconditional text) | 21x faster, better perplexity (93.05 vs 97.30) | 12 vs 128 |
| OpenWebText (unconditional text) | 10x faster, competitive (83.12 vs 77.69, lower is better, so the rival is slightly ahead) | 32 vs 1,024 |
| XSum (summaries) | 31x faster, matching ROUGE-L (28.0 vs 26.6) | 8 vs 128 |
| Training data seen (LM1B) | about 4.9x fewer tokens (13.36B vs 66B) | n/a |
Step counts and the LM1B, XSum and training-token figures are taken from the paper's tables as reported by ArXivIQ and Botmonster. The timing is wall-clock on one NVIDIA H100 GPU, generating 1,024 sequences at batch size 16, and every figure comes from the authors. The "fewer tokens" claim counts how many token positions the model saw, not compute or total cost.
Two lines in the paper are less quoted. On translation, NCM scored 22.6 BLEU unguided against 24.8 for the best diffusion rival, and Botmonster reports a same-size word-by-word model at 25.2. NCM catches up only by writing several drafts and picking the best, which costs more steps. On OpenWebText, the rival wins on perplexity. So the honest summary is "faster at similar quality on some tests", not "better everywhere".
What does the Sudoku chart really compare?#
This is the chart behind the viral posts. Sudoku is a good test of planning, because every cell must satisfy row, column and box rules at once, and only a fully correct board counts.
| Model (all about 25-29M parameters) | Passes | Easy | Medium | Hard |
|---|---|---|---|---|
| Word-by-word model (greedy) | 128 | 14.6% | 5.1% | 1.0% |
| MDLM (diffusion) | 128 | 92.0% | 77.1% | 30.2% |
| DUO (diffusion) | n/a | 96.3% | 84.7% | 58.4% |
| FMLM+ (diffusion) | 16 | 97.7% | 92.8% | 81.3% |
| NCM | 12 | 99.5% | 96.5% | 86.1% |
Look at what the "1 in 100" row actually is. It is a tiny 25-million-parameter model trained from scratch on Sudoku, not ChatGPT. The same paper shows diffusion models doing far better than that, and the closest rival (FMLM+) sits at 81.3% against NCM's 86.1%. The gap that matters to the paper is 5 points, not 85.
The authors' explanation is that building the whole board's structure first lets later levels fix earlier guesses, which a left-to-right model cannot do. That is their interpretation. The paper does not show that this is the only cause, and it does not show how a large chatbot with step-by-step reasoning would do on the same puzzle. That comparison is not in the paper.
Why is "NCM kills ChatGPT" the wrong headline?#
Four gaps separate the research result from the headline.
- Wrong opponent. The speed claims are against diffusion models. In a post on X the lead author described the comparison as "at similar parameter size" to diffusion baselines.
- Small scale. Most experiments use a generator of roughly 100 million parameters. The largest run was 725 million, described as preliminary. Chatbots run to many billions.
- One job per model. Each checkpoint is trained for one task. The model that summarises cannot translate, and none of them take a prompt the way a chatbot does.
- Same engine. NCM still uses a transformer. What changed is the order in which the answer is built, not the whole stack.
The paper also lists its own limits: a separately trained tokenizer that is then frozen, a fixed number of levels even for short outputs, and occasional glitches where coarse levels hand over to fine ones. The tokenizer rebuilds text almost perfectly, but Botmonster reports its rare slips fall on names and numbers, for example "Melbourne" coming back as "Brisbane."
What happened when someone else ran it?#
A developer on DEV Community (10 November, tagged AI-assisted) ran the official code on a laptop CPU and compared NCM with a word-by-word model trained on the same TinyStories data. This is one person's report, not a replication study, and it used small models:
- The paper's TinyStories quality figure reproduced closely: perplexity 16.23 against the paper's 15.41.
- NCM finished a whole story 1.42x faster than the word-by-word model at batch size 1, and 1.79x faster at batch size 4.
- The word-by-word model wrote more fluent text (perplexity 9.74 against 13.48 for NCM, lower is better) from a single small run.
- NCM cannot stream. The word-by-word model showed its first token in about 4 milliseconds, while NCM showed nothing until the whole passage was finished at about 307 milliseconds.
- The translation checkpoint produced visible repeats in some outputs, for example "I like like coffee in the morning."
That last point about streaming matters for chat. People judge a chatbot by how fast the first word appears, and NCM's design gives that up.
How does NCM compare with other "concept" and diffusion ideas?#
| Word-by-word (ChatGPT style) | Diffusion language model | NCM | |
|---|---|---|---|
| How it writes | One token at a time | Refines the whole text over many passes | Coarse-to-fine levels, parallel inside each level |
| Streaming first word | Yes, immediately | Depends | No, output appears at the end |
| Maturity | Widely deployed | Research and some products | Early research |
| Scale shown | Frontier models | Up to large models | About 100M, preliminary 725M |
| Production access | Hosted APIs | Varies | Research code and checkpoints only |
Meta's Large Concept Models (December 2024) are often mixed up with NCM. Meta treats a whole sentence as a "concept" and predicts the next sentence one at a time in an existing sentence-embedding space, with models up to 7 billion parameters. NCM learns its own code hierarchy and predicts it level by level. Same word, different design.
On the diffusion side, Google's DiffusionGemma is an experimental diffusion model built on the Gemma 4 backbone, and Google claims up to 4x faster generation. Botmonster also notes that diffusion text models already run at larger scale in LLaDA and the paid Mercury service. That is the field NCM is competing in, and it is a different race from the one chatbots run.
Should developers care today?#
Yes for the idea, not yet for the product. Here is a checklist for deciding whether to spend time on it:
- Is generation speed your real bottleneck? If most of your wait is retrieval, tool calls or network time, a faster generator helps only the generating share.
- Is your task narrow, such as batch summaries, translation or structured output? That is where one-job models fit.
- Do you need streaming? NCM does not stream.
- Do you need names and numbers to be exact? Test the tokenizer's slips first.
- Do you need a hosted API with support? There is none today.
If you build on large models through an API today, the useful habit is to keep your model choice flexible, so a faster approach can be swapped in when it is production-ready. Our guides on the OpenAI API, the Claude API and the OpenAI model shutdown schedule show how quickly model lineups change, and backend primitives for AI engineers covers the plumbing that stays the same either way.
What we did not test#
We did not run NCM ourselves. Every speed and quality number above comes from the paper and its authors, or from the third-party write-ups named in the text, and none has been independently replicated at scale. We did not test how ChatGPT, Claude or Gemini perform on hard Sudoku, so we make no claim about that. We also read only the free part of two paywalled Medium-hosted articles (Level Up Coding and a Medium deep dive), and only the first part of ArXivIQ's summary and Google's DiffusionGemma guide. Where this guide leans on a secondary source for a figure, it says so.
Sources#
Read for this guide, in the order of how much weight they carry:
- NCM paper abstract, arXiv 2610.14162
- Stanford Scaling Intelligence Lab, NCM page
- Hazy Research blog, 12 October 2026
- HazyResearch/ncm on GitHub
- alphaXiv paper page, including the lead author's thread
- ArXivIQ summary
- Eden AI, NCM vs LLM
- Botmonster, next-concept modeling writes the gist first
- AI Angst, why it isn't the end of LLMs
- ITECS, Stanford's Next-Concept Modeling
- AI Weekly coverage
- DEV Community, hands-on run of the official code
- DEV Community, technical analysis
- Medium, "Did Stanford Just Kill ChatGPT? I Read the Paper"
- Level Up Coding, the viral article (free portion)
- Meta, Large Concept Models, arXiv 2412.08821
- Google, DiffusionGemma developer guide