Skip to content
APIHiver

Next-Concept Modeling (NCM): Stanford's Idea Explained

Did Stanford kill ChatGPT? Next-Concept Modeling (NCM) is a real paper with real speed gains, but against diffusion models, not ChatGPT. Facts and limits.

10 min readBy Mivaan Sanghani
In this post

Every few months a headline says a lab has just "killed ChatGPT." This week it was Stanford's turn. A chart of hard Sudoku puzzles is going around: one small model solves about 1 in 100, and a new one called NCM solves about 86 in 100.

The chart is real. The conclusion drawn from it is not. This guide explains what Next-Concept Modeling actually is, what the numbers measure, and what they leave out. Everything below comes from the paper, the authors' own blog, the code repository and independent write-ups, as read on 11 November 2026.

Bar chart of hard Sudoku results: a word-by-word model solves 1 percent, three diffusion models 30, 58 and 81 percent, and NCM 86 percent

What did Stanford actually release?#

On 9 October 2026 a team from Stanford posted the paper Next-Concept Modeling: Fast, Parallel Coarse-to-Fine Text Generation. Nine authors are listed, including Hermann Kumbong and Ishan Khare as lead authors, with Simran Arora, Azalia Mirhoseini and Christopher Ré as senior authors. The Stanford lab page lists every author under Stanford University; some paper listings also show Together AI, an AI infrastructure company. The lab published its own explainer on its Hazy Research blog on 12 October.

Three things were released:

  • the paper on arXiv,
  • the code on GitHub (MIT licence),
  • checkpoints on Hugging Face, one per task: XSum summaries, German to English translation, Sudoku, and small text sets such as TinyStories, LM1B and OpenWebText.

What was not released: a chat model, an app, or a hosted API. That is a research release, not a product.

How does ChatGPT write, and how does NCM differ?#

ChatGPT and the models like it are autoregressive. They write one token (roughly a word or part of a word) at a time, left to right. Token 200 cannot start until token 199 exists. Long answers are slow because those steps queue up one behind another.

Diffusion language models are the main alternative. They start with a blanked-out text and fill in many positions per pass, over many passes. That removes the queue, but every pass reworks the whole text, and quality falls fast if you cut the number of passes.

NCM is a third route, borrowed from image generation. A picture can be drawn as a blurry thumbnail first and then sharpened. NCM does that for text:

  1. Learn a hierarchy. A trained tokenizer turns text into stacked levels of discrete codes. The top level has a very small number of codes and holds the broad shape. Each lower level is longer and adds only what the levels above missed. The last level has one code per token.
  2. Generate it top-down. A transformer predicts one whole level at a time, with every code in that level predicted in parallel. A decoder then turns the finished stack back into text.

The practical result is a dozen or so steps instead of one per token, and the early steps are cheap because the top levels are short. In the paper's translation example the first decoded draft is nearly unreadable, and by step 11 it reads "However, there is concern that small and medium-sized enterprises will remain vulnerable to hacking and surveillance." Quality does not rise smoothly: the paper notes it rises and dips on the way.

One note on the word "concept". It means the information held in the coarse levels. It does not mean the model thinks in human ideas, and the authors do not claim that.

What do the speed numbers really say?#

The headline numbers are three separate experiments. They should not be merged into one claim.

TestNCM vs strongest diffusion baselineSteps (NCM vs rival)
LM1B (unconditional text)21x faster, better perplexity (93.05 vs 97.30)12 vs 128
OpenWebText (unconditional text)10x faster, competitive (83.12 vs 77.69, lower is better, so the rival is slightly ahead)32 vs 1,024
XSum (summaries)31x faster, matching ROUGE-L (28.0 vs 26.6)8 vs 128
Training data seen (LM1B)about 4.9x fewer tokens (13.36B vs 66B)n/a

Step counts and the LM1B, XSum and training-token figures are taken from the paper's tables as reported by ArXivIQ and Botmonster. The timing is wall-clock on one NVIDIA H100 GPU, generating 1,024 sequences at batch size 16, and every figure comes from the authors. The "fewer tokens" claim counts how many token positions the model saw, not compute or total cost.

Two lines in the paper are less quoted. On translation, NCM scored 22.6 BLEU unguided against 24.8 for the best diffusion rival, and Botmonster reports a same-size word-by-word model at 25.2. NCM catches up only by writing several drafts and picking the best, which costs more steps. On OpenWebText, the rival wins on perplexity. So the honest summary is "faster at similar quality on some tests", not "better everywhere".

What does the Sudoku chart really compare?#

This is the chart behind the viral posts. Sudoku is a good test of planning, because every cell must satisfy row, column and box rules at once, and only a fully correct board counts.

Model (all about 25-29M parameters)PassesEasyMediumHard
Word-by-word model (greedy)12814.6%5.1%1.0%
MDLM (diffusion)12892.0%77.1%30.2%
DUO (diffusion)n/a96.3%84.7%58.4%
FMLM+ (diffusion)1697.7%92.8%81.3%
NCM1299.5%96.5%86.1%

Look at what the "1 in 100" row actually is. It is a tiny 25-million-parameter model trained from scratch on Sudoku, not ChatGPT. The same paper shows diffusion models doing far better than that, and the closest rival (FMLM+) sits at 81.3% against NCM's 86.1%. The gap that matters to the paper is 5 points, not 85.

The authors' explanation is that building the whole board's structure first lets later levels fix earlier guesses, which a left-to-right model cannot do. That is their interpretation. The paper does not show that this is the only cause, and it does not show how a large chatbot with step-by-step reasoning would do on the same puzzle. That comparison is not in the paper.

Why is "NCM kills ChatGPT" the wrong headline?#

Four gaps separate the research result from the headline.

  1. Wrong opponent. The speed claims are against diffusion models. In a post on X the lead author described the comparison as "at similar parameter size" to diffusion baselines.
  2. Small scale. Most experiments use a generator of roughly 100 million parameters. The largest run was 725 million, described as preliminary. Chatbots run to many billions.
  3. One job per model. Each checkpoint is trained for one task. The model that summarises cannot translate, and none of them take a prompt the way a chatbot does.
  4. Same engine. NCM still uses a transformer. What changed is the order in which the answer is built, not the whole stack.

The paper also lists its own limits: a separately trained tokenizer that is then frozen, a fixed number of levels even for short outputs, and occasional glitches where coarse levels hand over to fine ones. The tokenizer rebuilds text almost perfectly, but Botmonster reports its rare slips fall on names and numbers, for example "Melbourne" coming back as "Brisbane."

What happened when someone else ran it?#

A developer on DEV Community (10 November, tagged AI-assisted) ran the official code on a laptop CPU and compared NCM with a word-by-word model trained on the same TinyStories data. This is one person's report, not a replication study, and it used small models:

  • The paper's TinyStories quality figure reproduced closely: perplexity 16.23 against the paper's 15.41.
  • NCM finished a whole story 1.42x faster than the word-by-word model at batch size 1, and 1.79x faster at batch size 4.
  • The word-by-word model wrote more fluent text (perplexity 9.74 against 13.48 for NCM, lower is better) from a single small run.
  • NCM cannot stream. The word-by-word model showed its first token in about 4 milliseconds, while NCM showed nothing until the whole passage was finished at about 307 milliseconds.
  • The translation checkpoint produced visible repeats in some outputs, for example "I like like coffee in the morning."

That last point about streaming matters for chat. People judge a chatbot by how fast the first word appears, and NCM's design gives that up.

How does NCM compare with other "concept" and diffusion ideas?#

Word-by-word (ChatGPT style)Diffusion language modelNCM
How it writesOne token at a timeRefines the whole text over many passesCoarse-to-fine levels, parallel inside each level
Streaming first wordYes, immediatelyDependsNo, output appears at the end
MaturityWidely deployedResearch and some productsEarly research
Scale shownFrontier modelsUp to large modelsAbout 100M, preliminary 725M
Production accessHosted APIsVariesResearch code and checkpoints only

Meta's Large Concept Models (December 2024) are often mixed up with NCM. Meta treats a whole sentence as a "concept" and predicts the next sentence one at a time in an existing sentence-embedding space, with models up to 7 billion parameters. NCM learns its own code hierarchy and predicts it level by level. Same word, different design.

On the diffusion side, Google's DiffusionGemma is an experimental diffusion model built on the Gemma 4 backbone, and Google claims up to 4x faster generation. Botmonster also notes that diffusion text models already run at larger scale in LLaDA and the paid Mercury service. That is the field NCM is competing in, and it is a different race from the one chatbots run.

Should developers care today?#

Yes for the idea, not yet for the product. Here is a checklist for deciding whether to spend time on it:

  • Is generation speed your real bottleneck? If most of your wait is retrieval, tool calls or network time, a faster generator helps only the generating share.
  • Is your task narrow, such as batch summaries, translation or structured output? That is where one-job models fit.
  • Do you need streaming? NCM does not stream.
  • Do you need names and numbers to be exact? Test the tokenizer's slips first.
  • Do you need a hosted API with support? There is none today.

If you build on large models through an API today, the useful habit is to keep your model choice flexible, so a faster approach can be swapped in when it is production-ready. Our guides on the OpenAI API, the Claude API and the OpenAI model shutdown schedule show how quickly model lineups change, and backend primitives for AI engineers covers the plumbing that stays the same either way.

What we did not test#

We did not run NCM ourselves. Every speed and quality number above comes from the paper and its authors, or from the third-party write-ups named in the text, and none has been independently replicated at scale. We did not test how ChatGPT, Claude or Gemini perform on hard Sudoku, so we make no claim about that. We also read only the free part of two paywalled Medium-hosted articles (Level Up Coding and a Medium deep dive), and only the first part of ArXivIQ's summary and Google's DiffusionGemma guide. Where this guide leans on a secondary source for a figure, it says so.

Sources#

Read for this guide, in the order of how much weight they carry:

Try APIs on the APIHiver marketplace

Discover free and paid APIs for travel, finance, AI, weather, maps and more. Test each endpoint in your browser and use one key for all of them.

Share this post

Frequently asked questions

What is next-concept modeling (NCM)?
Next-Concept Modeling is a way of generating text, described in a Stanford paper posted to arXiv on 9 October 2026 (arXiv 2610.14162). Instead of writing one word after another, it first builds a rough, coarse version of the whole passage and then adds detail in a handful of steps. Inside each step, every position is predicted at the same time.
Did Stanford kill ChatGPT?
No. The NCM paper compares its speed and quality with diffusion language models, a different research family, not with ChatGPT, Claude or Gemini. Most of its experiments use models of roughly 100 million parameters, and the released checkpoints each do one job, such as summarising or solving Sudoku. Nothing in the paper shows it matching a modern chatbot.
How is NCM different from ChatGPT?
ChatGPT-style models are autoregressive. They write one token at a time, left to right, and each token waits for the one before it. NCM writes in levels, from a short coarse sketch to the full text, and predicts all the pieces inside a level at once. Both still use a transformer network under the hood.
Is NCM really 31 times faster?
The authors report 21 times faster on LM1B, 10 times on OpenWebText and 31 times on XSum summarisation, but each figure is against the strongest diffusion baseline for that task, measured on one NVIDIA H100 GPU. The numbers come from the authors and are not independent measurements. They are not speed-ups over ChatGPT.
Can I use NCM today?
You can run the research code and the task-specific checkpoints, which are published under the MIT licence on GitHub and Hugging Face. There is no hosted API and no chat model. You need Python 3.10 or newer and, ideally, an NVIDIA GPU.
Is NCM the same as Meta's Large Concept Models?
No. Meta's Large Concept Models (December 2024) treat each sentence as a concept and predict the next sentence in an existing sentence-embedding space. NCM learns its own hierarchy of discrete codes and generates it from coarse to fine. The names are similar, but the designs are different.
Will NCM replace large language models?
Nobody can say yet. The open question is whether the speed gains survive at billions of parameters, with long conversations, tool use and safety training. The more likely near-term role is a fast drafter or planner inside a larger system, which the paper itself does not claim to prove.