Skip to content
KinPilot™Learn & Explore
Menu
← TopicsReading path →

Grades 9–12 · Computing

How does an AI language model generate text?

A language model generates text by repeatedly extending a sequence of tokens. A token is a unit of representation that may be a whole word, part of a word or punctuation. The available context includes the preceding tokens and, in many systems, instructions describing the requested task.

D50 visual explanationAn invented table supplies next-token probabilities for context The cat: sat 0.6, slept 0.3, ran 0.1. They sum to one. Position 0.1 in the cumulative intervals selects sat; appending it yields The cat sat. Real language models learn context-dependent distributions over much larger token vocabularies. This table is not a neural network or an actual language model output. Likelihood of a token does not establish truth.A language model generates text one token at a timeThree next-token candidates • invented probabilitiesContextThe catModel predictspossible next tokenssat60%slept30%ran10%Select one → “The cat sat”Append the token, then predict again from the new context.Token likelihood is not a probability that a claim is true.

In the diagramA language model generates text one token at a timeThree next-token candidates • invented probabilitiesContextThe catModel predictspossible next tokenssat60%slept30%ran10%Select one → “The cat sat”Append the token, then predict again from the new context.Token likelihood is not a probability that a claim is true.

The invented table assigns three next-token probabilities. One token is selected and appended before prediction repeats; none of these probabilities measures factual truth.

During training, a neural network adjusts numerical parameters using examples, including the task of predicting a token from preceding text. A transformer represents tokens with lists of numbers. Its attention mechanism calculates how strongly information at different context positions should contribute, then mixes their representations with those weights. Further network calculations produce scores for possible next tokens, which are converted into a probability distribution. Context therefore changes the computation, rather than merely selecting a fixed word from a dictionary.

The picture replaces that large computation with a tiny invented table. After “The cat,” it assigns probabilities of 0.6 to “sat,” 0.3 to “slept” and 0.1 to “ran.” These sum to one. A selection rule can take the highest-probability token or sample according to the distribution. After a token is selected, it joins the context and the process can run again. The displayed whole words and probabilities are teaching choices, not predictions from an actual language model.

Next-token probability describes how the model is likely to continue in a context. It is not a probability that a claim is true. Learned patterns can support useful explanations, but a fluent continuation can also contain invented facts or inconsistent reasoning. Additional training, retrieved information and other tools can change a system’s behavior without turning linguistic likelihood into a guarantee of accuracy.

About this explanation

Starting knowledge: Understand probabilities between zero and one that sum to one, and a repeated computational step.

Connect this idea: Networks, Exchange & Time

Sources

  • Google Research — Mechanics of Next Token Prediction with Transformers
  • Google AI for Developers — Understand and count tokens
  • Zhang, Press, Merrill, Liu and Smith — How Language Model Hallucinations Can Snowball
  • Vaswani and colleagues, NeurIPS — Attention Is All You Need
← Grades 9–12 topics