Lesson 1 · about 6 minutes

An LLM writes by predicting what comes next.

The central skill: trace one generation step without imagining a hidden database of answers.

The one-sentence model

An LLM receives the text so far, estimates a probability for every possible token that could follow, chooses one, appends it, and repeats. The selected token becomes part of the context for the next step.

Why this is surprisingly powerful

During training, the model is shown vast quantities of text and repeatedly corrected when its next-token prediction is poor. Its weights gradually encode patterns that help with this task: spelling, grammar, facts that appeared in training, styles of argument, and many associations between ideas. At inference time, those weights are fixed; the model only runs the prediction loop.

The “large” in LLM means there are many learned weights and much training data—not that it uses a fundamentally different output rule.

Open-source model, same loop

With an open-weight model, people can download the weights and run this prediction loop on their own hardware or servers. Openness changes who can inspect, adapt, and host the model; it does not change the basic mechanism above.

Keep this distinction

Training changes the weights by learning from examples. Inference uses the weights to predict tokens. Confusing these two makes many LLM explanations feel mystical.

Primary source: Brown et al., “Language Models are Few-Shot Learners”, especially its description of autoregressive language modeling. See also the glossary.