Quick reference

LLM training pipeline

1. Collect and license data.
2. Filter, deduplicate, tokenize, and split it into sequences.
3. Pretrain: predict next tokens and update weights many times.
4. Evaluate on held-out data, benchmarks, and safety tests.
5. Post-train with instruction examples and preference/safety feedback.
6. Re-evaluate; release weights, code, or both.

What changes during training?

The model's weights change a tiny amount at every optimizer step. Training seeks lower average prediction loss on examples it has not directly used for that update. Lower loss is useful but does not itself prove factuality, fairness, or safety.

What does not happen?

Weights are not updated while you chat with a normally deployed model. That is inference, not training.

Sources: Vaswani et al. (2017); Hoffmann et al. (2022); Ouyang et al. (2022).