---
title: Pre-Training Checkpoint
type: vocabulary
url: "https://www.envisioning.com/vocab/pre-training-checkpoint"
summary: A snapshot of model weights saved at the end of (or during) the pretraining run, before post-training alignment, used as the starting point for fine-tuning and as a stable artifact for evaluation and deployment.
year: 2018
generality: 0.65
---

# Pre-Training Checkpoint

A snapshot of model weights saved at the end of (or during) the pretraining run, before post-training alignment, used as the starting point for fine-tuning and as a stable artifact for evaluation and deployment.
A pre-training checkpoint is a snapshot of a model's parameters saved at the end of — or periodically during — the pretraining phase of training, before any post-training steps such as instruction tuning, RLHF, or tool-use alignment. The checkpoint captures the raw language-modeling behavior learned from the pretraining corpus and serves as the base from which all downstream variants (chat, code, reasoning, distilled sizes) are produced. Frontier-model training pipelines typically produce a small number of major-version checkpoints (e.g., GPT-4 → GPT-5 base) plus many intermediate ones used for capability research; minor-version public releases are usually derived from a mixture of the most recent checkpoint and the strongest post-training techniques available at release time. Pre-training checkpoints are massive (often hundreds of gigabytes), expensive to produce (months of compute on thousands of accelerators), and treated as proprietary infrastructure — though some labs have released base checkpoints under research licenses. The term "base model" and "pre-training checkpoint" are sometimes used interchangeably, though "base model" more often refers to the public-facing release while "pre-training checkpoint" emphasizes the artifact itself.

---
Source: Envisioning — Technology Research Institute (https://www.envisioning.com/vocab/pre-training-checkpoint)