---
title: Data Mixture
type: vocabulary
url: "https://www.envisioning.com/vocab/data-mixture"
summary: The composition of training data — the proportions of different sources, domains, languages, and document types — used to train a machine learning model.
year: 2020
generality: 0.72
---

# Data Mixture

The composition of training data — the proportions of different sources, domains, languages, and document types — used to train a machine learning model.
A data mixture is the composition of a model's training corpus: the relative proportions of different data sources, domains (web, books, code, scientific papers, dialogue), languages, document types, and quality strata that are mixed together before or during pretraining. Data mixture is one of the most consequential design choices in modern pretraining pipelines — small changes in mixture proportions can shift a model's downstream capabilities substantially, and frontier model labs treat the recipe as proprietary. Mixture design involves tradeoffs between coverage (diversity of topics and languages), quality (weighting cleaner sources like curated textbooks over noisy web scrapes), and capability transfer (some data sources disproportionately improve downstream reasoning). Data mixture inference is the related but distinct practice of reverse-engineering a closed model's mixture from its external behavior. Public mixture specifications (e.g., the GPT-3, LLaMA, and DeepSeek papers) have become a reference grammar for the field even when exact ratios are not reproduced.

---
Source: Envisioning — Technology Research Institute (https://www.envisioning.com/vocab/data-mixture)