---
title: Incompressible Knowledge Probes
type: vocabulary
url: "https://www.envisioning.com/vocab/incompressible-knowledge-probes"
summary: A probing technique that uses rare or niche factual recall to estimate model capacity, on the assumption that facts not compressible into shared representations reveal parameter count.
year: 2025
generality: 0.60
---

# Incompressible Knowledge Probes

A probing technique that uses rare or niche factual recall to estimate model capacity, on the assumption that facts not compressible into shared representations reveal parameter count.
Incompressible Knowledge Probes (IKPs) are a class of evaluation probes designed to estimate the parameter count of a language model by testing it on niche, low-prevalence factual knowledge. The technique assumes that for sufficiently rare facts, a model cannot share representational capacity across examples, so the amount of niche knowledge it can recall scales approximately with its parameter count. By curating a benchmark of obscure but verifiable facts and measuring recall accuracy across models of known size, researchers can extrapolate to estimate the size of unknown models. The technique was introduced for analyzing frontier model releases where parameter counts are not publicly disclosed.

---
Source: Envisioning — Technology Research Institute (https://www.envisioning.com/vocab/incompressible-knowledge-probes)