---
title: Model Elicitation
type: vocabulary
url: "https://www.envisioning.com/vocab/model-elicitation"
summary: Systematically discovering what a model can do beyond documented capabilities.
year: 2023
generality: 0.60
---

# Model Elicitation

Systematically discovering what a model can do beyond documented capabilities.
## Opening
Model elicitation is the research practice of systematically discovering capabilities a model already has but does not surface under default prompting or evaluation. The term entered wide use around 2023 alongside the discovery that frontier models possess latent abilities — coding languages they were not explicitly trained on, tool-use patterns they improvise when given the chance, multimodal skills they exhibit when asked the "right" way — that standard benchmarks and standard prompting fail to surface. Boris Cherny named it explicitly at Y Combinator in 2026 as a research area in its own right, framing the post-training challenge as figuring out how to "ask the model to do the right thing."

## Mechanism
Elicitation methods fall into three families. Probe-style elicitation: targeted prompts that test specific capability hypotheses (Anthropic's internal example is asking Opus 5 to use OpenCV to draw portraits, landscapes, and animals — a capability the model was never explicitly trained on, discovered accidentally through "playing around"). Workflow-style elicitation: changing the harness to give the model harder tasks than the user thinks it can do, then letting the model self-verify (Cherny's example is giving Opus 5 an Electron-to-Swift rewrite with pixel-by-pixel verification — a 14-day task it solved without explicit scaffolding). Comparative elicitation: running the same task across model generations and observing what becomes solvable when, which is how Anthropic identified the Bun Zig-to-Rust rewrite as a "test problem" they throw at every new generation.

## Tradeoffs
Model elicitation's central trade-off is between probing aggressively (risking wasted compute on dead ends) and waiting for the model to surface capabilities organically through user feedback (risking missed opportunities). Cherny's heuristic is to "give the model slightly harder tasks than what you think it can do" — under-asking is the dominant failure mode. The methodology trade-off is reproducibility: elicited capabilities are often sensitive to small prompt variations, harness configurations, and even random seeds, making it hard to know whether a new capability is a real capability or a fragile artifact of the elicitation setup.

## Open Questions
Whether model elicitation has a stable ceiling (capabilities eventually exhaust through systematic probing) or whether it scales with model size (larger models have more latent capability per unit of probing effort). Whether elicitation methods transfer across model families (does an elicitation technique that works for Claude work for GPT, Gemini, Llama?). Whether elicited capabilities become more stable over training (i.e., eventually the model "internalizes" them and surfaces them without specific prompting) or remain fragile forever. Whether model-elicitation research should be standardized as benchmarks (measuring how much capability a model has under systematic probing) or remain a craft skill inside labs.

---
Source: Envisioning — Technology Research Institute (https://www.envisioning.com/vocab/model-elicitation)