Peer Prediction
A mechanism design technique for incentivizing honest reports by scoring agents based on how well their reports correlate with peers, without ground truth.
Peer prediction is a class of mechanism design scoring rules that incentivize honest reporting by paying agents based on the correlation between their reports and the reports of their peers, without access to a ground truth label. The original formulation by Miller, Resnick, and Zeckhauser (2004) showed that the Proper Bayesian Surprise scoring rule dominates truth-telling in expectation.
The technique was developed for crowdsourcing markets where the requester cannot verify answers but can ask multiple workers. In AI safety, peer prediction has been proposed as a way to score AI agent reports without a human or ground truth: if multiple agents see the same input and their honest reports agree, the agreement is rewarded.
The 2026 paper by Bergemann, Koh, and Morris uses peer prediction as a stylized example of discipline via higher-order beliefs. Agents in a mechanism that scores their reports against peers have an incentive to report what they actually believe, even when the principal cannot verify.
Sources
- Mechanism Design for Alignment and Control
arXiv · Sep 1, 2026
- Eliciting Informative Truthful Reports: Peer Prediction (Miller, Resnick, Zeckhauser, 2004)
ResearchGate · Dec 1, 2004
- Peer prediction (Wikipedia)
Wikipedia
Research this in Signals
Signals turns a topic into a sourced research record you can inspect and rerun. Your first scan is free, and this one starts with Peer Prediction already loaded, so edit it or scan as is.