XConf (Experiential Confidence)

XConf (eXperiential Confidence) estimates whether a language model's answer is likely correct. Caiqi Zhang, Xiaochen Zhu, Chengzu Li, Yulong Chen, Dharshan Kumaran and Nigel Collier introduced it in "Confidence Comes from Experience: Experiential Confidence Estimation from Reasoning to Agents" (arXiv:2609.17708, September 2026). Most estimators, such as self-consistency sampling or reading token-level probabilities, judge a response only from the current inference trace. XConf instead draws on the model's history of graded past episodes.

The model keeps a bank of past episodes, each recording the task, the model's reflection, its stated confidence, the outcome, and a lesson written once the grade arrived. XConf's Recall stage retrieves past episodes on similar tasks met with a similar stated confidence and reads off their historical success rate. Its Reflect stage shows the model this record, has it name a recurring failure mode, and restate a confidence informed by its own track record. The method works across output formats, needs no logit access or weight updates, and costs one extra answer generation.

Across nine benchmarks in reasoning, coding, multimodal QA and interactive agents, on four models from three model families, XConf beats or matches ten-sample self-consistency on discrimination (AUROC) in 23 of 24 comparisons, with lower calibration error, at roughly a tenth of the generation cost. Abstaining on the 10% least-confident episodes in selective prediction raises success rate by up to 8.7 points on agent tasks.