Calibration of large language models based on the conformal prediction

Shamil Chankaev, Eugene Ilyushin

Abstract


Modern language models are widely used across various domains due to their ability to generate coherent and contextually relevant text. However, despite significant advancements, such models remain prone to errors and may produce hallucinations—statements that are well-formed but factually incorrect. This article proposes a novel approach to evaluating uncertainty in language model responses based on conformal prediction methods. It explores techniques for calibrating the models’ probabilistic estimates in order to enhance the reliability and interpretability of their outputs. The study demonstrates how the proposed method can more effectively identify potential errors and improve the justification of  generated responses. The results open up new possibilities for increasing the reliability of language models in critical applications where accuracy and confidence in responses are of paramount importance.


Full Text:

PDF

References


Gammerman Alex, Vovk Volodya, Vapnik Vladimir. Learning by transduction // arXiv preprint arXiv:1301.7375. — 2013.— URL: https://arxiv.org/abs/1301.7375.

Sadinle Mauricio, Lei Jing, Wasserman Larry. Least ambiguous set valued classifiers with bounded error levels // Journal of the American Statistical Association. — 2019. — Vol. 114, no. 525. — P. 223–234. — URL: https://doi.org/10.1080/01621459.2017.1395341.

Conformal prediction with large language models for multi-choice question answering / Bhawesh Kumar, Charlie Lu, Gauri Gupta et al. // arXiv preprint arXiv:2305.18404. — 2023. — URL: https://arxiv.org/abs/2305.18404.

Robots that ask for help: Uncertainty alignment for large language model planners / Allen Z. Ren, Anushri Dixit, Alexandra Bodrova et al. // arXiv preprint arXiv:2307.01928. — 2023.— URL: https://arxiv.org/abs/2307.01928.

Kang Zhewei, Zhao Xuandong, Song Dawn. Scalable best-of-n selection for large language models via self-certainty // arXiv preprint arXiv:2502.18581. — 2025. — URL: https://arxiv.org/abs/2502.18581.

Angelopoulos Anastasios N., Bates Stephen. A gentle introduction to conformal prediction and distribution-free uncertainty quantification // arXiv preprint arXiv:2107.07511. — 2021. — URL: https://arxiv.org/abs/2107.07511.

Self-consistency improves chain of thought reasoning in language models / Xuezhi Wang, Jason Wei, Dale Schuurmans et al. // arXiv preprint arXiv:2203.11171. — 2022. — Published at ICLR 2023. URL: https://arxiv.org/abs/2203.11171.

Let’s verify step by step / Hunter Lightman, Vineet Kosaraju, Yura Burda et al. // arXiv preprint arXiv:2305.20050. — 2023. — URL: https://arxiv.org/abs/2305.20050.

Math-shepherd: Verify and reinforce llms step-by-step without human annotations / Peiyi Wang, Lei Li, Zhihong Shao et al. // Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). — Bangkok, Thailand : Association for Computational Linguistics, 2024. — August. — P. 9426–9439. URL: https://aclanthology.org/2024.acl-long.510.


Refbacks

  • There are currently no refbacks.


Abava  Кибербезопасность ИБП для ЦОД СНЭ

ISSN: 2307-8162