Mitigating Hallucinations in Large Language Models via Conformal Prediction

Title: Mitigating Hallucinations in Large Language Models via Conformal Prediction

Abstract: Conformal prediction has recently emerged as an effective technique for quantifying uncertainty of deep neural networks. It modifies the neural network to output sets of labels that are guaranteed to contain the true label with high probability under standard assumptions. In this talk, I will describe our recent work applying conformal prediction to improve trustworthiness of large language models (LLMs). First, I will describe how we adapt conformal prediction for LLMs for code generation, where the key challenge is constructing reasonably sized prediction sets. Second, I will describe how we apply conformal prediction to ensure trustworthiness of retrieval augmented question answering. Our results demonstrate how conformal prediction can be a valuable tool for avoiding issues such as hallucinations that plague LLMs.

Bio: Osbert Bastani is an assistant professor at the Department of Computer and Information Science at the University of Pennsylvania. He is broadly interested in techniques for designing trustworthy machine learning systems. Previously, he completed his Ph.D. in computer science from Stanford and his A.B. in mathematics from Harvard.