TruthTorchLM: A Comprehensive Library for Predicting Truthfulness in LLM Outputs

Abstract

TruthTorchLM is an open-source library for estimating the truthfulness of LLM outputs. It unifies over 30 truthfulness methods spanning uncertainty estimation, verification, and supervised detectors, with different trade-offs in compute, model access, and grounding requirements. The toolkit integrates with HuggingFace and LiteLLM and provides interfaces for generation, calibration, evaluation, and long-form truthfulness analysis. Benchmarking on TriviaQA, GSM8K, and FactScore-Bio demonstrates practical utility for reliable deployment workflows.

Yavuz Faruk Bakman

PhD Student in Computer Science Capital One Responsible AI Fellow

My research interests include Trustworthy LLM, Continual Learning and Federated Learning.