Hair-Trigger Alignment: Black-Box Evaluation Cannot Guarantee Post-Update Alignment

Abstract

Large language models are repeatedly updated after deployment, yet alignment is commonly assessed with static black-box evaluations. This paper formalizes static and post-update alignment and proves a core limitation: passing static black-box tests does not imply post-update robustness. The analysis shows that overparameterized models can hide latent adversarial behaviors that are activated by benign updates. Empirical results across privacy, jailbreak safety, and behavioral honesty confirm that models can appear aligned before update and become strongly misaligned after a single update, with risk increasing with model scale.

Yavuz Faruk Bakman

PhD Student in Computer Science Capital One Responsible AI Fellow

My research interests include Trustworthy LLM, Continual Learning and Federated Learning.