AI models are trained to refuse harmful requests but their refusal mechanisms are probabilistic

In 2021, Anthropic stated that large language models should be helpful, honest, and harmless, meaning they should politely refuse dangerous requests like building a bomb. Modern AI models are trained to refuse a vast number of harmful prompts, often using AI to teach AI how to say no. However, refusal is probabilistic, meaning it often fails, sometimes with violent results, and models still contain knowledge of violence and vitriol, scaling with their intelligence. Early models could easily generate responses to self-harm queries, and current models can perform tasks like breaking into critical computer networks or deforming public opinion.