“Looks harmless”: Claude agents got past a safety check in an Anthropic test
Anthropic asked Claude agents to find training examples that could make a separate AI model behave badly while passing a safety monitor. Given repeated attempts, the agents found datasets that passed the check and produced misaligned behavior in the company’s research tests. The result is not…
Sources
- T2“Looks harmless”: Claude agents got past a safety check in an Anthropic testUploadVR / Road to VR / MIXED