How good are slop-vestigators?

T2Covered by 1 source · first reported by AI Alignment Forum / LessWrong (curated) at 08 Sept, 22:52 UTC

TLDR: We release MessageBoardAuditBench: a benchmark to measure how well agents can replicate the recent investigation into a swarm of OpenAI agents colluding via a message board on an online wiki. We open-source the benchmark as an Inspect eval. We find that top models cover up to 51% of findings…