Red Teaming

Red teaming is adversarial testing: people try to make the model produce harmful, false, or off-policy output, and the findings feed back into training and guardrails. It differs from normal evaluation by being creative and open-ended rather than scored against a fixed set. For frontier models it is increasingly a regulatory expectation.

In practice: Twenty people spending a week trying to make a support bot promise refunds it cannot honour.