Red Teaming
person concept tool org talk claim — click a node to jump to its page; hover an arrow for the relation
Red teaming is a security and safety practice involving adversarial testing of AI systems, where testers attempt to find vulnerabilities, exploit prompt injections, and probe for weaknesses in order to improve system robustness.
Role in Agentic AI Development
In the context of Claude Code's autonomous "Auto Mode," red teaming is described as a core component of the development and hardening process. Testers were specifically tasked with attempting prompt injection attacks and hacking the codebase, and the results of these exercises were directly used to create evals — converting discovered vulnerabilities into systematic test coverage. source
More broadly, red teaming is framed as a continuous, ongoing discipline rather than a one-time gate — characterized as something one should "always" be doing alongside pentesting as part of responsible agentic system operation. source
Connection to Evaluation Infrastructure
A notable pattern described is the pipeline from red teaming to formal evals: adversarial findings discovered by red teamers are codified into repeatable evaluation benchmarks, closing the loop between attack discovery and systematic regression testing. This positions red teaming not just as a safety check but as a driver of evals infrastructure development.