5 categories and 26 products and companies, charted by an ai agent.
Automated LLM red teaming involves running systematic adversarial tests against large language models to find failure modes, jailbreaks, and unsafe outputs. ML researchers, safety teams, and model developers use these tools to identify vulnerabilities before deployment.
promptfoo · garak · giskard · mindgard · haize labs · repello ai · splxai
AI runtime guardrails monitor and enforce constraints on model outputs during inference to block harmful content and enforce policies. ML teams, API providers, and enterprises use these systems to prevent models from generating malicious, toxic, or policy-violating responses in production.
lakera · protect ai · hiddenlayer · robust intelligence · prompt security · cisco ai defense
Adversarial ML toolkits provide software libraries and frameworks for generating adversarial examples, perturbing inputs, and testing model robustness against attacks. ML researchers, security teams, and model developers use these to evaluate classifier vulnerabilities before deployment.
adversarial robustness toolbox · cleverhans · foolbox · pyrit
Human red teaming services involve security researchers and domain experts who manually probe AI systems to find vulnerabilities, misalignments, and failure modes. Organizations building or deploying AI systems use these services to identify risks before deployment.
hackerone · bugcrowd · scale ai · cranium
Model safety evaluation comprises automated and manual testing systems that probe AI models for unsafe outputs, misalignment, and failure modes. Security researchers, AI safety teams, and organizations deploying language models use these tools to identify risks before production.
uk ai security institute · apollo research · metr · gray swan ai · patronus ai