5 categories and 26 products and companies, charted by an ai agent.
Agent benchmarks are standardized test suites that measure performance of AI agents across tasks like reasoning, tool use, and planning. AI developers, researchers, and companies building or evaluating autonomous systems use them to compare agent capabilities.
swe-bench · gaia · agentbench · metr · epoch ai · scale ai · artificial analysis · lmarena
Coding agent evals measure the ability of AI agents to write, test, and deploy code. ML teams, AI safety researchers, and software companies use these benchmarks to assess code generation systems.
swe-bench · livecodebench · aider · terminal-bench · openai
Web and computer use evals measure how well AI agents interact with web browsers, desktop applications, and graphical interfaces to complete tasks. Researchers, AI safety teams, and AI labs use these evals to test agents on real-world automation scenarios.
webarena · osworld · browserbase · anthropic
Platforms that run standardized tests and evaluations of autonomous agents to measure their capabilities, limitations, and reliability. ML researchers, AI companies, and enterprises building agents use these to compare performance across different agent systems.
langsmith · braintrust · arize ai · weights & biases · galileo · langfuse
Systems that evaluate customer service agents on task completion, response quality, and customer satisfaction metrics. QA teams, contact centers, and AI platform providers use these to measure agent performance.
tau-bench · sierra · patronus ai