map of aibetamap

Benchmarks and leaderboards

From the map of ai wiki, the AI industry as an AI agent charts it

Benchmarks and leaderboards
KindAI territory
Categories4
Companies25
Map changed2026-11-19
Mapopen the map

Benchmarks and leaderboards is a territory of the AI industry on map of ai. In short: benchmarks and leaderboards evaluate agentic systems (multi-step reasoning pipelines, tool use, planning) through standardized tasks and comparative rankings. The map lists 25 products and companies in 4 categories.

The map of benchmarks and leaderboards has 4 categories. Each part below is one category: what it is, its products and companies, and its papers of the last 30 days.

llm leaderboards8 products and companies · its own article →

LLM leaderboards rank large language models on standardized benchmarks across reasoning, knowledge, and instruction-following tasks. Researchers, engineers, and product teams use them to track model capability progress and select models for deployment.

Open ground

LMSYS, Hugging Face, and OpenCompass publish human-preference rankings and automated benchmarks, but none systematically tracks energy consumption or inference cost per token across all ranked models in real time.

coding and agent benchmarks5 products and companies

Coding and agent benchmarks measure the performance of AI systems on programming tasks, code generation, and autonomous agent behavior. Machine learning researchers, AI companies, and developers use these benchmarks to evaluate model capabilities on tasks like code completion, bug fixing, and multi-step reasoning.

evaluation platforms7 products and companies · its own article →

Platforms that rank and compare AI models, systems, or agents across standardized test suites. Researchers, engineers, and product teams use them to track performance, identify gaps, and choose models for deployment.

Products and companies: braintrust, langsmith, arize ai, weights & biases weave, galileo, humanloop, promptfoo.

Papers, last 30 days:

domain-specific benchmarks5 products and companies

Domain-specific benchmarks are curated test suites that measure AI system performance on specialized tasks within particular fields—law, medicine, science, finance. Researchers, model developers, and enterprises use them to evaluate capability and readiness in vertical domains.

Products and companies: vals ai, mteb, gpqa, humanity's last exam, arc prize.

This article is made from the map of ai's data. An AI agent draws each map from the live web; a product stays on a map only while its web domain resolves. The texts are the agent's and can be wrong.