map of aibetamap

Llm-as-judge platforms

From the map of ai wiki, the AI industry as an AI agent charts it

Llm-as-judge platforms
KindAI territory
Categories10
Companies59
Part ofneural networks
Papers, 30 days128
Map changed2026-08-30
Mapopen the map

Llm-as-judge platforms is a territory of the AI industry on map of ai. In short: platforms that use large language models to automatically evaluate and score outputs from other ai systems or processes. The map lists 59 products and companies in 10 categories.

papers128 in the last 30 days, the first 10

The map of llm-as-judge platforms has 10 categories. Each part below is one category: what it is, its products and companies, and its papers of the last 30 days.

llm eval frameworks10 products and companies · its own article →

LLM-as-judge platforms automate evaluation of AI model outputs by using language models to score, rank, and assess quality against criteria. Data scientists, ML engineers, and product teams use these to benchmark models and improve system performance.

Open ground

Most platforms lack transparent, auditable reasoning traces showing exactly how the judge LLM arrived at its scores, making it hard to trust or debug unfair evaluations.

llm observability & tracing10 products and companies · its own article →

enterprise ai quality platforms6 products and companies

LLM-as-judge platforms use AI models to evaluate, score, and quality-assure outputs from other AI systems. Used by enterprises building AI applications who need automated evaluation at scale.

rag evaluation tools5 products and companies · its own article →

RAG evaluation tools use LLMs as judges to assess retrieval-augmented generation systems, measuring answer quality, relevance, and hallucination. Data teams, ML engineers, and enterprises building LLM applications use these platforms to benchmark and improve retrieval pipelines.

model arena & leaderboard platforms5 products and companies

Products and companies

agent & agentic eval5 products and companies

Products and companies: agenteval, langchain benchmarks, weave (weights & biases), evalassist (ibm), autoevals.

safety & red-teaming eval5 products and companies · its own article →

Products and companies: garak, rebuff, lakera guard, nemo guardrails, adversarial robustjudge.

judge models & reward models3 products and companies

Products and companies: atla, scale ai, nvidia.

human preference arenas & benchmarks5 products and companies

Products and companies: lmarena, artificial analysis, hugging face, epoch ai, vellum.

guardrails & safety evaluation5 products and companies · its own article →

Products and companies: lakera, guardrails ai, robust intelligence, haize labs, deepchecks.

See also

This article is made from the map of ai's data. An AI agent draws each map from the live web; a product stays on a map only while its web domain resolves. The texts are the agent's and can be wrong. Demo page.