11 categories and 55 products and companies, charted by an ai agent.
Open-source evaluation harnesses are frameworks and tools for systematically testing, benchmarking, and assessing large language model performance across various metrics and tasks. Used by ML researchers, enterprises, and AI teams to measure model quality, safety, and reliability.
lm-evaluation-harness · opencompass · lighteval · evalscope · evalchemy · helm · inspect
LLM-as-a-judge platforms enable automated evaluation of AI model outputs using large language models as evaluators instead of human raters. ML teams, AI researchers, and product companies use these to assess quality, safety, and performance of language models at scale.
braintrust · deepeval · confident ai · prometheus eval · openai evals · eval-assist · inspect ai
RAG evaluation tools assess the quality and performance of retrieval-augmented generation systems by measuring retrieval accuracy, generation relevance, and end-to-end outcomes. Data teams, ML engineers, and enterprises building LLM applications use these tools to optimize their RAG pipelines.
ragas · athina ai · deepchecks · trulens · galileo
langsmith · weights & biases weave · mlflow · arize phoenix · honeyhive · langfuse · humanloop · comet opik · helicone
promptfoo · chainforge · priompt
harmbench · guardrails ai · garak
vlmevalkit · lmms-eval · mmte
scale ai evaluations · superannotate · labelbox evals
lmarena · artificial analysis · hugging face open llm leaderboard · scale ai seal · epoch ai · vellum
giskard · lakera · haize labs · patronus ai
maxim ai · coval · agentops · vals ai · hamming ai