10 categories and 58 products and companies, charted by an ai agent.
langfuse · langsmith · arize phoenix · helicone · agentops · maxim ai · opentelemetry genai sig · honeyhive · traceloop · weights & biases weave · opik
LLM evaluation frameworks are tools for systematically testing, benchmarking, and monitoring language model outputs and agent behaviors. Used by ML teams, AI engineers, and enterprises building production LLM applications.
braintrust · confident ai (deepeval) · ragas · promptfoo · evals (openai) · inspect (uk aisi) · eleutherai lm-eval · uptrain · deepeval · openai evals · inspect ai
Enterprise AI observability platforms monitor, evaluate, and debug AI agents in production environments. Used by ML engineers, platform teams, and enterprises deploying autonomous AI systems at scale.
arize ax · galileo · whylabs · comet opik · fiddler ai · truera · aporia · arthur ai · lakera
Infrastructure-native APM for AI systems that provides deep observability into agent behavior, performance metrics, and evaluation frameworks. Used by AI teams, ML engineers, and platform builders to monitor and debug autonomous agents in production.
datadog llm observability · new relic ai monitoring · dynatrace davis ai · grafana llm observability
trulens · ragas · context (arthur ai) · quotient ai
guardrails ai · nemo guardrails (nvidia) · lakera guard · protect ai
scale ai (rlhf eval) · labelbox · argilla · humanloop · haystack (deepset)
giskard · arthur bench · adversa ai
Agent observability evals measure whether AI agents behave as intended by running automated tests that check task completion, decision logic, and state transitions. ML engineers and AI platform teams use these tools to verify agent behavior before deployment.
coval · langwatch · patronus ai · hamming ai
Product analytics for agents captures interaction logs, decision traces, and outcome metrics from autonomous agent systems to identify failure modes and performance bottlenecks. Data teams and AI engineers use it to debug agent behavior, measure task completion rates, and optimize prompt-action sequences.
raindrop · posthog llm analytics · mixpanel