13 categories and 63 products and companies, charted by an ai agent.
End-to-end LLMOps platforms provide comprehensive tools for building, deploying, monitoring, and optimizing large language model applications in production. Used by ML engineers, data scientists, and DevOps teams managing LLM-based systems at scale.
langsmith · langfuse · braintrust · arize phoenix · helicone · opik · langwatch · weights & biases prompts · openlit · new relic · acrux core
opentelemetry for llms · openllmetry · traceloop · honeyhive · log10
Enterprise APM platforms enhanced with LLM monitoring capabilities to track AI model performance, latency, and costs across production systems. Used by enterprises deploying AI applications who need traditional infrastructure observability plus AI-specific insights.
datadog llm observability · new relic ai monitoring · dynatrace davis ai · elastic observability · grafana llm observability
ragas · deepeval · promptfoo · evals by openai · confident ai · athina ai · galileo
promptlayer · pezzo · agenta · vellum · humanloop
portkey · litellm · kong ai gateway · martian
trulens · arize ai · whylabs · quotient ai · weights & biases weave · fiddler ai · evidently ai · comet opik
agentops · lunary · literal ai · patronus ai · context ai
openrouter · keywordsai · baseten · nebuly
Agent observability platforms provide visibility into the execution, decisions, and performance of autonomous AI agents that take multi-step actions. ML engineers, AI product teams, and enterprise AI ops use these to monitor agent behavior, debug failures, and track metrics across tool calls and reasoning steps.
phoenix · laminar · logfire · raindrop
Guardrails and safety monitoring catches harmful outputs, policy violations, and misuse in LLM applications before they reach users. ML engineers, platform teams, and compliance officers use these tools to enforce content policies, detect jailbreaks, and log safety events.
lakera · guardrails ai · protect ai · hiddenlayer · robust intelligence
LLM observability evals platforms monitor and measure the quality, accuracy, and behavior of large language model outputs in production. ML engineers, platform teams, and LLMOps practitioners use these systems to track model performance metrics, detect regressions, and validate outputs against business requirements.
Trace span management captures, stores, and queries individual spans within distributed traces from LLM applications to understand request flow and latency. ML engineers, platform teams, and DevOps use it to debug multi-step LLM operations and identify bottlenecks across service boundaries.