11 categories and 59 products and companies, charted by an ai agent.
langfuse · langsmith · arize phoenix · braintrust · helicone · traceloop · agenta · langwatch · opik · lunary · maxim ai · logfire
datadog llm observability · new relic ai monitoring · dynatrace davis ai · elastic observability · honeycomb · new relic · dynatrace · grafana cloud
ragas · deepeval · promptfoo · confident ai · patronus ai · trulens · galileo · humanloop
opentelemetry (otel) · openinference · openllmetry · mlflow tracing · opentelemetry · openlit
RAG pipeline observability provides visibility and monitoring for retrieval-augmented generation systems, tracking data flow from retrieval through generation stages. Used by ML engineers, data teams, and LLM application builders to debug and optimize RAG performance.
arize ai · whylabs · vectara · ragtrace
AI agent monitoring and LLM observability/tracing tools that track, debug, and analyze the behavior and performance of language model applications and autonomous agents in production. Used by ML engineers, DevOps teams, and AI product teams.
langfuse (agents) · weights & biases (weave) · comet ml · portkey ai · pydantic logfire
helicone (cost) · openrouter · keywordsai
litellm · portkey · martian
langchain hub · pezzo ai · agenta (prompts)
Agent observability tracks the execution, state, and decisions of autonomous AI agents in production. DevOps teams, ML engineers, and application developers use it to debug agent behavior, monitor tool calls, and understand action sequences.
agentops · raindrop · laminar
LLM guardrails and security monitoring detects and blocks harmful outputs, prompt injections, and policy violations in language model applications. ML engineers, platform teams, and security operations use these tools to enforce safety constraints on model behavior.
lakera · protect ai · arthur ai · fiddler ai