AI Infrastructure

Best LLM Observability Tools in 2026: 12 Monitoring Platforms Ranked and Compared

The best LLM observability and LLM monitoring tools in 2026, checked against their own docs, pricing pages and repositories: Langfuse, LangSmith, Arize Phoenix, Opik, Datadog, Grafana, Braintrust, MLflow, W&B Weave, OpenLLMetry, OpenLIT and Helicone, ranked by use case and compared on self-hosting, licence, pricing model, OpenTelemetry, evals, cost tracking and PII handling. Verified October 2026.

The best LLM observability tools in 2026 are Langfuse for an open-source all-rounder, LangSmith for LangChain and LangGraph teams, Arize Phoenix and Opik for self-hosted tracing with evaluations, Datadog and Grafana Cloud for LLM monitoring inside an existing observability stack, and Braintrust for evaluation-led release gates. All of them trace prompts, model and tool calls, latency, tokens and cost; they differ on self-hosting, licence and what they bill for. Instrument with OpenTelemetry and the choice stays reversible.

Best LLM observability tools in 2026, ranked

Ranked for a team starting fresh in late 2026, on breadth of tracing, evaluation and cost features, self-hosting, licence and vendor status. Your existing stack can reorder the list: a company that already pays for Datadog should start at Datadog. Prices are list prices from each vendor’s pricing page (verified October 2026).

RankToolBest forLicenceSelf-hostPricing model
1LangfuseOpen-source all-rounder: tracing, evals, prompt managementMIT coreYes, freeCloud Hobby free to 50k units a month; Core $29, Pro $199, Enterprise $2,499 a month; $8 per extra 100k units
2LangSmithLangChain and LangGraph teamsCommercialEnterprise add-onDeveloper $0 with 5k base traces; Plus $39 per seat with 10k, then pay-as-you-go
3Arize PhoenixOpenTelemetry-native tracing and evals on your own serversElastic License 2.0Yes, freeFree to self-host; Arize AX is the commercial platform
4Opik (Comet)Fully open-source tracing, evals and guardrailsApache 2.0Yes, freeFree to self-host, backend included
5DatadogLLM monitoring for teams already on DatadogCommercialNoPer LLM span: free to 40k a month; Pro $160 a month for 100k
6Grafana Cloud AI ObservabilityPrometheus and OpenTelemetry shops; GPU and MCP monitoring tooCommercial service; OpenLIT Apache 2.0Through OpenLITGrafana Cloud subscription, or free self-hosted OpenLIT
7BraintrustEvaluation-led teams that gate releases on experimentsCommercialEnterprise onlyStarter free (1 GB processed data, 14-day retention); Pro $249 a month (5 GB, 30 days)
8MLflow TracingTeams already running MLflowApache 2.0Yes, freeFree
9W&B WeaveWeights & Biases and CoreWeave usersSDK Apache 2.0Self-managed optionNot checked for this list
10OpenLITSelf-hosted LLM and GPU monitoring in one stackApache 2.0Yes, freeFree
11OpenLLMetryVendor-neutral instrumentation feeding any backendApache 2.0Not a backendFree
12HeliconeExisting users onlyApache 2.0YesIn maintenance mode since March 2026

Langfuse bills every tracing data point it receives as a unit: traces, spans, events, generations and scores. Datadog bills LLM spans and Braintrust bills gigabytes processed plus evaluation scores. An agent run that makes ten model and tool calls is one trace but many units or spans, so price each option on a week of your own traffic before comparing headline numbers.

What LLM observability covers

Application monitoring records requests, errors and infrastructure metrics. LLM observability adds what a model-driven system needs on top: the full trace of a request through prompts, model calls, retrievals and tool calls, token usage and cost at each step, evaluation scores, and user feedback. For an agent, that trace becomes a multi-step trajectory, which our explainer on agent observability covers as a concept. This page is the tools list.

The same records often double as audit evidence, which raises the bar on retention, access and privacy. The layers an agent platform needs on top of tracing, from structured logs to tamper-evident audit trails, are set out in our piece on logs, traces and audit records for agents.

LLM monitoring vs LLM observability

People search for an “LLM monitoring tool” and an “LLM observability platform” as if they were different products. They are mostly the same products used in two ways.

LLM monitoringLLM observability
Question it answersIs the application healthy right now?Why did this request behave the way it did?
Data it usesAggregates: latency percentiles, error rate, tokens, cost, average evaluator scoreFull per-request traces: prompts, retrievals, tool calls, model responses and scores
What it producesDashboards, alerts, budgets and service-level objectivesA trace explorer, session views and datasets built from failed requests
Usual ownerPlatform or SRE teamThe engineers who build the application
Strongest tools hereDatadog, Grafana Cloud, OpenLITLangfuse, LangSmith, Phoenix, Opik, Braintrust

Monitoring tells you something is wrong; observability tells you why. A cost alert fires because daily spend on one feature doubled, and the trace view then shows a retrieval step that started returning twenty chunks instead of five. Pick a tool that does the half your team lacks today, and make sure it can do the other half well enough.

These are the alerts worth setting up first for any LLM application:

  • Latency. p95 end-to-end duration and time to first token, per model and per provider.
  • Errors. Failure, timeout, retry and fallback rates, per provider.
  • Spend. Cost per day per feature or team against a budget, with an alert well before the limit.
  • Context growth. Input tokens per conversation turn, which climbs quietly as history and retrieved context pile up.
  • Quality. A drop in the rolling average of an online evaluator score or a rise in negative user feedback.
  • Guardrails. A spike in blocked prompts or flagged outputs, often the first sign of misuse.
  • Agent loops. Model and tool calls per agent run above a ceiling, the usual signature of a loop.

The metrics to collect

The OpenTelemetry GenAI semantic conventions, now maintained in their own repository, give standard names for most of these signals. Every metric and attribute below is marked Development, so names can still change between releases; instrumentations can opt into the newest experimental names with OTEL_SEMCONV_STABILITY_OPT_IN=gen_ai_latest_experimental (verified October 2026).

SignalWhat to recordOpenTelemetry GenAI nameWhy it matters
End-to-end latencyDuration of each model callgen_ai.client.operation.durationSpots slow models, providers and prompts
Time to first tokenFirst chunk seen by the client, first token produced by the servergen_ai.client.operation.time_to_first_chunk, gen_ai.server.time_to_first_tokenPerceived speed in chat interfaces
Generation speedTime per output token or chunkgen_ai.server.time_per_output_token, gen_ai.client.operation.time_per_output_chunkCapacity planning for self-hosted models
TokensInput and output, plus cache reads and reasoning tokensgen_ai.client.inference.usage.input_tokens, …output_tokens, …cache_read.input_tokens, …reasoning.output_tokensCost and context growth
Model identityRequested model, responding model, providergen_ai.request.model, gen_ai.response.model, gen_ai.provider.nameCatches silent model swaps; audit
Agent and tool stepsTool durations, tool calls and model calls per agent rungen_ai.execute_tool.duration, gen_ai.invoke_agent.tool_calls, gen_ai.invoke_agent.inference_callsFinds loops and slow tools
CostTokens multiplied by a price table per modelNot defined in the metric documents; computed by each toolBudgets per team, feature or user
QualityEvaluator scores, user feedback, guardrail hitsNot defined in the metric documentsRegressions that latency charts miss
ErrorsFailures, timeouts, retries, fallbacksError attributes on spansProvider reliability

One more rule from the span conventions matters for privacy. The attributes gen_ai.input.messages and gen_ai.output.messages carry the conversation itself, and the specification says instrumentations should not capture them by default because they are likely to contain personal data; capture should sit behind an explicit opt-in.

LLM observability tools compared

Each row reflects the project’s own documentation or repository, verified October 2026. A dash means the feature was not documented on the pages we checked.

ToolSelf-hostableLicenceOpenTelemetryEvalsCost trackingPII handling
LangfuseYesMIT core; SCIM, audit logging and retention modules licensed separatelyOTLP over HTTP; SDK v4 built on OpenTelemetryYesIngested, or inferred from model pricesMasking in the SDK before export
Arize PhoenixYes, on Docker or KubernetesElastic License 2.0Built on OpenTelemetry and OpenInferenceLLM, code and human evaluatorsBuilt-in price table, custom pricesHide settings in the instrumentation
OpenLLMetryInstrumentation; send to any OTel backendApache 2.0Native––Content captured by default; can be switched off
MLflow TracingYesApache 2.0Exports and ingests the GenAI conventionsLLM-judge scorers, human feedbackFrom MLflow 3.10, using LiteLLM’s price tablePII redaction from traces
LangSmithEnterprise add-on onlyCommercialOTLP ingest and SDK exportYes–Client-side hiding and anonymisers
Datadog Agent ObservabilityNo, SaaSCommercialGenAI conventions supported nativelyExperiments and quality checksCost dashboardsSensitive Data Scanner redaction
Grafana Cloud AI ObservabilityGrafana Cloud; OpenLIT UI can be self-hostedCommercial service; OpenLIT Apache 2.0OpenTelemetry-nativeHallucination and toxicity scoringSpend tracking and budgets–
HeliconeYes, with Docker or HelmApache 2.0AI gateway supports OpenTelemetryYesYesHeaders that skip storing bodies
W&B WeaveMulti-tenant, dedicated or self-managedSDK Apache 2.0OTLP endpointLLM judges and custom scorers–Presidio redaction in the Python SDK
OpikYes, with Docker Compose or HelmApache 2.0, backend includedFirst-party OpenTelemetry ingestDatasets, experiments, LLM-judge metrics, online rulesToken usage over time–
OpenLITYes, with Docker Compose or KubernetesApache 2.0Native; GenAI attribute namesLLM-judge, programmatic and human evalsCustom prices for fine-tuned modelsGuardrails for sensitive topics
BraintrustEnterprise plan onlyCommercialOTLP endpoint and span processorsExperiments and scorers––

Open-source and self-hostable tools

Langfuse

Langfuse covers tracing, evaluations, prompt management, experiments, annotation and a playground under the MIT licence, and had more than 35,000 GitHub stars in October 2026. When you self-host, SCIM, audit logging and data-retention policies sit in separately licensed enterprise modules. It accepts OpenTelemetry traces over OTLP HTTP, though not gRPC, maps GenAI, OpenInference and MLflow attributes, and its v4 SDK is a thin layer over the OpenTelemetry client. Cost comes from the values you send or is inferred from model price definitions, including your own for self-hosted or fine-tuned models. Masking runs in the SDK before data leaves your application, and there is no server-side redaction. ClickHouse acquired Langfuse in January 2026 and committed to keeping the core MIT licensed and self-hostable at production scale.

Arize Phoenix

Phoenix is built on OpenTelemetry and OpenInference instrumentation, runs on Docker or Kubernetes, and supports LLM-based, code-based and human evaluations attached to datasets and experiments. Its licence is the Elastic License 2.0, which is source-available rather than OSI-approved and limits offering the software as a managed service, so check it against your plans. Cost tracking uses a built-in price table, with custom prices for other models and for cache, audio, reasoning and image tokens. Masking is set in the instrumentation through OPENINFERENCE_HIDE_INPUTS, OPENINFERENCE_HIDE_OUTPUTS and related settings, so hidden values never leave the process. Arize AX is the managed commercial platform built on the same standards.

OpenLLMetry and the OpenTelemetry conventions

OpenLLMetry, maintained by Traceloop under Apache 2.0, is instrumentation rather than a backend. It extends OpenTelemetry so traces from LLM frameworks reach any compatible destination, and its README lists more than 20, including Datadog, Grafana, Honeycomb, New Relic, Splunk, Dynatrace and Sentry. By default it records prompts, completions and embeddings as span attributes; setting TRACELOOP_TRACE_CONTENT=false turns that off globally, and it can also be disabled per workflow. Traceloop joined ServiceNow in March 2026 and said OpenLLMetry will remain open source. Because its conventions have been folded into OpenTelemetry itself, instrumenting this way keeps the backend replaceable.

MLflow Tracing

MLflow, licensed Apache 2.0, describes its tracing as fully OpenTelemetry-compatible: it exports and ingests the GenAI conventions, and traces stay on infrastructure you host. Token usage is recorded from MLflow 3.2 and cost from 3.10, when the tracking server is installed with the [genai] extra. Cost relies on LiteLLM’s price table, so a model missing from it shows $0.00 even with correct token counts. Evaluation runs through datasets, scorers and built-in LLM-as-a-judge scorers, and human feedback is attached to traces with its metadata. It is the natural pick for teams already using MLflow for classic models.

Helicone

Helicone pairs an AI gateway with an observability platform, licensed Apache 2.0, that can be self-hosted with Docker or Helm. Mintlify acquired Helicone on 3 March 2026, and Helicone now runs in maintenance mode: security updates, bug fixes and new models keep shipping, and Mintlify says it will help customers migrate to another platform. Its omit headers stop request and response bodies from being stored but, as its documentation notes, the content is still sent to Helicone’s backend. Plan new deployments on another tool.

Opik

Opik, built by Comet, is Apache 2.0 for the whole platform, backend included, and self-hosts with a Docker Compose script for local use or a Helm chart on Kubernetes. It traces nested calls through a @track decorator and a long list of framework integrations, and accepts OpenTelemetry traces from any language with an OTel SDK, including Java, Ruby and .NET. Evaluation covers datasets, experiments, LLM-as-a-judge metrics, a PyTest integration for CI and online evaluation rules on production traffic, and guardrails can be switched on at install. Its README shows token usage over time but does not describe cost tracking, so test that against your needs (verified October 2026).

OpenLIT

OpenLIT is the open-source project behind Grafana Cloud’s AI observability, and it works on its own. Apache 2.0 SDKs for Python, TypeScript and Go auto-instrument more than 70 integrations using the GenAI attribute names, an OTLP receiver accepts gRPC and HTTP, and a self-hosted dashboard stores telemetry in ClickHouse. It adds GPU monitoring for NVIDIA, AMD and Intel cards, evaluations, guardrails against prompt injection and sensitive topics, custom prices for fine-tuned models, a versioned prompt hub and a vault for provider keys (verified October 2026). For teams running their own GPUs, it is the closest thing to one self-hosted LLM monitoring stack.

Commercial platforms

LangSmith

LangSmith, from LangChain, is a hosted platform for tracing, evaluation and prompt engineering. Developer seats cost $0 with 5,000 base traces a month and Plus costs $39 per seat with 10,000, then pay-as-you-go; Enterprise is custom (list, verified October 2026). Base traces are kept for 14 days and extended traces for 180. Self-hosting on Kubernetes, with ClickHouse, PostgreSQL and Redis inside your environment, is an add-on to the Enterprise plan. LangSmith ingests OpenTelemetry traces and maps GenAI attributes to its own fields, and masking happens on the client through LANGSMITH_HIDE_INPUTS, LANGSMITH_HIDE_OUTPUTS, regex anonymisers or integrations such as Presidio.

Datadog Agent Observability

Datadog now uses the name Agent Observability on both its product page and its documentation, although the documentation still lives under the llm_observability path. It traces LLM calls, fixed workflows and agent steps, compares prompts, models and agent configurations in experiments, tracks token cost, and uses Sensitive Data Scanner to redact sensitive data and flag prompt-injection attempts. The documentation says it supports the OpenTelemetry GenAI conventions natively. Pricing is per LLM span: free up to 40,000 spans a month, and Pro at $160 a month for 100,000 (list, verified October 2026). It is SaaS only, so trace content leaves your environment.

Grafana Cloud AI Observability

Grafana Cloud’s AI observability is OpenTelemetry-native and built with OpenLIT. It covers LLMs, vector databases, GPUs, MCP servers and agents, with spend tracking, budget management and automated checks for hallucinations, factual accuracy and toxicity. The OpenLIT SDK and its self-hosted dashboard on ClickHouse are Apache 2.0, so the same instrumentation can stay inside your network if Grafana Cloud is not an option.

W&B Weave

Weave traces LLM calls and functions and evaluates responses with LLM judges and custom scorers. It accepts OTLP traces on multi-tenant cloud, dedicated cloud and self-managed instances, and its Python SDK can redact personal data with Presidio before traces leave the client; other SDKs lack that option. Its documentation now lives on CoreWeave’s site and names CoreWeave Agent Lens, in public preview on multi-tenant SaaS, as the successor to Weave. The two share a backend, and Weave still covers evaluations and guardrails that Agent Lens does not yet offer.

Braintrust

Braintrust is an evaluation-first platform: production traces feed datasets, experiments and scorers that teams use to decide whether a prompt or model change ships. It takes OpenTelemetry traces through an OTLP endpoint and span processors for Python, TypeScript and Go. It bills processed data and evaluation scores rather than spans: Starter is free with 1 GB of processed data, 10,000 scores and 14-day retention; Pro is $249 a month with 5 GB, 50,000 scores and 30-day retention, extendable to 180 days at extra cost; Enterprise is custom and is the only plan offering on-premises or hosted deployment (list, verified October 2026).

Helicone vs Grafana for LLM observability

Is Helicone a better LLM observability tool than Grafana overall? For anything you deploy from now on, no. The two were never quite the same kind of product, and Helicone’s status settles the question.

HeliconeGrafana Cloud AI Observability
How data arrivesMainly through its AI gateway in front of model providersOpenTelemetry, with the OpenLIT SDK as the instrumentation
ScopeModel requests passing through the gatewayLLMs, agents, vector databases, MCP servers and GPUs
Next to other telemetrySeparate productSame place as your existing metrics, logs and traces
EvaluationsYesQuality and safety scoring
Self-hostingDocker or HelmThrough self-hosted OpenLIT; Grafana Cloud is a service
Status in October 2026Maintenance mode since 3 March 2026Actively documented product

Helicone’s appeal was speed: change the base URL of your model client and every request is logged with its cost. Since Mintlify bought it, Helicone ships security fixes, bug fixes and new models, but Mintlify says it will help customers move to another platform. Existing users can keep running it while they plan a migration; new users should not start there. If gateway-style logging is what you liked, an AI gateway that writes per-call records, paired with OpenTelemetry tracing, gives the same view without the dependency.

Grafana vs LangSmith for tracing LLM applications

The two answer different questions, which is why the comparison comes up so often.

  • Grafana Cloud treats an LLM call as one more OpenTelemetry span. Traces sit beside the latency, GPU and database telemetry your platform team already watches, with spend tracking, budgets and quality and safety scoring on top. Its AI observability documentation does not mention prompt management, datasets or experiments.
  • LangSmith is organised around the engineer improving the application: a trace tree for every run, datasets built from production examples, evaluations and prompt engineering. It ingests OpenTelemetry and maps GenAI attributes to its own fields, but it is SaaS by default, with self-hosting as an Enterprise add-on and base traces kept for 14 days.

Choose Grafana when the main need is LLM monitoring and correlation with infrastructure. Choose LangSmith when the main need is debugging and evaluating prompts and agent logic, especially on LangChain or LangGraph. Many teams run both: instrument once with OpenTelemetry, then let an OpenTelemetry Collector export the same spans to each backend, with redaction applied before either sees them.

Keeping prompts and personal data out of traces

Traces of an LLM application hold the most sensitive text the system ever sees, so treat the observability store like any other system of record:

  • Capture content only by decision. The GenAI conventions make message capture opt-in; keep it that way unless a team needs content to debug.
  • Mask at the source. Langfuse masking functions, LangSmith hide settings, OpenInference hide settings in Phoenix and Weave’s PII redaction all act before data leaves the process.
  • Redact in the pipeline. The OpenTelemetry Collector’s redaction processor deletes attributes that are not on an allowlist and masks values matching blocked patterns; it is beta for traces and alpha for logs and metrics.
  • Read what “omit” means. Helicone’s omit headers prevent storage, not transmission.
  • Set retention deliberately. LangSmith keeps base traces for 14 days; in self-hosted Langfuse, retention policies sit in the licensed enterprise modules.
  • Control access and export. Traces used as audit evidence need role-based access and a route into your SIEM.

A minimal LLM monitoring setup in five steps

Whichever platform you pick, the same five steps get a production application from no visibility to useful alerts, and they keep the backend replaceable.

  1. Instrument with OpenTelemetry. Use OpenLLMetry, OpenLIT or a vendor SDK built on OpenTelemetry, and leave prompt and response capture off until a team asks for it with a reason.
  2. Put a Collector in the path. Run the OpenTelemetry Collector between the application and the backend, with the redaction processor and an allowlist of attributes. It also lets you send the same spans to two backends during a trial.
  3. Make cost accurate. Load prices for every model you call, including self-hosted and fine-tuned ones. Langfuse, Phoenix and OpenLIT accept custom prices; MLflow relies on LiteLLM’s price table, so a missing model reports $0.00.
  4. Create the first dashboards and alerts. Start with the seven signals in the monitoring section above, split by model, feature and team.
  5. Close the loop with evaluations. Sample production traces into online evaluators, turn failures into a dataset, and run that dataset against every prompt or model change before release. Our list of LLM evaluation tools covers the evaluation side in depth.

How to choose an LLM observability tool

  1. Decide where trace data may live. If prompts contain regulated data, start with self-hostable tools or with SaaS whose masking runs in your process.
  2. Instrument with OpenTelemetry. The backend then becomes replaceable. Check which version of the GenAI conventions each candidate expects.
  3. Read licence and tier labels. Langfuse’s enterprise modules, Phoenix’s Elastic License and LangSmith’s Enterprise-only self-hosting all change the total cost.
  4. Check vendor status. Helicone is in maintenance mode, Weave has a named successor, and Langfuse and Traceloop now belong to larger companies.
  5. Price on your own unit. Spans, traces and seats scale differently, and an agent run can produce dozens of spans.
  6. Add gateway logs. If model traffic passes through an AI gateway, its per-call records are a second source of cost and latency data; our AI gateway comparison notes what each one logs.
  7. Tie evaluation to traces. Use offline test sets before release and online scores after, following an agent evaluation method and the practice of testing agents before production.

How VDF AI fits

For workflows that run on VDF AI Networks, observability is part of the platform. Every node execution is tracked with its status, input and output, token usage and cost. Latency is reported at P50 and P95 per network and per node, service-level objectives can be set per network with alerts on breaches, and execution traces include model routing decisions and their rationale.

Across the platform, the Trust Center describes audit records for every prompt, tool call, retrieval, model route and response, with actor, timestamp and context, streamed to your SIEM and kept in your environment, whether on-premises, in a private cloud or air-gapped. VDF AI Router adds the reason, candidate list and scores behind each routing decision. The product pages do not document an OpenTelemetry export, so if you standardise on one of the tools above, confirm the integration path during evaluation.

Sources

Frequently asked questions

What are LLM observability tools?

LLM observability tools capture what an application built on language models did for each request: the prompt and response, every model call, retrieval and tool call, latency, token counts, cost, errors and evaluation scores. They combine traces, metrics and quality signals so engineers can debug failures, find slow or expensive steps and catch quality regressions. Most now accept OpenTelemetry data, and many add prompt management, datasets and automated evaluators on top of tracing.

What is the best open-source LLM observability tool?

It depends on the licence and how you work. Langfuse is MIT licensed for its core features and self-hosts at scale, with SCIM, audit logging and retention policies in paid modules. MLflow Tracing is Apache 2.0 and suits teams already on MLflow. Arize Phoenix is source-available under the Elastic License 2.0 rather than an OSI licence. OpenLLMetry and OpenLIT are Apache 2.0 instrumentation that can feed any OpenTelemetry backend. Helicone's platform is open source but has been in maintenance mode since March 2026.

Which metrics should we track for LLM applications?

Track end-to-end latency and time to first token, generation speed per output token, input, output, cached and reasoning tokens, and cost per call, user and feature. Add error, timeout and retry rates, and for agents the number and duration of tool and model calls in each run. Then add the quality signals that infrastructure metrics miss: evaluator scores, user feedback and guardrail hits. The OpenTelemetry GenAI conventions name most of these signals, though every one is still marked Development.

Do LLM observability tools support OpenTelemetry?

Most do. Langfuse, LangSmith and W&B Weave accept OTLP traces, Phoenix is built on OpenTelemetry and OpenInference, MLflow exports and ingests the GenAI semantic conventions, and Datadog supports those conventions natively. OpenLLMetry and OpenLIT are themselves OpenTelemetry instrumentation. The conventions now live in their own repository and every signal is marked Development, so attribute names can change; check which version each backend expects before standardising.

How do we keep personal data out of LLM traces?

Capture less, then redact. The GenAI conventions make prompt and response content opt-in, and SDK settings in Langfuse, LangSmith, Phoenix and Weave mask or hide content before it leaves the application. An OpenTelemetry Collector running the redaction processor can drop attributes that are not on an allowlist and mask values that match blocked patterns. Keep raw traces for a short time, restrict who can read them, and check whether an omit feature stops only storage or transmission as well.

Is LLM observability different from agent observability?

They overlap. LLM observability grew around single model calls: prompts, responses, latency, tokens and cost. Agent observability extends the same records to multi-step runs, adding tool calls, retrievals, planning steps and decisions, and treats the whole trajectory as the unit to inspect. The tools in this list increasingly cover both, and the OpenTelemetry conventions now define agent and tool spans as well as model spans.

What is the best LLM monitoring tool?

For most teams starting in late 2026, Langfuse is the strongest all-round choice: MIT-licensed core, free to self-host, with tracing, evaluations, prompt management and cost tracking. Teams already running Datadog or Grafana should start with the LLM monitoring built into those platforms, because alerts and dashboards then sit next to the rest of their telemetry. LangSmith suits LangChain and LangGraph shops, and Braintrust suits teams that gate releases on evaluations.

Is Helicone a better LLM observability tool than Grafana overall?

Not for a new deployment. Helicone made logging easy by acting as a proxy in front of model providers, but it has been in maintenance mode since Mintlify acquired it on 3 March 2026: security fixes and new models still ship, while Mintlify helps customers migrate elsewhere. Grafana Cloud AI Observability is OpenTelemetry-native, built with OpenLIT, and covers LLMs, agents, vector databases, MCP servers and GPUs alongside your existing metrics, logs and traces.

How does Grafana compare to LangSmith for tracing LLM applications?

Grafana treats LLM traces as OpenTelemetry data next to infrastructure telemetry, which suits platform and SRE teams who want spend tracking, quality scoring and alerting in one place. LangSmith is built around the AI engineer's loop: a trace tree per run, datasets, evaluations and prompt engineering. Grafana's AI observability documentation does not cover prompt management, datasets or experiments, and LangSmith self-hosting is an Enterprise add-on. Many teams instrument once with OpenTelemetry and send traces to both.

What is the difference between LLM monitoring and LLM observability?

LLM monitoring answers known questions continuously: is latency, error rate, token use, cost or evaluation score drifting past a threshold? It produces dashboards, budgets and alerts. LLM observability lets you answer new questions about a single request after the fact, using the full trace of prompts, retrievals, tool calls and scores. In practice the same tools do both, but monitoring is usually owned by platform teams and observability by the engineers who build the application.

Which LLM observability platforms can be self-hosted for free?

Langfuse (MIT core), Opik (Apache 2.0, backend included), MLflow Tracing (Apache 2.0), OpenLIT (Apache 2.0) and Arize Phoenix (source-available under the Elastic License 2.0) can all be self-hosted without a licence fee. Helicone is self-hostable too but is in maintenance mode. LangSmith and Braintrust offer self-hosted or on-premises deployment only on their Enterprise plans, and Datadog is SaaS only.

Filed under
AI observabilityAI audit trailsAI evaluationAI infrastructureopen-source AIAI governance
AI Governance

Is your AI governance audit-ready?

Get a readiness review of your AI controls — policy, oversight, audit trails, and EU AI Act evidence — mapped against what production actually requires.

Keep reading