OpenLLMetry: Open-Source LLM Observability Built on OpenTelemetry

The Problem: Every LLM Provider Speaks a Different Language

Traditional services had this problem in 2012. Every database driver, HTTP client, and message queue emitted logs in its own format. Teams wrote custom parsers. Dashboards broke when libraries updated. OpenTelemetry fixed it by defining one standard for traces, metrics, and logs across all services.

LLM applications have the same problem today, but worse. A single application might call OpenAI for chat completions, Anthropic for tool use, Cohere for embeddings, and Pinecone for vector search. Each SDK returns different field names for the same concepts. Token counts, latencies, model identifiers, and cost data all live in different shapes.

OpenLLMetry applies the OpenTelemetry model to this problem. One instrumentation layer, one trace format, any backend. Your application calls Traceloop.init() and every LLM interaction becomes a standard OpenTelemetry span with consistent attributes.

How OpenLLMetry Works

OpenLLMetry is a collection of OpenTelemetry instrumentation packages. Each package wraps a specific LLM provider or framework SDK, intercepting calls to capture spans, attributes, and events. The traceloop-sdk initializes all installed instrumentations automatically.

When your code calls openai.chat.completions.create(), the OpenAI instrumentation intercepts the request and creates an OpenTelemetry span with:

• Model name and version (e.g., gpt-5.4)
• Token counts: input tokens, output tokens, total tokens
• Latency: time to first token, total duration
• Prompts and completions (optionally, with privacy controls)
• Cost: calculated from token counts and model pricing
• Error details: rate limits, API failures, timeouts

These spans follow OpenTelemetry's gen_ai semantic conventions, so any OTel-compatible backend can parse them without custom configuration.

Setup: Two Lines of Code

Python

pip install traceloop-sdk

from traceloop.sdk import Traceloop

Traceloop.init(app_name="my_llm_app")

# Your existing code works unchanged
from openai import OpenAI
client = OpenAI()
response = client.chat.completions.create(
    model="gpt-5.4",
    messages=[{"role": "user", "content": "Hello"}]
)
# ^ This call is now automatically traced

TypeScript / Node.js

npm install @traceloop/node-server-sdk

import * as traceloop from "@traceloop/node-server-sdk";

traceloop.initialize({ appName: "my_llm_app" });

// All OpenAI, Anthropic, etc. calls are now traced

SDKs also exist for Go (go-openllmetry) and Ruby (openllmetry-ruby). All emit standard OTLP, so the same backend configuration works regardless of language.

Exporting to Your Backend

By default, traces export to Traceloop's hosted platform. To send to your own backend, set the standard OpenTelemetry environment variables:

export OTEL_EXPORTER_OTLP_ENDPOINT="https://your-collector:4318"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer your-token"

# Or in code:
# Traceloop.init(exporter=your_custom_exporter)

Supported Providers and Frameworks

OpenLLMetry ships 40+ instrumentation packages. Each is a separate pip/npm package that the SDK auto-detects and initializes.

LLM Providers (16)

Frameworks (10+)

Vector Databases (7)

Chroma, Pinecone, Qdrant, Weaviate, Milvus, LanceDB, and Marqo. Each instrumentation captures query vectors, result counts, similarity scores, and latency.

Architecture: Instrumentation, Not Platform

OpenLLMetry sits between your application code and your observability backend. It does not store data. It does not provide a UI. It collects and exports.

Your App (OpenAI, Anthropic, LangChain, etc.)
    |
    v
OpenLLMetry Instrumentations (auto-patched at import)
    |
    v
OpenTelemetry SDK (spans, metrics, events)
    |
    v
OTLP Exporter
    |
    v
Any Backend: Datadog, Grafana Tempo, Honeycomb,
Jaeger, SigNoz, New Relic, Langfuse, Traceloop...

This architecture means OpenLLMetry is not competing with observability platforms. It feeds them. You can use OpenLLMetry with Langfuse, with Datadog, with a self-hosted Jaeger instance, or with all three simultaneously using OpenTelemetry's fan-out exporters.

OpenLLMetry vs Langfuse vs LangSmith vs Helicone

These tools occupy different layers of the LLM observability stack. OpenLLMetry is an instrumentation library. Langfuse, LangSmith, and Helicone are observability platforms with UIs, storage, and analysis tools.

When Each Tool Fits

Traceloop: From YC to ServiceNow

Traceloop was founded in 2023 by Nir Gazit (former chief architect at Fiverr, ML team lead at Google) and Gal Kleinman (senior data engineering at Fiverr). The company graduated from Y Combinator and raised $6.1M from Samsung NEXT, Ibex Investors, and Grand Ventures.

On March 2, 2026, ServiceNow acquired Traceloop for an estimated $60-80M. This was ServiceNow's third Israeli acquisition in months, following the $7.75B Armis Security deal. At acquisition, Traceloop's customer list included IBM, HiBob, Miro, and Dynatrace.

The OpenLLMetry project is Apache 2.0 licensed and continues independently of the acquisition. The open-source codebase has 105 contributors and 1,327 commits. Whether ServiceNow maintains the same investment in the open-source project remains an open question, but the Apache 2.0 license ensures the community can fork and maintain it regardless.

OpenTelemetry GenAI Semantic Conventions

OpenLLMetry is converging with a broader industry effort. The OpenTelemetry Generative AI Observability SIG (started April 2024) is defining official semantic conventions for LLM telemetry under the gen_ai.* namespace. These conventions standardize attribute names for prompts, completions, token counts, model identifiers, and tool calls across all providers.

Datadog already natively supports these conventions (OTel v1.37+). As the standard matures, OpenLLMetry's instrumentation packages are aligning their output to match, meaning traces emitted by OpenLLMetry will be parseable by any backend that implements the OTel GenAI spec.

Traceloop proposed donating OpenLLMetry's instrumentation code to the OpenTelemetry project directly. If accepted, OpenLLMetry's instrumentations would become official OTel packages, giving them the same maintenance guarantees as OTel's HTTP, gRPC, and database instrumentations.

When to Use OpenLLMetry

When to Look Elsewhere

• You want a turnkey platform: OpenLLMetry provides no UI, no storage, no evaluation tools. If you need those, use Langfuse or LangSmith directly.
• Proxy-based simplicity: Helicone requires zero code changes, just a URL swap. If you want the absolute minimum integration effort, a proxy approach is simpler than SDK instrumentation.
• LangChain-only stack: LangSmith's LangChain integration is deeper than OpenLLMetry's. If LangChain is your entire stack, LangSmith may provide better debugging.

Frequently Asked Questions

What is OpenLLMetry?

An open-source instrumentation library that extends OpenTelemetry to LLM applications. Built by Traceloop (acquired by ServiceNow, March 2026), it auto-instruments 40+ providers and frameworks including OpenAI, Anthropic, LangChain, and LlamaIndex. Apache 2.0 license, 6.9k GitHub stars.

How do I set up OpenLLMetry?

Python: pip install traceloop-sdk, then add Traceloop.init() to your entry point. TypeScript: npm install @traceloop/node-server-sdk and call traceloop.initialize(). Auto-instrumentation starts immediately. No code changes to your LLM calls.

Is OpenLLMetry free?

The library is free and Apache 2.0. Traceloop's hosted platform has a free tier (50K spans/month, 24-hour retention). You can also export to any free, self-hosted backend like Jaeger, Zipkin, or SigNoz with no cost.

How does OpenLLMetry compare to Langfuse?

Different layers. OpenLLMetry is instrumentation (collects traces). Langfuse is a platform (stores, visualizes, evaluates). They work together: OpenLLMetry instruments your app, Langfuse receives the OTLP data. Use OpenLLMetry alone if you already have Datadog or Grafana. Use Langfuse if you need prompt management and evaluation tools.

What happened to Traceloop?

ServiceNow acquired Traceloop on March 2, 2026 for an estimated $60-80M. The OpenLLMetry open-source project continues under Apache 2.0. Traceloop's customers included IBM, Miro, and Dynatrace. The acquisition was ServiceNow's third Israeli deal in months, following the $7.75B Armis purchase.

Related

• Agent Trace: The Open Spec for AI Code Attribution
• AI Agent Orchestration Patterns
• Multi-Agent Orchestration