TraceLLM
TraceLLM · Coding
TraceLLM is a local-first LLM observability platform for teams running AI applications in production. It records sessions, spans, prompts and outputs (when you allow it), token usage, latency, and errors across chatbots, agents, and RAG pipelines, then shows everything on one trace timeline. A policy layer decides what gets captured, redacted, or dropped, and an OpenTelemetry export path forwards selected traces to backends like SigNoz or Honeycomb. So what does that buy you? A single place to answer why one answer went wrong.
About TraceLLM
What Is TraceLLM
TraceLLM is an observability platform built specifically for AI applications. Instead of scattering clues across provider dashboards, app logs, and user bug reports, it gives each AI workflow a single trace timeline: what triggered it, which model call ran, how long each step took, how many tokens it burned, and where it failed.
The project is open source under the MIT license, with the code on GitHub and the hosted app at tracellm.in. It grew out of a common problem: a wrong answer shows up in production, yet the app code, the vector store, and the model provider each hold only fragments of the story. TraceLLM stitches those fragments back into one inspectable record.
The main limit is maturity. TraceLLM is young. The Node SDK is a workspace package rather than a published npm release, and the docs still point to a local quickstart for setup. If you need a battle-tested tool with a big integration gallery today, this isn't that yet. If you want to read the code and self-host it, it's a clean fit.
Getting Started
- Create an account at tracellm.in, or run the stack locally with
pnpm install,pnpm infra:build, andpnpm infra:up. - Create a project and copy the project-scoped API key (
trllm_...). - Install the Node SDK in your app and start a session with your endpoint, API key, and service name.
- Wrap your model calls in spans, then record events, token counts, and errors as you go.
- Open the Trace Explorer to inspect each session, and optionally add an OTLP endpoint to forward traces to your own backend.
Product Information
A quick look at TraceLLM's pricing, supported platforms, and performance.
Best for
The users, tasks, and scenarios where this tool fits best.
Users
- AI engineers and backend developers
- Small product teams without a platform budget
- Debugging-focused support staff
Tasks
- Tracing a bad chatbot answer
- Watching token spend
- Keeping errors attached to real sessions
- Exporting to an existing APM
Scenarios
- Shipping an agent to production and needing a way to replay a failing conversation
- Working under content rules where prompts contain personal data
- Standardizing observability across several AI services
Key features
Session and Span Tracing
TraceLLM organizes an AI workflow as a session with nested spans. A session holds the user turn. Each span covers one unit of work, such as a provider call, a retrieval step, or a tool run. That's the core of LLM tracing. You start a session, open spans inside it, record events as they happen, and close both with a status. The structure turns a loose pile of log lines into one replayable run.
Capture Policy Controls
Not everything should be stored. You set the policy per project: whether prompt and output content is captured, how much metadata and usage is kept, and whether redaction masks API keys and bearer tokens before they hit the database. This is where prompt tracing gets its guardrails. Sampling can capture 0% to 100% of sessions, and specific span kinds can be skipped entirely.
Token Usage and Latency Tracking
Each span can carry token counts and timing alongside its attributes. Cost and speed sit in the same view as the call itself. You can see which step is slow and which one is expensive. No joining two dashboards. That's the difference between knowing the bill went up and knowing which prompt caused it.
OpenTelemetry and OTLP Export
TraceLLM keeps its own product timeline and can forward selected traces to a customer-owned OTLP HTTP collector. You paste in an endpoint, choose which signals to forward, and the data flows to SigNoz, Honeycomb, Tempo, Datadog, or any OTLP-compatible backend. Teams already running an APM can keep their existing stack and treat TraceLLM as the AI-specific layer on top.
Provider-Agnostic Instrumentation
Think of it as an AI debugging tool that doesn't lock you to one vendor. The spans don't care which model you call. TraceLLM works with OpenAI, Claude, Gemini, custom routers, internal services, and MCP-shaped workflows, because you wrap the call yourself rather than relying on a provider-specific plugin. Swapping models or gateways doesn't break your tracing setup.
Project-Scoped API Keys
Each API key maps requests to one project and carries that project's capture policy into the SDK runtime. Create several keys. Name them. Revoke old ones when a service stops using them. That keeps a shared policy from leaking across unrelated apps, and it makes key rotation a simple two-step job.
Pros and cons
Pros
- MIT-licensed and open source, so there's no subscription and the code is readable.
- Local-first design lets you self-host and keep trace data on your own infrastructure.
- Capture policy separates content, metadata, usage, and errors, which helps with sensitive prompts.
- OTLP export means you don't abandon an existing SigNoz or Datadog setup.
- Provider-agnostic spans cover OpenAI, Claude, Gemini, and custom gateways without plugins.
Cons
- The Node SDK isn't published to npm yet, so early adopters install it from the repo or a workspace package.
- It's a young project, so expect fewer prebuilt integrations and community examples than mature tools offer.
- Self-hosting means you handle deployment, upgrades, and storage sizing yourself.
- The docs are still filling out, so some setup steps need reading the source.
Frequently asked questions
TraceLLM is used to debug and monitor AI applications in production. It records sessions, spans, token usage, latency, and errors for chatbots, agents, and RAG pipelines, then shows them on one timeline so you can find why a specific answer went wrong.
Related content
Explore related tools, skills, and articles for TraceLLM.
TraceLLM Alternatives
Udon
Udon · Coding · ProductivityUdon turns an Apple Silicon Mac into a home server you run from a browser, with container controls, file shares, and an optional AI sysadmin.
offstage
offstage cloud · Coding · Productivityoffstage gives AI coding agents and CI jobs a real Mac: a fresh macOS account on Apple silicon, driven over an API and billed by the second.
BotBus
Chengdu Tuyang Tusen Technology Co., Ltd. · Coding · ProductivityBotBus lets you control AI coding agents from your phone and watch: approve commands, start tasks and read results while the code stays on your desktop.