OpenLIT

OpenLIT records every LLM call, tool call, MCP request, retrieval and agent step as a standard OpenTelemetry trace. You can filter by user, session, cost or latency, and you can send the same data to Grafana, Datadog or any OTLP endpoint instead of keeping it in OpenLIT alone. That portability is the main reason teams pick it over a closed tool.

Adding openlit.init() is enough to instrument supported providers, frameworks and vector databases, and the list runs past 50 integrations including OpenAI, LangChain, CrewAI and ChromaDB. For coding agents like Claude Code, Codex and Cursor, a CLI installs instrumentation without touching your source.

OpenLIT scores outputs for hallucination, bias, toxicity, relevance, completeness and instruction following. You can run these checks on live traffic or on an experiment, and gate a CI pipeline on the results, so a quality drop blocks a release instead of reaching users first.

Prompt Hub stores prompts in one place, versions them with Major, Minor and Patch tags, and swaps in {{variable}} values at runtime. You can deploy or roll back a prompt without shipping new app code, which keeps copy changes out of the release cycle.

Guardrails inspect requests as they arrive and act on prompt injection, sensitive topics and topic restrictions. You set the rules, and OpenLIT enforces them on the request path, which gives you a safety layer that doesn't depend on the model behaving.

The platform handles cost tracking and latency across models, providers, users, sessions and agents, and supports custom pricing for fine-tuned models. A separate GPU collector pulls utilization, memory, temperature and power from NVIDIA, AMD and Intel hardware, so cost and hardware pressure sit in the same dashboard.