Phoenix AI
Arize AI · Coding
Phoenix AI is Arize's open-source LLM observability platform. It lets developers trace, evaluate, and troubleshoot LLM applications, running locally or self-hosted for free.
About Phoenix AI
What Is Phoenix AI
Phoenix AI is Arize's open-source platform for observing and improving LLM applications. This AI observability platform records how a request moves through your app, lets you score the output, and keeps versioned datasets so you can compare changes over time. It sits in the same space as tools like Langfuse and LangSmith, but it's fully open end-to-end with no paywalled tier.
The project started as a way to see inside LLM workflows when something breaks. A chatbot gives a wrong answer, a retrieval step pulls the wrong document, an agent loops on itself. Phoenix turns that run into a trace you can read, then lets you evaluate it and track the fix. For teams shipping AI products, that's the difference between guessing and knowing where the problem sits.
The main tradeoff is ownership. Phoenix is self-hosted, which means you run the server, handle upgrades, and bring your own infrastructure. Larger teams that want auto-scaling, guaranteed support, and compliance paperwork usually step up to Arize AX, the vendor's managed product. If your priority is keeping data on your own network, that self-hosting is exactly the point.
Getting Started
- Install the package with
pip install arize-phoenixor runuvx arize-phoenix serveif you'd rather skip the install. - Start the server with
phoenix serve, which opens on port 6006 and accepts traces out of the box. - Instrument your app by running
npx @arizeai/phoenix-cli setupin your project folder, or wire up the OpenInference SDK by hand. - Open the Phoenix UI, run your app, and watch traces land in real time; then build evals and datasets to track quality over versions.
Product Information
A quick look at Phoenix AI's pricing, supported platforms, and performance.
Best for
The users, tasks, and scenarios where this tool fits best.
Users
- AI and ML engineers
- Backend and platform developers
- Data and eval teams
- Teams with strict data rules
Tasks
- Debugging an LLM app
- Scoring output quality
- Comparing prompt changes
- Managing prompts at scale
Scenarios
- A RAG app returning bad answers
- An agent failing on long tasks
- A regulated team that can't send data out
- A team tuning evaluation as they scale
Key features
Tracing
Tracing records the runtime of your LLM app using OpenTelemetry-based instrumentation. You see each step of a request, from the prompt to the retrieval to the tool call, laid out as a trace you can open and inspect. It's the feature most people come for, because it answers the question logs can't: where exactly did this run go sideways.
Evaluation
Evaluation is open source LLM evaluation: it uses LLMs to benchmark your app's performance through response and retrieval evals. Instead of reading outputs one by one, you run a grader over a batch and get scores back. That matters when you need to compare two versions of an app and show that one is actually better, not just different.
Datasets
Datasets lets you build versioned sets of examples for experimentation, evaluation, and fine-tuning. A fixed dataset means the same inputs run every time, so a change in the score reflects the app, not the test. Teams use it to keep a stable baseline as the product moves.
Experiments
Experiments tracks changes to prompts, LLMs, and retrieval. You run the same dataset against a new setup and compare results side by side. This is how prompt engineering stops being guesswork and starts being a measurement.
Playground
The Playground is a prompt engineering playground where you improve prompts, compare models, adjust parameters, and replay traced LLM calls. It's built for the hands-on part of prompt work: try a variation, see the output, keep what works. Replaying a real trace means you test against a case that actually happened.
Prompt Management
Prompt Management handles prompt changes with version control, tagging, and experimentation. Prompts get treated like code, so a team can see what changed, who changed it, and roll back if quality drops. That structure keeps a shared prompt library from drifting.
PXI (Phoenix Intelligence)
PXI is an AI engineering agent built into Phoenix that helps debug traces, iterate on prompts, and navigate the product. It acts like an assistant inside the tool, so you can ask it to explain a trace or suggest a prompt tweak rather than hunting through menus.
Remote MCP Server
The Remote MCP Server connects Claude Code, Cursor, and other MCP clients to your Phoenix instance's /mcp endpoint. From there you can query traces, datasets, and experiments without leaving your editor. It fits teams already working inside a coding agent.
Pros and cons
Pros
- It's open source and free, with no feature locked behind a paid tier.
- Self-hosting keeps prompts, traces, and user data on your own infrastructure.
- Tracing stays vendor and language agnostic through OpenTelemetry and OpenInference, so it fits many stacks.
- Out-of-the-box support covers frameworks like LangGraph, LlamaIndex, CrewAI, and DSPy, plus major LLM providers.
- Datasets and experiments turn quality work into repeatable measurements instead of one-off checks.
- The Remote MCP Server lets coding agents query your Phoenix data directly.
Cons
- Self-hosting means your team runs the server, handles upgrades, and provisions scaling, which adds operational work.
- Heavy evaluation jobs and background scanning run on your own compute, so cost shifts from a subscription to infrastructure.
- There's no built-in production alerting or dashboards for non-technical stakeholders; those live in the managed Arize AX product.
- The range of features takes time to learn, and a small project may only need tracing to start.
Frequently asked questions
Phoenix is used to observe, evaluate, and troubleshoot LLM applications. It traces how a request moves through an app, scores the output with LLM evals, and keeps versioned datasets so you can compare changes.
Related content
Explore related tools, skills, and articles for Phoenix AI.
Phoenix AI Alternatives
Base Code
Base44 (Wix) · Coding · ProductivityBase Code is a shared cloud development environment from Base44. Teammates describe changes to an existing GitHub codebase in chat, preview them live and send them back as pull requests.
Google EmbeddingGemma 2
Google DeepMind · Writing · Image · CodingGoogle EmbeddingGemma 2 is a 740M-parameter open model that turns text, code, images, audio, and video frames into vectors in one 768-dimension space, running on phones and laptops.
GitGlow
GitGlow · Coding · ProductivityGitGlow is a local-first desktop Git client that lets independent developers review their own code and their AI agents' changes in one place before shipping.