Polarity

Polarity is an evaluation platform built for teams shipping AI agents. Instead of testing agents against a scripted mock, it runs them in a sandboxed Docker environment wired to real backing services, so the agent sees the same APIs, databases, and tools it would touch in production. That setup matters, because most agent failures don't come from a bad answer. They come from the agent taking an action nobody wanted it to take.

The platform checks two things at once. First, whether each run respects the rules you set, which Polarity splits into invariants that must always hold and forbidden rules that must never fire. Second, how stable the behavior is. Running the same task across several replicas surfaces non-determinism that a single pass would hide entirely. Variance is the tell. And that distinction matters, because an agent that passes one run and fails the next is the kind that quietly breaks production long after everyone stopped watching it.

The main limit is scope. Polarity is a developer tool, not a consumer app, so you'll need your agent packaged and your backing services reachable from the sandbox before you get value. Teams with no automated test setup yet will spend their first few days wiring things up.

Getting Started

1. Sign in at the Polarity website and create a project for the agent you want to evaluate.
2. Point the project at your agent and define the backing services it needs, so the sandbox comes up with the same dependencies production has.
3. Write your invariants and forbidden rules as the checks each run gets scored against.
4. Run a task in one or more replicas to see whether behavior holds steady or drifts between attempts.
5. Open any failure, replay it from its seed, and fix the root cause before shipping.