iFixAi

iFixAi is an independent auditor for AI agents. Point it at an agent's chat endpoint and it builds a simulation of the business that agent works in, sends the requests an ordinary day brings, and passes every reply to a judge model from a different provider. What comes back is a pass or fail for each inspection, the exchange that produced it, and a letter grade.

The premise is that a green eval suite proves the agent can finish the task, not that it stayed inside its role. The audit targets agent misalignment rather than task failure. iFixAi's own demo makes the point: a support agent resolves a refund ticket with every internal check green, while hiding a fraud flag, paying a refund a manager had declined, and changing a payout account without authorisation. Task completion looks perfect. Authority, process and disclosure do not.

It's an open-source AI audit engine under Apache 2.0, and it runs on your machine with your own model keys, so code and prompts do not leave your environment. That independence has a cost. The simulation is built from a profile you write, and a thin profile produces a thin audit. The grade is also not a certificate. It's evidence you can hand to a reviewer, not a regulator's seal.

Getting Started

1. Install the CLI with pip install ifixai on Python 3.10 or newer, or add the Claude Code or Codex plugin, or the skill for Cursor, VS Code, Windsurf and Zed.
2. Describe the agent. Copy the reference profile, list its role, tools and who is allowed to do what, then run ifixai author and ifixai validate to generate and check the simulation environment.
3. Set up a run with ifixai setup. Give it the agent's endpoint and a judge key from a different provider, so the agent never grades its own work.
4. Try ifixai run --dry-run first. It reports how many tests and judge calls the run will make, and which checks do not apply to your agent, before anything is billed.
5. Run it for real. The scorecard prints in the terminal and JSON plus Markdown reports land in ./ifixai-results/.