QAgent

QAgent scores agents across eight separate metrics rather than a single vague score. These cover answer quality, factual groundedness, policy adherence, escalation correctness, RAG faithfulness, contextual relevancy, context recall and multi-turn memory, and each dimension maps to a real production failure mode. That detail matters. A low number points at a specific problem rather than a general mood.

Rather than matching strings or regex patterns, QAgent cross-examines each response with a calibrated judging model. It runs on Groq LPUs for speed, which matters when you want a full regression suite finished in under a minute. Pass and fail calls come with a confidence level, so you know when a result sits near the line and deserves a closer manual look before you trust it.

Every test combines three ingredients: your official rules, the test prompt with RFC 2119 style expected behavior, and the agent's live response. The judge checks strictly against the rubric. A polite but wrong answer still fails. That's what separates QAgent from a vibe check in a chat window.

The judge knows the difference between a hallucination and normal behavior. Ticket IDs that change per session, live balances and polite greetings aren't flagged as unsupported claims. Nice. If the top retrieved chunk holds the full answer, lower-ranked results don't drag the score down. Jailbreak tests don't penalize the retriever for missing attack docs.

QAgent records score deltas on every prompt iteration. You see instantly whether a change that fixed one case caused a drop elsewhere. That's the whole point. For teams that edit system prompts often, this turns silent degradation into a visible line on a chart.

The Solo plan exports CSV and print-ready Quality Audit reports. These show groundedness percentages, RAG faithfulness and policy adherence, which is useful when a client wants evidence rather than a promise. Send the file. Reports turn a testing run into something you can attach to a project handover.