Roark

Most tools grade a transcript with an LLM and call it a day. Roark runs purpose-built audio models on the call itself. That's what lets it measure things a transcript loses, like pronunciation, accent clarity, vocal stress, pacing and how the agent handles interruptions. For teams whose product is the voice, that difference matters, since a transcript will never show that "metoprolol" came out wrong.

Every call is scored against more than 500 metrics, split across audio, conversational, compliance and performance groups. Conversational metrics cover resolution, empathy, task success, hallucination and tone. Compliance tracks disclosures, PII exposure, identity checks and script adherence. You can write your own rubric too, and Roark scores it on every call automatically.

Before launch, you run the agent against hundreds of simulated callers built from your real call types. There's the angry one, the rambler and the interrupter. The simulation runs over real PSTN and WebRTC, so it exercises the same network path a customer uses. Red teaming callers try to break the agent on purpose, which is how you find out whether policy holds under attack.

Once the agent is live, Roark scores every call as it lands and files the failures it finds. For a call center AI QA workflow, that means every conversation gets a score instead of a sampled handful, with the audio kept behind it. Alerts fire on any metric threshold to Slack or a webhook, and dashboards let you track score trends by agent and version. OpenTelemetry traces break a conversation into per-turn spans, so you can follow one issue around the whole loop.

The product is built around one loop. Catch a failure in production, replay the fix against simulated callers, review the diff, then verify the metric moved on live calls. Roark drafts evidence-grounded prompt edits from the failing calls. When wording was never the problem, it suggests model, voice or infra swaps. You approve each change. The loop stays under your control.

Roark connects to the major voice stacks, including Vapi, Retell, LiveKit and Pipecat, and ships an SDK, a REST API, an MCP server and a CLI. Its public leaderboard, Roark Live Bench, calls every speech-to-text, language model and speech engine combination over the real phone network and scores them end to end with the same engine that scores your calls.

Roark holds SOC 2 Type II and offers a HIPAA BAA on every plan, including pay-as-you-go. Enterprise adds SSO and SAML with SCIM provisioning, role-based access, IP whitelisting, data residency and a signed DPA. The company says it runs annual penetration tests, which is what a security review usually asks for before a vendor touches customer audio.