Here’s a demo video: https://www.youtube.com/watch?v=eu8mo28LsTc.
We ran into this problem while building a voice AI agent for a dental clinic. Patients kept getting stuck in loops, failing to confirm insurance, or misunderstanding responses. The only way to test fixes was to manually call the agent or read through hundreds of transcripts, hoping to catch issues. It was slow, frustrating, and unreliable.
Talking to other teams, we found this wasn’t just a niche issue - every team building Voice AI struggled to validate performance efficiently. Debugging meant calling the agent over and over. Updates shipped with unknown regressions. Sentiment analysis relied only on text, missing key audio cues like hesitation or frustration, which often signal deeper issues.
That’s why we built Roark. Instead of relying on scripted test cases, Roark captures real production calls from VAPI, Retell, or a custom-built agent via API and replays them against your latest agent changes. We don’t just feed back text, we preserve what the user said, how they said it, and when they said it, mimicking pauses, sentiment, and tone up until the conversation flow changes. This ensures your agent is tested under real-world conditions, not just synthetic scripts.
For each replay that we run, Roark checks if the agent follows key flows (e.g. verifying identity before sharing account details). Our speech based evaluators also detect sentiments such as frustration and confusion, long pauses, and interruptions - things that regular transcripts miss.
After testing, Roark provides Mixpanel-style analytics to track failures, conversation flows, and key performance metrics, helping teams debug faster and ship with confidence. Instead of hoping changes work, teams get immediate pass/fail results, side-by-side transcript comparisons, and real-world insights.
We’re already working with teams in healthcare, legal, and customer service who rely on Voice AI for critical interactions. They use Roark to debug AI failures faster, test updates before they go live, and improve customer experiences - without manually calling their bots dozens of times.
Our product isn’t quite ready yet for self-service, so you’ll still see the dreaded “book a demo” on our home page. If you’re reading this, though, we’d love to fast-track you, so we made a special page for HN signups here: https://roark.ai/hn-access. If you’re working on Voice AI and want to try us out, please do!
Would love any feedback, thoughts, or questions from the HN community!