Basata scales healthcare voice agent testing

Basata's AI agents are highly configurable per healthcare practice, each with its own conversation flows, scheduling rules, and guardrails. Manually writing and maintaining test cases for every configuration was not sustainable.

With Hamming's API, the team defines agent behavior in YAML and programmatically converts it into test cases. This approach enables them to:

• Auto-generate test cases from configured agent functionality
• Manage state across multi-step conversations (e.g., verifying appointments were actually booked)
• Inject scenario-specific test data into each run
• Keep tests in sync with agent configuration changes

The programmatic layer eliminates the need to manually create or update test cases through a UI.

Every healthcare practice Basata serves has its own agent with unique conversation flows and conditionals. Manually testing every path required individual phone calls, one at a time, with a single person on each.

With Hamming, the entire test suite runs concurrently. Instead of spending 90 minutes on serial calls, the team fires off all tests at once and gets results in 15 minutes. This concurrent execution is critical because:

• Every agent has its own conversation flow with unique conditionals
• LLMs are stochastic, so tests must run multiple times for confidence
• The number of required tests grows exponentially with each new customer

What previously required an hour and a half of focused manual effort now completes in a fraction of the time, freeing the team to focus on building rather than testing.

Basata serves a diverse patient population, many of whom rely on the phone as their primary way to interact with their healthcare practice. The team has observed that regional differences in voice and communication style significantly affect how patients respond to the AI agent, with some combinations causing immediate hang-ups.

With Hamming's persona library, the team tests across demographics they simply cannot replicate manually. Switching between genders, accents, and speaking speeds doesn't scale with a human team, but it's built into the platform:

• Different accents and regional speech patterns
• Varying speaking speeds and communication styles
• Voice and gender combinations that trigger different patient responses
• Demographics representative of real patient populations

The team plans to leverage this capability further through A/B testing different voice and persona combinations in production, using Hamming's data to validate which configurations work best for different patient demographics and regions, moving from reactive fixes to proactive optimization based on real data.

One of the most sophisticated uses of Hamming at Basata goes beyond traditional testing. The team feeds Hamming's LLM judge evaluator descriptions back into a local agent that automatically tunes system prompts, model selection, and agent configuration.

This creates a closed-loop optimization cycle:

• Run test suite via Hamming API
• Collect evaluator descriptions and failure analysis
• Feed results to a local agent that updates the system prompt
• Re-run tests to validate improvements

For the team, manual prompt engineering felt less like engineering and more like trial and error. This closed-loop system replaces that guesswork with data-driven optimization, letting the agents improve themselves.