Try our Voice Characters to automate voice agent testing.

Hamming Logo
𝕏
Lilac Labs Logo

Case Study with Lilac Labs

Shelden Shi

Shelden Shi

Co-Founder & CTO @ Lilac Labs

We automate the person taking orders at the drive-thru with voice. Order accuracy is extremely critical for our customers. Getting an order wrong (i.e., missing allergies) means significant financial loss.

Before we had Hamming, we were spending hours per day calling our system by hand. Every time we changed our prompts or function definitions, we would need to retest our system to make sure it still worked and didn't break anything.

Our approach

Step 1. Create scenarios and LLM judges

We worked closely with the team at Lilac to create 1000s of characters and scenarios that mimic how real customers order food - paying attention to the details like dietary restrictions, allergies, and preferences.

We also created an array of LLM judges that scored the performance of their voice agents for each scenario.

Feature Post 02

Step 2. Our voice agents called their voice agents

Now, instead of testing voice agents by hand, the team at Lilac could press a button and have our system make 1000s of concurrent calls to their voice agents.

For each phone call, the team had detailed analytics on where their system needed more work.

See more details on our approach here here.

Voice Agent Testing Demo

Step 3. Continuous monitoring

In production, we gave the team full visibility into how users were interacting with their AI voice agents in production and flagged cases where their agents were struggling.

The team could easily convert calls and traces into test cases and add them to their golden dataset of scenarios.

See more details on our approach here here.

Feature Post 04

Our agents call your agent

Icon 01
Icon 02
Icon 02
Icon 03
Icon 04
Icon 05
Icon 06

Impact

0K

Scenarios created

0K

Tests run / year

0K

Hours saved / year

$0K

Costs saved / year

Tony Kam

Co-Founder & CEO @ Lilac Labs

Hamming helps us simulate 1000s of customer calls (dietary restrictions, allergies, large group orders, etc.) and find gaps in our system in minutes. This gives us a huge peace of mind and clarity on where we need to improve our AI drive through agent.

The Hamming team is ready to jump and help us wherever and whenever we need them. These guys do whatever it takes to deliver.

Hamming is an essential part of our infra that directly unlocks revenue.

Tony Kam
Logo

Ship reliable AI voice agents with confidence