Co-Founder & CTO @ Lilac Labs
We automate the person taking orders at the drive-thru with voice. Order accuracy is extremely critical for our customers. Getting an order wrong (i.e., missing allergies) means significant financial loss.
Before we had Hamming, we were spending hours per day calling our system by hand. Every time we changed our prompts or function definitions, we would need to retest our system to make sure it still worked and didn't break anything.
We worked closely with the team at Lilac to create 1000s of characters and scenarios that mimic how real customers order food - paying attention to the details like dietary restrictions, allergies, and preferences.
We also created an array of LLM judges that scored the performance of their voice agents for each scenario.
Now, instead of testing voice agents by hand, the team at Lilac could press a button and have our system make 1000s of concurrent calls to their voice agents.
For each phone call, the team had detailed analytics on where their system needed more work.
See more details on our approach here here.
In production, we gave the team full visibility into how users were interacting with their AI voice agents in production and flagged cases where their agents were struggling.
The team could easily convert calls and traces into test cases and add them to their golden dataset of scenarios.
See more details on our approach here here.
Scenarios created
Tests run / year
Hours saved / year
Costs saved / year
Co-Founder & CEO @ Lilac Labs
Hamming helps us simulate 1000s of customer calls (dietary restrictions, allergies, large group orders, etc.) and find gaps in our system in minutes. This gives us a huge peace of mind and clarity on where we need to improve our AI drive through agent.
The Hamming team is ready to jump and help us wherever and whenever we need them. These guys do whatever it takes to deliver.
Hamming is an essential part of our infra that directly unlocks revenue.
