How Lilac Labs Ensures Drive-Thru Order Accuracy with Hamming AI

Background

Lilac Labs, founded by Tony Kam and Shelden Shi, develops AI voice agents for automating drive-thru order taking in Quick Service Restaurants (QSRs). Their primary challenge was ensuring consistent order accuracy while handling complex scenarios like dietary restrictions and allergies.

The Challenge

Before implementing automated testing, Lilac Labs faced several operational hurdles:

• Manual testing required hours of daily team time
• System changes necessitated comprehensive retesting
• Limited coverage of edge cases and special scenarios
• Difficulty in consistently testing complex orders

Implementation

Lilac Labs implemented Hamming AI's testing platform in three key phases:

1. Test Scenario Development

• Created thousands of customer personas and ordering scenarios
• Developed LLM judges to evaluate voice agent performance
• Focused on critical cases like dietary restrictions and allergies

2. Automated Testing

• Enabled concurrent testing of thousands of calls
• Provided detailed analytics on system performance
• Identified specific areas needing improvement

3. Production Monitoring

• Implemented real-time monitoring of customer interactions
• Flagged problematic cases automatically
• Converted real calls into test cases for continuous improvement

Impact

The implementation has driven significant operational improvements, with current annualized metrics showing:

• 5,000 test scenarios in production
• 130,000 automated tests run per year
• 5,200 hours saved annually
• $520,000 in run-rate cost savings per year

Key Outcomes

1. Automated testing replaced manual processes
2. Improved coverage of edge cases and special requests
3. Faster iteration and development cycles
4. Enhanced confidence in system reliability

For more information about voice AI testing solutions, contact our team.