Most teams do not need another dashboard when production voice-agent calls fail. They need a daily artifact that says which failures matter, why they happened, who owns the fix, and which regression tests should exist tomorrow.
A voice agent daily failure report is that artifact. It sits between your voice agent dashboard and your incident response runbook: smaller than a postmortem, more actionable than a chart, and specific enough for engineering to act on before the same failure repeats.
TL;DR: Use Hamming's Daily Voice Agent Failure Report Template to summarize yesterday's failed calls in 15 minutes:
- Quantify total calls, failed calls, escaped severity, and week-over-week movement.
- Cluster failures by caller-visible symptom, likely root cause, evidence, owner, and next test.
- Escalate only when severity, recurrence, or compliance/business risk crosses a threshold.
Quick filter: If you handle fewer than 50 production calls per week, a spreadsheet and manual transcript review may be enough. This template is for teams with enough volume that "we looked at some calls" no longer explains what is breaking.
Methodology Note: The report structure and failure taxonomy in this guide are based on Hamming's analysis of 4M+ production voice agent calls across 10K+ voice agents (2025-2026).Calibrate the thresholds to your own call volume, regulated-workflow risk, SLOs, and support staffing. The template is intentionally short because a daily report that takes an hour to write stops being daily.
Last Updated: June 2026
Related Guides:
- Voice Agent Dashboard Template - metrics and chart layout that feed this report
- Voice Agent SLOs - reliability targets and burn-rate rules for escalation
- Voice Agent Incident Response Runbook - what to do when the report finds active customer impact
- Failed Production Call Regression Test Runbook - how to turn confirmed misses into tests
- Voice Agent Call Evidence Export Runbook - how to package call evidence for review
- Voice Agent Monitoring KPIs - KPI definitions for trend lines and thresholds
What should a daily voice agent failure report include?
A daily voice agent failure report should include the smallest set of fields that force a decision: volume, failed-call rate, top clusters, severity, evidence, owner, next action, and regression-test status.
A daily failure report is the operating layer between monitoring and incident response. Dashboards show that something moved. The report says whether anyone needs to change the agent.
Use this section order:
| Section | What to include | Decision it should force |
|---|---|---|
| Executive summary | 3 bullets: volume, biggest risk, action needed | Does anyone outside the agent team need to care today? |
| Metrics snapshot | Total calls, failed calls, failure rate, severe failures, repeat clusters | Is the system getting better or worse? |
| Top failure clusters | 3 to 7 clusters with sample calls and owners | Which failures deserve engineering time? |
| Severity review | Escalations, SLO impact, compliance or safety risk | Should this become an incident or release blocker? |
| Regression-test queue | New tests, existing tests updated, tests still missing | Will this failure be caught before the next deploy? |
| Open questions | Ambiguous clusters, missing evidence, product-policy questions | What needs human judgment? |
This report should not replace raw call evidence. Link to the supporting calls, traces, transcripts, or redacted evidence packets. For the evidence package itself, use the call evidence export runbook.
Who needs this report and who does not?
Use the daily report when production quality is a shared operating problem: the voice agent is live, has measurable call volume, and changes enough that yesterday's failures can predict tomorrow's regressions.
| Team situation | Use this report? | Why |
|---|---|---|
| Pilot with fewer than 50 calls/week | Not yet | Manual review is faster than report maintenance |
| Production agent with daily call volume | Yes | Failure clusters repeat and need owners |
| Regulated workflow or revenue-critical calls | Yes | Severity and audit evidence matter even at lower volume |
| Active incident in progress | Use incident process first | Daily report can summarize after mitigation |
| No access to transcripts, traces, or call metadata | Fix instrumentation first | The report will become opinion without evidence |
We used to pack these reports with every interesting transcript. It felt rigorous, but the review meetings got worse. The useful report is the one a product lead, QA lead, and engineer can read in 4 minutes and use to make the same prioritization call.
Copy-paste daily failure report template
Copy this into Slack, Notion, Linear, or your on-call handoff doc.
# Daily Voice Agent Failure Report - [Agent Name] - [YYYY-MM-DD]
## 1. Executive Summary
- Calls reviewed: [N reviewed] of [N total] production calls
- Failed-call rate: [X%] ([up/down] [Y pts] vs prior day)
- Highest-risk issue: [cluster name, severity, owner]
- Decision needed today: [none / incident / release block / product policy / customer follow-up]
## 2. Metrics Snapshot
| Metric | Today | Prior Day | 7-Day Baseline | Status |
|--------|-------|-----------|----------------|--------|
| Total calls | [N] | [N] | [N/day] | [normal/watch] |
| Failed calls | [N] | [N] | [N/day] | [normal/watch] |
| Failure rate | [X%] | [Y%] | [Z%] | [normal/watch/critical] |
| Severe failures | [N] | [N] | [N/day] | [normal/watch/critical] |
| Repeat clusters | [N] | [N] | [N/day] | [normal/watch/critical] |
| Regression tests created | [N] | [N] | [N/day] | [on-track/behind] |
## 3. Top Failure Clusters
| Rank | Cluster | Symptom | Evidence | Likely cause | Owner | Next action | Test status |
|------|---------|---------|----------|--------------|-------|-------------|-------------|
| 1 | [name] | [caller-visible failure] | [call ids/evidence link] | [ASR/prompt/tool/TTS/telephony/policy] | [team/person] | [fix or investigation] | [created/missing] |
| 2 | [name] | [caller-visible failure] | [call ids/evidence link] | [ASR/prompt/tool/TTS/telephony/policy] | [team/person] | [fix or investigation] | [created/missing] |
| 3 | [name] | [caller-visible failure] | [call ids/evidence link] | [ASR/prompt/tool/TTS/telephony/policy] | [team/person] | [fix or investigation] | [created/missing] |
## 4. Severity and Escalation
- Incident opened: [yes/no, link]
- SLO or error-budget impact: [yes/no, which SLO]
- Compliance/safety risk: [yes/no, why]
- Customer follow-up needed: [yes/no, owner]
- Release blocked: [yes/no, release link]
## 5. Regression-Test Queue
- New tests created today: [N]
- Existing tests updated: [N]
- Missing tests that need owner: [list]
- Production failures that should become golden calls: [list]
## 6. Open Questions
- [Question 1]
- [Question 2]
- [Question 3]Keep the summary short. Put raw transcripts, audio, trace IDs, and screenshots behind links. If a reviewer has to scroll through 40 call snippets before seeing the owner, the report is no longer doing its job.
How should you classify failed voice calls?
Classify failures by caller-visible symptom first, then likely technical cause. Error codes are useful for debugging, but the report should start from what the caller experienced.
| Caller-visible symptom | Likely cause buckets | Evidence to attach | First owner | Next action |
|---|---|---|---|---|
| Call never connected | Telephony, SIP, carrier, number reputation | Call setup logs, provider status, connection error | Infrastructure or telephony | Check provider dashboard and call-routing changes |
| Caller heard silence | Audio routing, TTS delay, VAD, websocket disconnect | Audio trace, silence duration, TTS log | Voice runtime | Reproduce with same provider and route |
| Agent interrupted or talked over caller | Turn detection, latency, response length, barge-in | Interruption timestamps, latency trace, transcript | Agent engineering | Tune endpointing and shorten response |
| Agent misunderstood request | ASR, intent routing, prompt, missing test persona | Transcript, ASR confidence, expected intent | QA or prompt owner | Add scenario to regression suite |
| Agent gave wrong or unsafe answer | Knowledge grounding, policy, hallucination, tool failure | Response, source/tool result, policy reference | Product and engineering | Block unsafe response and add validation |
| Tool action failed | API, auth, schema, retry policy, timeout | Tool call log, parameters, error, retry count | Backend owner | Fix integration and add tool-call test |
| Bad escalation | Routing policy, handoff availability, CRM mapping | Escalation event, queue status, final outcome | Operations or workflow owner | Update escalation rule and test handoff |
A useful failed-call cluster groups calls by caller-visible symptom and next action, not just by provider error code. "ASR timeout" is a clue. "Spanish callers cannot reschedule appointments after office noise" is a cluster.
Public tools like Twilio Voice Insights expose useful call-quality metrics, timelines, and quality indicators. That data is a starting point. Voice-agent teams still need the agent-layer context: prompt version, tool result, expected outcome, escalation rule, and whether the issue already has a regression test.
How do you choose severity and owners?
Severity should follow customer harm, recurrence, and business or compliance risk. Do not base it only on whether infrastructure was down.
| Severity | Trigger | Sample | Owner | Response |
|---|---|---|---|---|
| SEV-1 | Active widespread failure or unsafe regulated behavior | 40% of calls cannot complete account verification | Incident commander + engineering | Open incident now |
| SEV-2 | Repeated high-impact cluster or SLO burn | Appointment reschedule fails for 18% of callers after a prompt change | Engineering owner | Same-day fix or rollback decision |
| SEV-3 | Contained cluster with clear workaround | Tool timeout affects one low-volume workflow | Owning team | Fix in next planned release |
| SEV-4 | Rare or cosmetic issue | Agent wording is awkward but task completes | Product or prompt owner | Backlog with evidence |
Tie this table to your voice agent SLOs. If task completion, escalation correctness, or latency burns through the daily budget, the report should say so plainly.
One trap: teams under-escalate failures when the agent technically stayed online. A voice agent can be "up" while it gives unsafe advice, loops the caller, or fails the highest-value workflow. Treat caller outcome as the severity source of truth.
How do you write the report in 15 minutes?
The report should be fast because the decisions should be pre-wired.
| Minute | Action | Output |
|---|---|---|
| 0-2 | Pull yesterday's dashboard and failed-call sample | Volume, failure rate, top metric movement |
| 2-5 | Sort by severity signals | Severe failures and possible incident triggers |
| 5-9 | Cluster by caller-visible symptom | 3 to 7 named clusters |
| 9-12 | Attach evidence and owner | Links, likely cause, next action |
| 12-14 | Add regression-test status | New tests, missing tests, golden-call candidates |
| 14-15 | Write the executive summary | 3 bullets and one decision request |
If this takes longer than 15 minutes, the problem is upstream. Either your dashboard does not expose the right filters, your traces do not connect calls to tool and prompt versions, or your team has not agreed on ownership. The daily report will surface that gap quickly.
Sample report summary
## Executive Summary
- Reviewed 24 failed calls from 3,842 production calls yesterday. Failure rate rose from 2.8% to 4.1%.
- Highest-risk cluster: pharmacy refill callers heard correct eligibility status but the agent failed the final confirmation step in 11 calls.
- Decision needed today: block the refill-flow prompt release until the missing confirmation regression test is added.Notice what is not in the summary: 24 transcript excerpts. Keep the evidence behind links and put the decision in the thread where the owner will actually respond.
What should happen after the report is sent?
A daily failure report earns its keep only when it changes tomorrow's queue. End every report with one of four outcomes.
| Outcome | When to use it | Follow-up |
|---|---|---|
| Create regression test | Failure is real and reproducible | Add scenario, expected behavior, and owner |
| Open incident | Failure is active, severe, or customer-visible | Use the incident response runbook |
| Update monitoring | Failure was found manually or too late | Add KPI, alert, or dashboard filter |
| Make product decision | The agent followed current policy but outcome was bad | Product owner decides expected behavior |
We found that strong reports create fewer debates by the second week. The same clusters should either have tests, owners, or an explicit "we accept this risk" decision. If the same issue appears every morning with no movement, the report is documenting drift, not driving improvement.
After-report checklist
- Every SEV-1 and SEV-2 cluster has an owner.
- Every confirmed product or agent failure has a regression-test decision.
- Every missing evidence field has an instrumentation owner.
- Every compliance or safety concern has a review path.
- Every repeated cluster has a trend note, not just another sample call.
- The next report can reuse the same taxonomy.
For teams using OpenTelemetry-style traces, connect the report to the trace or span that explains the failure. The voice agent observability tracing guide covers the instrumentation side; this template covers the human operating loop.
Flaws but not dealbreakers
The template depends on decent evidence. If you do not log call IDs, prompt versions, tool calls, and outcomes, the report will become a guessing exercise. Fix logging before asking reviewers to classify every miss by hand.
Daily reporting can create false precision. A cluster with 4 sample calls may be a real regression or just a noisy day. Use the report to decide what to inspect next, then confirm with more calls before making broad product claims.
Not every failure deserves engineering work. Some callers will be out of scope, abusive, silent, or impossible to satisfy with the current workflow. Keep a "known non-actionable" bucket so the team does not reopen the same debate every morning.
