A voice agent failure rate is easy to calculate and surprisingly easy to manipulate. Change which calls count, treat a required human handoff as a failure, or count three retries as three failed calls, and the same production day can produce three different percentages.
If you are running a low-volume pilot, start with counts and reviewed examples. A rate based on five calls adds polish, not signal. This guide is for production teams that need one definition to survive dashboards, weekly reports, release comparisons, and incident reviews.
Voice agent failure rate is the percentage of eligible production calls that end in a defined bad outcome. A complete definition names the population, eligibility rule, terminal outcomes, exclusions, attribution policy, measurement window, and contract version.
TL;DR: Use one versioned Voice Agent Failure Rate Measurement Contract:
- Count calls once: use one terminal outcome per eligible call.
- Separate the questions: report customer-visible, agent-caused, and component failure rates independently.
- Expose the denominator: publish raw calls, exclusions, eligible calls, failures, unknowns, and evaluator coverage.
- Slice before comparing: start with intent and agent version, then examine language, provider, route, and authorized customer metadata.
The formula is one line. The contract behind it is the real work.
Methodology Note: The measurement contract and operational guidance in this dictionary are based on Hamming's analysis of production voice agent calls across 10K+ voice agents (2025-2026). Hamming's platform has 10M+ mins protected.Google SRE's service-level guidance informs the eligible-event model. OpenTelemetry's error-recording conventions inform the low-cardinality component error fields. The alert examples are starter policies to calibrate, not universal benchmarks.
Last Updated: August 2026
Related Guides:
- Voice Agent Analytics Metrics Guide - the broad production analytics hub
- Voice Agent Daily Failure Report Template - the operating report that uses this rate
- Voice Agent SLOs and Error Budgets - turn bad-call rates into reliability policy
- Voice Agent Monitoring KPIs - place failure rate beside latency, completion, and escalation metrics
- Voice Agent Dashboard Template - show rates, counts, baselines, and cohorts together
- Call Logging Taxonomy - preserve the events needed to calculate the metric
Which Voice Agent Failure Rate Do You Mean?
Do not start by debating the threshold. Start by naming the question.
| Rate | Numerator | Denominator | Decision |
|---|---|---|---|
| Customer-visible failure rate | Eligible calls where the caller did not receive the expected outcome | All eligible calls | How often did the experience fail, regardless of cause? |
| Agent-caused failure rate | Eligible calls where agent behavior or an agent-owned dependency caused the bad outcome | All eligible calls | How often did the deployed agent cause the failure? |
| Component failure rate | Failed operations for one component, such as tool calls or ASR requests | Eligible operations for that component | Which subsystem needs engineering attention? |
These rates can move in different directions. A carrier outage can raise customer-visible failures without changing agent-caused failures. A tool can timeout twice and recover on the third attempt, raising component errors while the call still succeeds.
Collapsing those events into one failure percentage makes the executive chart look cleaner and the operating decision less clear. The owner, remediation, and release decision depend on which rate moved.
The same rule applies to voice agent troubleshooting: diagnose a call-level symptom with component-level evidence, but do not pretend the component attempt and the caller outcome are the same unit.
What Is the Voice Agent Failure Rate Formula?
The headline formula is:
Customer-visible failure rate = customer-visible failed eligible calls / all eligible calls x 100The agent-attributed version is:
Agent-caused failure rate = agent-caused failed eligible calls / all eligible calls x 100For a component such as a booking tool:
Booking-tool failure rate = failed booking-tool operations / all eligible booking-tool operations x 100Google's SRE guidance describes error rate as a fraction of all received requests and recommends standardizing indicator definitions. The SRE workbook uses the related pattern of good events divided by total events. For voice agents, calls replace requests only after you define which calls are eligible and what outcome counts as bad.
A worked example with 3,842 calls
Suppose yesterday's production export contains 3,842 call records:
| Population step | Calls | Rule |
|---|---|---|
| Raw production records | 3,842 | All records in the daily export |
| Synthetic and internal test calls | -54 | Excluded by environment and test marker |
| Caller cancellations before first agent audio | -126 | Excluded by failure-rate-v1 eligibility (agent never began the flow); reported in the connection or abandonment metric |
| Eligible calls | 3,662 | Denominator for both call-level rates |
| Customer-visible failed calls | 151 | Caller did not receive the expected outcome |
| Agent-caused failed calls | 133 | Agent or agent-owned dependency caused the outcome |
The calculations are:
Customer-visible failure rate = 151 / 3,662 x 100 = 4.12%Agent-caused failure rate = 133 / 3,662 x 100 = 3.63%If the prior comparable window was 2.81%, the customer-visible failure rate increased by 1.31 percentage points. The relative increase was 46.7%.
Say which one you mean. “Failure rate increased 46.7%” sounds much larger than “failure rate increased 1.31 points,” even though both describe the same movement.
Which Calls Belong in the Denominator?
The denominator should represent the opportunities the agent had to succeed inside the measured scope. Write the rule before looking at the result.
Use this measurement contract:
| Contract field | Required definition | Example |
|---|---|---|
| Population | Environment, agent, route, and time window | Production booking calls, UTC day |
| Eligibility | Event that puts a call in scope | Platform accepted call and agent began the supported flow |
| Terminal outcome | States that close measurement | succeeded, failed, policy_handoff, caller_cancelled, unknown |
| Failure rule | Outcomes counted in numerator | failed after agent entered supported flow |
| Exclusions | Explicit non-eligible states | synthetic test, duplicate event, pre-agent caller cancellation |
| Attribution | Customer-visible versus agent-caused | provider outage visible to caller but external to agent |
| Classification | One primary failure plus optional secondary types | primary: tool_execution; secondary: recovery |
| Coverage | Scored and unscored eligible calls | 3,610 scored, 52 unknown |
| Version | Contract revision and effective date | failure-rate-v1, 2026-08-30 |
| Owner | Team that approves definition changes | Voice QA and agent platform owners |
Denominator drift happens when eligibility or exclusions change without a version change. The trend line moves, but production behavior may not have changed at all.
Put this count sheet next to every rate:
raw_calls=3842excluded_test_calls=54excluded_pre_agent_cancellations=126eligible_calls=3662customer_visible_failures=151agent_caused_failures=133unknown_outcomes=52contract_version=failure-rate-v1Do not silently exclude inconvenient traffic. A policy-required human handoff may be neutral for agent capability, but it still belongs in the operating report. If the caller cancels before the first agent word, exclude that call from call-level failure rates only when the contract's eligibility rule says so. Keep it visible in a connection or abandonment metric.
This is why the daily failure report template keeps total calls, eligible calls, failures, and severe clusters close together. Percentages without counts invite false certainty.
How Should You Classify a Failed Voice Call?
Use one primary failure type for the headline distribution. Keep secondary types for causal detail.
| Primary failure type | Terminal caller outcome | First evidence to inspect | Likely owner |
|---|---|---|---|
| Connection or media | Call never connects, audio is one-way, or greeting never plays | SIP/WebRTC events and media trace | Telephony or platform |
| Speech recognition | Supported request is materially mistranscribed | Audio, transcript alternatives, ASR confidence | Voice runtime |
| Conversation behavior | Agent loops, ignores context, or chooses the wrong next action | Turn sequence, prompt and model version | Agent engineering |
| Tool execution | Required tool times out, errors, or returns unusable data | Tool request, response, timeout, retry chain | Integration owner |
| Knowledge or policy | Agent gives an unsupported answer or violates policy | Retrieved context, response, evaluator evidence | Product, knowledge, or safety owner |
| Handoff | Required escalation fails or loses context | Transfer event, destination, summary payload | Operations or telephony |
| Evaluation unknown | Evidence is missing or evaluators disagree | Scoring coverage and review queue | QA owner |
The evaluation unknown row is not a failure-rate escape hatch. Report it beside coverage. If 12% of calls are unscored, a precise-looking 3.63% rate is incomplete.
OpenTelemetry's error-recording conventions recommend predictable, low-cardinality error.type values and consistent classification across spans and metrics. Apply that principle to component operations. Keep raw exception messages and high-cardinality details in traces or logs, not metric labels.
Copy-ready failure event
{ "event_name": "voice_call_outcome", "contract_version": "failure-rate-v1", "environment": "production", "eligible_for_failure_rate": true, "terminal_outcome": "failed", "customer_visible_failure": true, "agent_caused_failure": true, "primary_failure_type": "tool_execution", "secondary_failure_types": ["conversation_recovery"], "intent": "appointment_booking", "agent_version": "agent-2026-08-30", "prompt_version": "prompt-42", "language": "en-US", "provider": "example-provider", "evaluator_status": "reviewed"}Keep customer identifiers out of metric dimensions unless the access model and use case require them. The call logging taxonomy covers evidence fields, retention, and privacy boundaries in more detail.
How Should Retries and Recovered Errors Count?
Count the call once at the terminal outcome layer. Count component attempts at the component layer.
| Scenario | Call-level result | Component result |
|---|---|---|
| Tool times out twice, succeeds on third attempt, caller completes task | Success, with degraded experience if latency crossed policy | 2 failed attempts out of 3 |
| ASR mishears a slot, agent confirms and corrects it | Success, with recovery signal | 1 recognition or extraction miss |
| Agent loops after a tool error and caller hangs up | Failure | Tool error plus recovery failure |
| Required policy handoff completes with context | Policy-neutral or success, based on contract | Successful handoff operation |
| Duplicate completion webhook arrives | One terminal call outcome | Duplicate event rejected, not another call |
OpenTelemetry makes a similar distinction for operations: handled errors that allow an operation to complete gracefully should not mark the completed operation as failed. Preserve the underlying attempt error for diagnosis, but do not multiply the caller-level numerator.
Retry accounting is where a resilient system can get punished by a bad dashboard. Recovery produces more recorded attempts; double-counting those attempts makes the caller-level rate look worse. One unit, one terminal outcome.
Use voice agent observability tracing to connect the call outcome to the failed attempts without merging their units.
How Should You Segment Voice Agent Failure Rate?
Always show the numerator and denominator for each slice. Start with the dimensions closest to the behavior change:
- Intent or workflow.
- Agent, prompt, model, and configuration version.
- Language and locale.
- Provider and dependency route.
- Authorized customer, workspace, location, or campaign metadata.
| Slice | Why it moves | Comparison trap | Decision |
|---|---|---|---|
| Intent | Some tasks are harder or riskier | Blended rate improves when easy intents grow | Fix or narrow the failing workflow |
| Agent version | Prompt or code changed | Unequal traffic allocation | Roll back or expand the better version |
| Language | ASR, TTS, prompt, and data coverage differ | Small-language cohorts look volatile | Add samples and language-specific tests |
| Provider | Routing and regional performance differ | Provider handles a different traffic mix | Compare like-for-like cohorts |
| Customer or location | Policy, catalog, integration, and caller mix differ | High-volume accounts dominate aggregate | Prioritize customer-specific remediation |
Aggregate rates can improve while an important cohort gets worse. The common version of this mistake is simple: a release receives more easy calls than the control, so its blended rate looks healthier.
Freeze the cohort definition before comparing versions. If the mix differs, compare within each intent and report the traffic weights.
When Should Failure Rate Trigger Action?
A percentage alone should not page anyone. Pair it with count, severity, baseline movement, and reliability policy.
| Signal | Starter rule | Action |
|---|---|---|
| Confirmed safety or compliance failure | Any event that policy classifies as severe | Page or open the required incident immediately |
| Fast regression | At least 2x the 28-day comparable baseline and 10 excess failures in 30 minutes | Investigate recent changes and active customer impact |
| Slow regression | At least +0.5 percentage points for two daily windows and 30 failures | Create an owned reliability task and regression cohort |
| Small cohort volatility | Rate moved but fewer than 20 eligible calls | Review examples; do not page on percentage alone |
| SLO burn | Current bad-event rate consumes budget faster than policy allows | Apply the release and incident policy |
These are starter rules, not benchmarks. Backtest them against at least 28 days of comparable traffic, then tune for volume and risk.
Google's SRE workbook defines burn rate as how fast a service consumes its error budget relative to the objective. The voice agent SLO guide shows how to translate bad-call rates into fast-burn alerts, slow-burn work, and release gates.
After a confirmed cluster, use the failed-call regression test runbook to turn production evidence into a pre-release test. The operating loop is measure, inspect, fix, test, and compare the same cohort again.
Failure Rate vs Success, Containment, Escalation, and Error Rate
Do not treat these terms as synonyms.
| Metric | Direct definition | Can be high while failure rate is high? |
|---|---|---|
| Success rate | Eligible calls that meet the success contract / eligible calls | No, if every call is exactly success or failure; yes when unknown and neutral states exist |
| Containment rate | Calls resolved by AI without inappropriate human handoff / eligible calls | Yes, if callers are trapped or abandon without resolution |
| Escalation rate | Calls handed to a human / eligible calls | Yes; required escalations may be correct behavior |
| Component error rate | Failed component operations / eligible component operations | Yes; retries may recover before the call ends |
| Failure rate | Eligible calls ending in the defined bad outcome / eligible calls | It is the outcome being measured |
The voice agent analytics metrics guide explains how containment, sentiment, flow, and quality work together. Failure rate is the narrower operating metric. It should point to a review queue, not replace the broader picture.
What Are the Limitations of a Failure-Rate Metric?
No measurement contract removes judgment. Three tradeoffs are worth understanding:
Outcome labels can be wrong. Automated evaluators miss context, and human reviewers disagree. Report evaluator coverage, calibrate against reviewed calls, and preserve an unknown state.
Exclusions can become incentives. Teams may redefine unsupported or abandoned calls out of scope to improve the headline. Version the contract and review exclusion movement beside the rate.
Small cohorts are noisy. A single failure in five calls is 20%. Show counts, widen the window, and inspect the examples before changing production.
Manual QA is a valid starting point. Pull 20 to 30 calls, label terminal outcomes together, document the disagreements, and only then automate the contract. A careful spreadsheet beats a dashboard built on an undefined denominator.
Voice Agent Failure Rate Implementation Checklist
- Name the customer-visible, agent-caused, or component rate being reported.
- Define the production population, window, timezone, and environment.
- Write the eligibility rule and every exclusion before computing the rate.
- Use one terminal call outcome and one primary failure type.
- Preserve retries and secondary causes at the component/evidence layer.
- Report raw calls, eligible calls, failed calls, unknowns, and evaluator coverage.
- Segment by intent and version before comparing broader cohorts.
- Backtest alert rules against at least 28 days of comparable traffic.
- Connect confirmed failure clusters to owners and regression tests.
- Version the contract and review definition changes like code changes.
The result should fit in one sentence: “Customer-visible failure rate was 4.12% of 3,662 eligible production calls under failure-rate-v1, up 1.31 percentage points from the 2.81% comparable baseline, concentrated in appointment booking on prompt-42.”
That sentence can drive a decision. “Failure rate is up” cannot.
