DTMF testing for voice agents looks finished the moment a keypad digit appears in a transcript or tool log. It isn't. The production question is whether the caller or agent sent the right digit, the telephony path delivered it, the collector grouped it correctly, and the workflow reached the expected branch.
A demo agent with no keypad input does not need this guide. If one stable phone route has a single “press 1” step, a short manual matrix may be enough. This guide is for teams handling PINs, account numbers, menu navigation, transfers, or any mixed voice-and-keypad workflow that has to survive provider changes.
Dietmar Meister called us with exactly that problem. His team had built a banking voice agent whose spoken flows worked, but every PIN and account-number path still required a person to dial in and press buttons.
“I spend two hours a day testing keypad flows,” he told us. “There has to be a better way.”
DTMF testing for voice agents verifies the complete keypad-input chain: digit emission, telephony transport, event collection, workflow routing, recovery behavior, and final outcome. A test passes only when the expected branch and evidence agree with the digits sent.
TL;DR: Treat DTMF as a two-direction, evidence-backed regression problem.
- Test inbound DTMF from a caller's keypad to the agent.
- Test outbound DTMF from the agent to a downstream IVR or phone tree.
- Cover all 12 common keypad symbols:
0-9,*, and#.- Vary digit count, spacing, terminators, silence, invalid input, and voice-to-keypad handoffs.
- Keep proof of the emitted digit, transport event, collected sequence, branch reached, and final outcome.
A green transcript is not enough. The workflow branch is the assertion that matters.
Methodology Note: This guide combines Hamming's production voice-agent testing experience with public telephony standards and provider documentation across 10K+ voice agents (reviewed August 2026). Hamming's platform has 10M+ mins protected. We've tested agents built on LiveKit, Pipecat, ElevenLabs, Retell, Vapi, and custom-built solutions.The checklist is provider-neutral. Exact event names, timeout defaults, and supported transport modes vary by carrier, SIP trunk, PBX, and voice runtime.
Last Updated: August 2026
Related Guides:
- IVR and Voice Agent Log Correlation - join keypad, routing, telephony, transcript, and outcome evidence
- Voice Agent Workflow Testing - assert state transitions and side effects, not only spoken responses
- Voice Agent Interruption Handling - test DTMF while prompts or disclosures are playing
- Voice Agent Tests as Code - keep critical phone paths in versioned regression suites
- Voice Agent Production Readiness - make telephony-path evidence a launch gate
- Failed Production Calls to Regression Tests - preserve the exact carrier and timing conditions that exposed a failure
What Does DTMF Testing Actually Prove?
Dual-tone multi-frequency (DTMF) input is the signaling behind phone keypad digits. Modern voice systems often carry named keypad events alongside audio rather than trusting compressed audio to preserve the tones.
RFC 4733 defines the RTP telephone-event payload for named telephony events. It assigns codes 0-9 to the matching digits, code 10 to *, and code 11 to #. The payload also carries duration and end-state information. That means “the recording contains a beep” and “the runtime received digit 7” are different claims.
The same separation appears in provider APIs. LiveKit documents distinct send and receive DTMF surfaces, including both a numeric code and string digit when publishing. Twilio's Gather documentation exposes digit count, terminator, timeout, and mixed speech-plus-DTMF behavior. Dialogflow CX separately configures finish digits, interdigit timeouts, endpointing, and no-match behavior.
So what should a test prove? Five things:
- The expected party emitted the intended digit or sequence.
- The production-like telephony path delivered a DTMF event.
- The collector grouped the sequence with the right timing and terminator rules.
- The voice workflow reached the intended state or IVR branch.
- The final outcome matched the caller's goal without exposing sensitive digits.
This is the mistake we see most often: teams assert step 2 and assume steps 3 through 5 followed. We call it the logged-digit illusion. A digit in an event stream can still be truncated, duplicated, attached to the wrong prompt, or ignored by downstream routing.
Inbound vs. Outbound DTMF: Which Direction Are You Testing?
Inbound and outbound DTMF are different test contracts. Keep them separate even if one product calls both “DTMF support.”
| Direction | Sender | Receiver | Representative job | Primary assertion | Typical evidence |
|---|---|---|---|---|---|
| Inbound | Caller | Voice agent or IVR | Enter a PIN, menu choice, phone number, or confirmation | The collector returns the intended sequence and the agent takes the correct branch | received event, collected digits, prompt state, route, outcome |
| Outbound | Voice agent | External IVR or PBX | Navigate a phone tree, enter an extension, or confirm a menu choice | The external system accepts the digits and reaches the intended destination | sent event, timing, remote prompt/branch, transfer or task result |
| Mixed | Caller and agent | Multiple systems | Speak, enter digits, resume speech, then transfer | State survives each modality change without replaying or dropping input | turn timeline, DTMF events, state transitions, tool/transfer receipt |
For inbound tests, start with the caller experience. Does the agent wait for the right number of digits? What happens if the caller pauses or omits #? Does spoken “one” take a different path from pressing 1?
For outbound tests, start with remote system behavior. Did the agent wait until the menu was ready? Did it send the correct extension? Did the call reach the expected queue rather than merely logging that digits were sent?
Mixed flows deserve their own cases. A caller may speak an account type, enter four digits, resume speaking, and then transfer. The workflow testing runbook explains how to assert each state transition; the DTMF test adds exact keypad and timing evidence.
What Evidence Should Every DTMF Test Keep?
A DTMF test needs a small evidence contract. The goal is not to store sensitive keypad values everywhere. The goal is to retain enough protected evidence to prove what the system did.
{ "testCase": "enter-four-digit-pin", "direction": "inbound", "expected": { "digitCount": 4, "terminator": "#", "branch": "identity_verified" }, "observed": { "eventCount": 5, "maskedDigits": "****#", "collectionStatus": "complete", "branch": "identity_verified", "outcome": "passed" }, "timing": { "firstDigitAfterPromptMs": 740, "maxInterdigitGapMs": 1180 }}| Evidence layer | Keep | Pass when | Privacy rule |
|---|---|---|---|
| Emission | sender, intended sequence length, send timestamp | expected number of events emitted | mask PIN, account, and payment values |
| Transport | negotiated mode or provider event, event code, duration, end marker when available | each intended event is delivered once and in order | keep event metadata separate from raw sensitive digits |
| Collection | prompt state, count, terminator, timeout reason | sequence completes under the configured rules | store masked or tokenized values in broad analytics |
| Workflow | expected branch, observed branch, recovery path | the state transition matches the scenario | avoid copying secrets into prompts or transcripts |
| Outcome | task result, transfer receipt, tool result, final status | the user-visible job completes correctly | retain access-controlled pointers to detailed evidence |
The IVR log-correlation runbook goes deeper on joining call IDs, telephony events, transcripts, and outcomes. For DTMF-heavy financial or authentication flows, mask by default. A debug dashboard should not become a second store for keypad secrets.
Which DTMF Test Cases Belong in the Minimum Regression Suite?
Start with the 12 common keypad symbols, then test collection and state behavior. A suite that presses 1 once is a demo, not regression coverage.
| Test case | Input | Expected behavior | Failure caught |
|---|---|---|---|
| Digit smoke test | each of 0-9, *, # | every symbol arrives once with the correct code | mapping, provider, or event-decoding mismatch |
| Fixed-length sequence | four digits | collector completes after digit 4 | off-by-one count or premature timeout |
| Terminator present | 4821# | # ends collection and is handled per contract | terminator included or dropped incorrectly |
| Terminator omitted | 4821 then wait | configured timeout or fixed-length rule completes safely | infinite wait or wrong no-input branch |
| Fast entry | four digits with short gaps | all digits remain ordered | coalescing, duplication, or debounce error |
| Slow entry | four digits with longer gaps | collection follows the documented interdigit rule | partial submission or unexpected no-match |
| Partial input | two of four digits, then silence | reprompt or explicit failure branch | silent abandonment or accidental acceptance |
| Invalid digit | digit outside the current menu | clear retry without corrupting state | wrong route or poisoned buffer |
| Repeated digit | 1111 | all four events are retained | deduplication that removes valid repeats |
| Voice during collection | caller speaks instead of pressing | behavior matches the mixed-input policy | speech and DTMF racing for the same state |
| DTMF during prompt | digit arrives before prompt finishes | interruption policy routes or buffers it deliberately | digit dropped while TTS is active |
| Resume voice | valid sequence, then spoken request | conversation resumes with verified state intact | context reset after keypad collection |
| Outbound IVR timing | wait, send menu digit, then extension | remote branch and destination are verified | send-before-ready and wrong-branch failures |
| Retry after transport failure | first event path fails, second succeeds | one final branch transition, no duplicate side effect | non-idempotent retry or duplicate routing |
The exact gaps depend on the provider and workflow. Google's documentation makes the distinction concrete: interdigit timeout governs pauses within a sequence, while endpointing decides when a matched variable-length sequence finishes. Twilio separately exposes numDigits, finishOnKey, and timeout. Test the settings you actually deploy rather than copying someone else's defaults.
One practical rule: vary one condition at a time in the blocking suite. Put carrier, codec, handset, locale, and long-tail timing combinations into nightly or pre-release coverage. Otherwise a failed test tells you only that something changed, not where.
How Do You Score DTMF Branch Reliability?
Use branch outcome as the numerator, not raw digit detection.
DTMF branch success rate = correct expected branch outcomes / eligible DTMF test runs × 100Suppose a suite runs 200 eligible keypad scenarios. The runtime receives digits in 194 calls, but only 187 calls reach the expected branch and outcome.
- Digit receipt rate:
194 / 200 × 100 = 97% - DTMF branch success rate:
187 / 200 × 100 = 93.5%
The seven-call gap is the important part. Those are cases where DTMF existed but collection, state, routing, or downstream behavior still failed.
Do not treat 93.5% as a recommended benchmark. It is example math. Set the release threshold from workflow risk: a low-stakes survey menu and an authentication or payment flow should not share the same tolerance.
Why Did the DTMF Test Pass in a Simulator but Fail on a Phone Call?
A simulator usually validates application logic. A phone call validates the transport path too.
| Symptom | Likely boundary | Evidence to inspect | First corrective action |
|---|---|---|---|
| No digit event, audio is otherwise normal | transport negotiation or provider relay | SDP/provider configuration, event stream, carrier route | verify both ends support the deployed DTMF path |
| First digit missing | prompt timing or listener readiness | prompt end, listener start, first-event timestamp | arm collection before inviting input or add explicit readiness |
| Repeated digits collapse | collection or deduplication | event sequence numbers, digit timestamps, buffer logic | distinguish duplicate packets from legitimate repeated digits |
| Sequence ends early | interdigit timeout | gap timestamps and timeout reason | tune the rule to observed caller pacing and retest |
# appears in the captured value | terminator contract mismatch | configured finish digit and collected payload | define whether the terminator submits or belongs to the value |
| Digits arrive but route is wrong | state machine or mapping | prompt state, menu version, branch decision | assert the active menu and expected branch together |
| Agent speaks over keypad entry | interruption policy | TTS state, DTMF timestamp, playback action | define whether DTMF interrupts, buffers, or is ignored per prompt |
| Correct branch, wrong side effect | downstream workflow | tool call, transfer receipt, backend record | test the side effect and use idempotency on retries |
Do not start by rewriting the prompt. First identify the earliest evidence layer that disagrees with expectation. The voice-agent troubleshooting guide covers the wider call stack; the table above keeps the search narrow for keypad failures.
For Genesys, Asterisk, or other enterprise telephony paths, correlate the provider call ID, IVR events, DTMF events, recording, and voice-agent trace. The Genesys and Asterisk testing runbook shows the broader evidence package.
How Often Should DTMF Regression Tests Run?
Run the smallest high-risk matrix on every change that can affect the path. Run the combinatorial matrix on a slower cadence.
| Gate | Trigger | Minimum DTMF coverage | Blocking? |
|---|---|---|---|
| Pull request or CI | prompt, tool, state, provider adapter, telephony config | one inbound and one outbound happy path plus partial input and invalid digit | yes for affected critical flows |
| Nightly | scheduled regression | all 12 symbols, timing variants, mixed modality, retry paths | alert and triage |
| Pre-release | carrier, SIP trunk, codec, PBX, or runtime change | production-like phone paths across supported routes | yes |
| Post-incident | failed production keypad flow | exact failed route, timing, sequence shape, and recovery behavior | yes before closing the incident |
| Production monitoring | ongoing live traffic | masked branch outcomes, timeouts, no-match, abandonment, replay candidates | operational alert |
The tests-as-code template is useful for versioning the small blocking suite. When a production call exposes a new edge case, use the failed-call regression runbook and call replay checklist to preserve it.
Tradeoffs Worth Knowing Before You Automate Everything
No harness covers every phone path. Handsets, carriers, trunks, PBXs, and runtime versions create combinations that a deterministic suite cannot exhaust. Keep representative production-path tests and production monitoring.
Sensitive digits limit observability. The best debugging artifact may be the value you should not store. Mask values, retain counts and event timing, and keep raw evidence behind access controls when policy permits it.
Timing realism costs runtime. Testing slow input, long prompts, retries, and downstream IVRs makes suites slower. Keep a small blocking matrix and move the wider combinations to nightly or pre-release runs.
We used to frame DTMF testing as “simulate keypad presses.” That is necessary, but it is too narrow. The durable test is transport plus state plus outcome.
How to Test DTMF Manually Before You Need a Platform
You can start with a spreadsheet and two real phone routes.
- List every keypad prompt, allowed digit, expected branch, terminator, digit count, and timeout.
- Call from two representative routes or carriers.
- Run happy path, invalid digit, partial input, no input, repeated digit, and voice-to-keypad transition cases.
- Capture the call ID, timestamp, masked digits, branch reached, and final result.
- Repeat after every telephony or workflow change.
Manual testing is reasonable for a small, stable surface. It stops scaling when multiple routes, workflows, and provider changes make the matrix too large to repeat consistently. That is where automated DTMF controls, smart IVR navigation, evidence capture, and regression scheduling in Hamming become useful.
DTMF Production-Readiness Checklist
- Inbound and outbound DTMF paths have separate test cases.
- All 12 common keypad symbols (
0-9,*,#) pass a smoke test. - Fixed-length, variable-length, and terminator-based collection rules are explicit.
- Fast, slow, repeated, partial, invalid, and no-input cases have expected outcomes.
- DTMF during TTS follows a documented interruption or buffering policy.
- Mixed voice-to-keypad and keypad-to-voice transitions preserve state.
- The test asserts the workflow branch and final outcome, not only digit receipt.
- Evidence links emission, transport, collection, branch, and outcome under one call ID.
- Sensitive digits are masked or tokenized outside restricted evidence stores.
- Carrier, trunk, PBX, codec, or runtime changes trigger production-like regression tests.
- Failed production keypad paths become permanent regression cases.
- Retried DTMF workflows cannot create duplicate transfers or side effects.
The final question is simple: can your team prove why the call reached that branch? If the answer is “the digit is in the transcript,” the test is not done.
