Voice agent dynamic context testing proves that an agent uses the right context value at the right turn, especially when the value changes while the call is still running.
A static FAQ agent with no personalization, tools, handoffs, or changing backend state does not need this template. A few multi-turn transcript tests are enough.
This guide is for agents where context changes what the system may say or do: eligibility becomes approved, an order status changes, a human adds a note, a tool returns a new account state, or a handoff moves the call into a different policy boundary. The transcript can sound coherent while the agent acts on yesterday's value.
TL;DR: Treat dynamic context as versioned state, not prompt decoration.
- Name the source, trust level, version, effective boundary, and expiry for every context value.
- Test initial, missing, updated, delayed, duplicated, handed-off, and expired context.
- Assert both the expected behavior and the forbidden action after each mutation.
- Save the update receipt, decision turn, tool arguments, spoken response, and final state together.
- Block CI when stale context can change identity, money, compliance, routing, or a backend write.
Scope and sources: This is a proposed test contract with synthetic examples, not a report of measured production outcomes. The provider notes cite public LiveKit, Retell, and Vapi documentation checked on September 25, 2026. Each runtime exposes a different update boundary; adapt the fixture to the boundary your adapter can observe.
Last Updated: September 2026
Related Guides:
- Voice Agent Workflow Testing Runbook - prove tools, state transitions, side effects, and handoffs
- Customer-Specific Workflow Rules Template - load the correct tenant policy before the call
- Structured Output Validation Checklist - tie extracted fields to caller evidence
- Voice Agent Tests as Code - keep context fixtures and assertions reviewable in Git
- Tool Call Contract Testing Template - define allowed tools, arguments, and side effects
- Caller Identity Testing Checklist - separate verified identity from caller-spoken claims
- Handoff and Transfer Testing Runbook - prove context survives a transfer
- OpenTelemetry for Voice Agents - correlate the mutation with the decision and tool call
- Voice Agent Sandbox Testing - verify context-driven writes without touching production
What Is Dynamic Context in a Voice Agent?
Dynamic context is data that influences a voice agent's decisions and may differ by call, caller, turn, workflow state, or time. It includes account status, identity confidence, order state, policy version, available appointment slots, transfer destination, and data returned by tools.
Dynamic context testing verifies which context version was visible when the agent made a decision. Here is the trap: the call ends with
eligibility=approved, but the agent still refused the caller at a decision boundary where the approved version should already have been effective.
That timing distinction is the whole problem. A refusal before the update became effective can be correct; a refusal based on stale state after that boundary is a failure. We call this context time travel: the final record looks correct, but the behavior came from a stale point in the call.
Call variables can be ordinary fixtures when a name or campaign label stays fixed after the greeting. Once a tool, webhook, supervisor, or handoff changes the correct answer mid-call, the test must also model when that change becomes effective.
Define the Context Contract Before You Test
Do not begin with a caller script. Begin with the context contract.
| Field | Question to answer | Sample value | Failure if missing |
|---|---|---|---|
| Key | What stable name identifies the value? | account.eligibility | Tests assert the wrong field. |
| Source | Which system produced it? | account service | Caller speech gets mistaken for backend truth. |
| Trust | How may the agent use it? | verified, claimed, derived, advisory | Untrusted text drives a sensitive tool. |
| Version | Which revision does the value belong to? | account-v18 | A later snapshot hides a stale read. |
| Observed at | When did the runtime receive it? | turn_06 + 184ms | Ordering cannot be reconstructed. |
| Effective boundary | Which decision may first use it? | next LLM turn | The test expects an impossible same-turn update. |
| Expiry | When must the runtime refresh it? | after payment attempt | Old eligibility survives a state change. |
| Allowed uses | Which prompt, branch, or tool may read it? | refund eligibility check | Context leaks into unrelated behavior. |
| Forbidden uses | What must never depend on it? | identity authorization | A convenient field becomes a security boundary. |
Separate values by trust. A phone number from the telephony layer, an account tier from your backend, and a caller saying "I am a premium member" are three different facts even if they contain the same words.
The caller identity checklist covers that boundary in depth. For dynamic context, the rule is simpler: the mutation record must retain the source and trust label alongside the value.
Which Dynamic Context Cases Should Every Voice Agent Test?
Start with seven cases. Add domain-specific cases only after these pass.
| Case | Mutation | Expected behavior | Forbidden behavior | Evidence to retain |
|---|---|---|---|---|
| Initial value | Load plan=standard before greeting | Agent uses standard policy from turn one | No premium promise | Fixture hash, first prompt version, first response |
| Missing value | Omit account_id | Agent asks for safe recovery or hands off | No guessed lookup key | Missing-key event, recovery branch, tool ledger |
| Mid-call update | Change eligibility=pending to approved after tool result | Next eligible decision uses approved state | No stale refusal | Update receipt, effective turn, response, tool args |
| Delayed update | Hold the update until after one decision boundary | Agent follows documented old-state policy, then changes later | No nondeterministic branch | Send time, receive time, decision timestamps |
| Duplicate update | Deliver version v18 twice | One logical mutation, no repeated side effect | No duplicate booking or message | Dedupe key, receipts, side-effect count |
| Handoff | Transfer from intake to specialist with case_id and consent=true | Destination receives allowed context and continues correctly | No lost consent or cross-role leakage | Source and destination versions, handoff receipt |
| Expired value | Advance beyond the value's TTL | Agent refreshes or blocks the action | No write from expired state | Expiry timestamp, refresh attempt, blocked action |
The negative assertion matters. “The agent used the updated value” is weaker than “the agent used version v18 and did not execute the action allowed only by v17.” The second statement tells you what regression the test will catch.
For workflows with customer-specific rules, pair each row with the customer-specific rule matrix. The context fixture should identify the tenant and rule version, but this template owns when that state becomes effective.
A Copyable Dynamic Context Test Fixture
Keep the fixture small enough to review in a pull request. This illustrative format needs an adapter in your test runner; it is not a provider API payload.
id: refund_eligibility_mid_call_updateowner: voice-platformrisk: blockinginitial_context: - key: account.eligibility value: pending source: account-service trust: verified version: account-v17 effective_at: call_start expires_after: payment_attemptmutation: id: eligibility_update_18 trigger: tool_result trigger_ref: verify_recent_payment key: account.eligibility from: pending to: approved source: account-service trust: verified version: account-v18 expected_effective_boundary: next_agent_decision delivery: delay_ms: 347 duplicate_count: 1caller_turns: - "I paid this morning. Can you check again?" - "Am I eligible for the refund now?"assertions: context_version_at_decision: account-v18 spoken_outcome: refund_eligible allowed_tools: - create_refund_review forbidden_tools: - deny_refund - issue_refund_without_confirmation side_effect_count: create_refund_review: 1evidence: retain: - context_mutation_receipt - conversation_turns - trace - tool_call_ledger - final_sandbox_stateThe 347ms delay is illustrative, not a measured provider latency. Here, duplicate_count: 1 means one additional delivery of the same mutation ID. Run the fixture at delays on both sides of the decision boundary, then keep the smallest set that reproduces the risk.
Store this beside your tests-as-code suite. If a provider adapter changes its update semantics, the fixture diff should show which expected boundary changed and why.
How Do You Prove a Context Update Took Effect?
Capture one evidence envelope across the update, agent decision, and downstream action. Do not reconstruct it from separate dashboard screenshots.
{ "testRunId": "ctx_run_2026_09_25_0047", "callId": "synthetic_call_0047", "traceId": "4bf92f3577b34da6a3ce929d0e0e4736", "mutationId": "eligibility_update_18", "contextKey": "account.eligibility", "source": "account-service", "trust": "verified", "previousVersion": "account-v17", "newVersion": "account-v18", "mutationSentAtMs": 18420, "mutationObservedAtMs": 18767, "timebase": "test_runner_monotonic_ms_since_call_start", "effectiveBoundary": "next_agent_decision", "decisionTurnId": "turn_08", "decisionStartedAtMs": 18810, "contextVersionAtDecision": "account-v18", "spokenOutcome": "refund_eligible", "toolCalls": [ { "name": "create_refund_review", "contextVersion": "account-v18", "idempotencyKey": "ctx_run_2026_09_25_0047:create_refund_review" } ], "forbiddenToolsObserved": [], "finalSandboxState": { "refundReviewCount": 1 }}Use the same run ID in the transcript, mutation receipt, trace, tool ledger, and sandbox query. The OpenTelemetry guide explains how to correlate those records across services.
The timestamps above share one test-runner clock. Do not subtract unsynchronized clocks from different services. Capture contextVersionAtDecision at prompt assembly or the equivalent runtime boundary; a tool's later version or an update receipt cannot establish what the model saw. If the adapter cannot observe that boundary, mark the decision evidence unavailable and keep the result inconclusive.
Context mutation evidence is complete only when it connects the versioned update to the exact decision turn and the resulting action. An update receipt without the decision is delivery evidence, not behavior evidence.
If the action writes data, verify the final state using the sandbox testing runbook. A correct spoken response does not prove the backend used the same context version.
Test Ordering, Retries, and Handoffs
Ordering deserves a separate test even when every required field is present.
| Symptom | Likely cause | Diagnostic | Test correction |
|---|---|---|---|
| Agent uses old value for one turn | Update arrived after prompt assembly | Compare observed time with decision start | Define the next legal effective boundary. |
| Agent flips back to old value | Out-of-order delivery | Compare monotonic versions, not timestamps alone | Reject versions older than the current one. |
| Tool runs twice after update retry | Delivery retry lacks a stable mutation ID | Count receipts and side effects by idempotency key | Replay the same update and assert one write. |
| Handoff loses a field | Destination gets a filtered or rebuilt context | Compare source and destination allowlists | Assert required keys and forbidden keys separately. |
| Caller correction is ignored | Conversation-derived value loses to a stale cached field | Inspect source precedence at the decision turn | Test correction, conflict, and confirmation policy. |
| Sensitive value reaches the prompt | Tool-only context was added to model-visible history | Inspect prompt and tool request independently | Split model-visible and tool-only context channels. |
Do not fix these tests with arbitrary sleeps. A sleep can make the race disappear without defining correctness. The test should wait for an observable receipt or intentionally exercise both sides of the boundary.
Handoffs need two assertions: required context arrives, and forbidden context does not. Use the handoff testing runbook for destination, summary, and receipt checks, then add the context version checks from this page.
Where Do Providers Expose Context?
The test contract is shared. The capture point is not.
| Runtime | Documented context surface | Testing implication |
|---|---|---|
| Retell dynamic variables | Per-call variables can be used in prompts, messages, tools, transfer settings, and webhooks; user-provided values are strings. | Test missing values, string parsing, defaults, and every field where substitution affects behavior. |
| Retell live call update | An ongoing call can receive fields_to_override.override_dynamic_variables and call_control.additional_context. | Capture the update receipt and define the first decision allowed to observe the change. |
| LiveKit external data | Initial context can be loaded before session start. In STT–LLM–TTS pipelines, on_user_turn_completed can retrieve context before the next LLM response. | Test initialization and turn-time retrieval separately; do not assume that hook is available for every realtime model path. |
| Vapi variables | Call-start overrides populate variables used by prompts and messages, alongside built-in call and transport identifiers. | Verify the exact call fixture and keep provider-generated identity separate from caller claims. |
None of these docs promises one universal update latency or precedence rule. That contract belongs to your adapter, so write it down and keep a regression case for it.
What This Template Cannot Prove
This template has a hard limit: it cannot tell whether the source data was true. It can only show that the runtime received a version and behaved according to the contract you declared.
That proof has three costs:
- More observability can expose more data. Keep secrets and sensitive fields out of model-visible history when the agent only needs them for a tool request.
- Deterministic timing is limited. Streaming speech, network delivery, and prompt assembly can overlap. Define a visible boundary instead of promising an exact millisecond.
- Fixtures drift. A perfectly repeatable fixture can still encode last month's policy. Give context contracts owners and refresh dates.
Caller corrections and verified backend state need a policy per key. A shipping-address correction and an identity claim have different risks. Encode that choice explicitly instead of hiding it in a global precedence list.
Dynamic Context Release Checklist
- Every context key has a source, trust label, version, effective boundary, expiry, and owner.
- The suite covers initial, missing, updated, delayed, duplicated, handed-off, and expired values.
- Each test names the expected behavior and a forbidden behavior.
- Tool arguments record the context version used at execution time.
- Retried mutations use a stable ID and do not duplicate side effects.
- Handoff tests assert required and forbidden context separately.
- Model-visible context is separated from tool-only or storage-only data.
- Context-driven writes run against a sandbox, mock, or scoped test target.
- Failures retain one run ID across transcript, mutation receipt, trace, tool ledger, and final state.
- Identity, eligibility, money, compliance, routing, and side-effect context cases block CI.
When a real call reveals stale or misordered context, add it to the failed production call regression runbook. Fixing the prompt once is not a regression strategy.
