Time to First Word (TTFW)

Time to First Word (TTFW) measures the delay from when a user stops speaking until the first word of the agent's response becomes audible. Unlike end-to-end latency which measures complete response time, TTFW captures the perceived responsiveness that most impacts user experience. Human conversations typically have TTFW of 200-400ms, and matching this creates the illusion of natural dialogue. TTFW is particularly critical because humans judge responsiveness within the first 500ms of silence - beyond this threshold, conversations feel increasingly artificial.

TTFW is the single most important metric for perceived conversation quality. MIT research shows that users form judgments about system responsiveness within 400ms, and these first impressions strongly influence overall satisfaction. In A/B tests conducted by major voice platforms, reducing TTFW from 1.2s to 600ms increased task completion rates by 23% and reduced hang-ups by 31%. For customer service applications, every 100ms reduction in TTFW correlates with a 2-point increase in CSAT scores. Voice agents with TTFW under 700ms receive 'natural' ratings 3x more often than those above 1 second.

TTFW optimization requires parallel processing and intelligent prefetching. Modern systems achieve low TTFW by starting TTS synthesis before the complete LLM response is available, using streaming architectures that process tokens as they're generated. Voice Activity Detection (VAD) with aggressive endpointing can save 200-300ms by detecting speech end quickly. Some platforms implement speculative execution, beginning to generate likely responses before the user finishes speaking. Advanced implementations use prosody analysis to predict sentence boundaries and start processing mid-utterance. The key is overlapping operations: while the final words are being transcribed, the LLM begins processing, and TTS starts synthesizing the opening phrase.

High TTFW often results from sequential processing where each step waits for the previous to complete fully. VAD settings that are too conservative add 300-500ms waiting for silence confirmation. Using non-streaming LLM APIs forces waiting for complete responses before TTS can begin. Cold starts are particularly problematic for TTFW, with first interactions taking 2-3x longer. Network jitter and packet loss can cause stuttering starts even when average latency is acceptable. Many implementations fail to optimize the critical path, processing optional enrichments (like sentiment analysis) before generating the response.

Implement streaming at every layer: streaming STT, streaming LLM inference, and streaming TTS playback. Configure aggressive VAD with 200-400ms end-of-speech detection. Use smaller, faster models for the first 1-2 sentences, then switch to larger models for complex responses. Pre-generate common opening phrases ('Sure, I can help with that') and play them while processing the full request. Deploy TTS models at the edge to minimize network delay for initial audio. Monitor TTFW separately from overall latency, setting alerts for p90 exceeding 800ms. Consider using optimistic UI patterns - play acknowledgment sounds immediately while processing begins.