TTFT is the metric chosen for voice APIs, and it misleads everyone

Why TTFT is not enough for voice

First token latency (TTFT) measures when the model starts generating, but a text-to-speech (TTS) model cannot produce sound until a full sentence is ready. The gap between these two moments—generation start and first audio—determines whether a voice agent feels conversational or choppy. LiveKit calls the real metric “time to first sentence” (TTFS) and says that this is what the user actually perceives. In short: there are two knobs, not one. TTFT decides when generation begins; token-per-second rate determines how quickly the first sentence finishes. A provider that wins on one and loses on the other will not feel fast.

Voice latency budget: cost of one turn

According to LiveKit’s breakdown, a voice turn consists of STT taking 100-200 ms, an LLM- taking 300-500 ms with streaming, TTS taking 100-200 ms, and the network adding 50-150 ms over WebRTC. The practical end-to-end target is 700 ms to 1.2 seconds. Kwindla Hultman Kramer, a co-founder of Pipecat, recommends aiming for an 800 ms median, with 1,500 ms as an upper bound for a POC. His calculation splits the budget into four roughly 200 ms chunks: media transport and processing, STT with end-of-sentence detection, LLM inference, and TTS. Daily’s human baseline shows a natural conversational response sits around 500 ms, and pauses above 800 ms already feel artificial.

The 700 ms threshold providers must meet

Daily’s benchmark from February 2026 translates this into a concrete LLM requirement: a natural conversation needs voice-to-voice under 1,500 ms, leaving roughly 700 ms of TTFT budget for the text model within the transcription-LLM-voice chain. This number is the bar every provider must meet, not the raw TTFT they publish.

Five methodological notes that change the numbers

First, load shape matters; Artificial Analysis changed the default in March 2026 to 10,000 input tokens instead of 1,000, and longer prompts raise both TTFT and output speed. LiveKit says this is closer to production because real agents push policies, personas, escalation rules, fetched data, and tool schemas into the prompt. Second, the server location is baked in; the tests run from a virtual machine in the us-central1-a region of Google Cloud, and TTFT includes network latency that can help or hurt depending on where providers submit. Third, thinking tokens are counted; under Artificial Analysis’s definition, TTFT for a reasoning model is the first token of the reasoning, not of the answer—these are separate columns. Fourth, measurement side matters; Daily notes that providers sometimes quote an internal TTFT for their stack, whereas Daily measures from request submission to the first usable token from the API. Fifth, runs are not repeatable; Daily is explicit that TTFT varies substantially between runs, and providers swap stacks and sometimes weights without changing model names.

What’s missing from the picture

The source cuts off right at the start of the LLM layer, and the comparative data themselves were not published in the text we received. Without the numbers it is hard to say who meets the 700 ms threshold and who only looks good on paper. The practical takeaway for now: do not rely on a single TTFT figure. Check the full stack, measure from your side, and remember that users hear sentences, not tokens.