Time to first token (TTFT) is the time between sending a request to a large language model (LLM) and receiving the first token of its reply. It measures how quickly a model starts to answer, not how long the answer takes.
NVIDIA's benchmarking guide defines it as "the time from query submission to the first received token". On a phone call, TTFT is one part of the latency a caller hears. Two other measures sit on top of it: time to first sentence and end-to-end voice latency.
What does TTFT include?
TTFT covers everything before the model streams its first token. IBM lists four parts: queueing delay, prefill time (processing the prompt), first token generation, and network and API overhead.
IBM calls prefill "often the largest contributor to TTFT, especially for longer prompts". The model reads the full prompt before it writes anything, so a longer prompt means a longer wait. Load matters too. Anyscale's docs note that a server under heavy usage may queue incoming requests, and that wait adds to TTFT.
How do you measure TTFT?
- TTFT = time the first token is received minus time the request is sent
Three choices change the result. State them with any TTFT figure:
- Where the clock sits. Measured at the client, TTFT includes network time. Measured inside the model server, it does not.
- Which percentile. The median (P50) shows the typical request. P90 and P99 show the slow ones.
- How much load. One request on an idle server is the best case. In production, NVIDIA notes, one request's prefill can overlap another's generation.
How do TTFT, time to first sentence and end-to-end latency differ on a phone call?
TTFT stops at the first token. Time to first sentence stops when a full sentence has arrived. End-to-end voice latency covers the whole gap the caller hears.
A caller never hears a token. Retell's custom LLM best practices say: "Retell starts speaking as soon as it has your first complete sentence." The number that matters there is "time to first token plus the time to finish that first sentence".
Retell's latency docs measure end-to-end latency "from when the user stops talking to when the agent starts talking".
| Measure | Clock starts | Clock stops | Leaves out |
|---|---|---|---|
| Time to first token (TTFT) | Request sent to the LLM | First token received | The rest of the first sentence, speech recognition, speech synthesis |
| Time to first sentence | Request sent to the LLM | First complete sentence received | Speech recognition, speech synthesis, telephony |
| End-to-end voice latency | Caller stops talking | Agent starts talking | Nothing in the pipeline. Check whether the figure counts the network leg to the caller |
One naming trap: in LLM benchmarking, end-to-end latency runs to the final token. NVIDIA defines it as "how long it takes from submitting a query to receiving the full response".
Worked example (illustrative numbers). One turn of a call:
- Endpointing and transcription: 300 ms
- LLM first token: 250 ms
- Rest of the first sentence: 150 ms (15 more tokens at 10 ms each)
- Text-to-speech first audio: 150 ms
- Network and telephony: 150 ms
TTFT is 250 ms. Time to first sentence is 250 + 150 = 400 ms. End-to-end voice latency is 300 + 400 + 150 + 150 = 1,000 ms, so TTFT is a quarter of the caller's wait. Our guide to voice AI latency benchmarks uses the same budget.
Why can TTFT mislead when you compare models?
A TTFT figure only compares with another TTFT figure taken the same way. Set one model's TTFT against another's time to first sentence and the first always looks faster. The same goes for model-only time against time measured on a live call, and for a median against a P99.
A model can also send its first token quickly and then stream slowly, which still leaves the caller in silence.
How do you lower TTFT on a voice agent?
Retell's latency troubleshooting guide names model choice and prompt length as the two most common causes of high latency.
- Pick a faster model. Tell-1 is one option built for calls. Per the guide, larger, reasoning-oriented models "are slower to first token".
- Shorten the prompt. "Every token in your prompt adds to the time-to-first-token", on every turn. Prompts beyond roughly 8k tokens become noticeably slower.
- Stream the reply. With a custom LLM, sending the full response in one event "puts the entire generation in the caller's silence".
Retell's Get Call API returns end-to-end, LLM and text-to-speech latency for each call at P50, P90, P95 and P99. The LLM figure runs "to the first speakable chunk received", so it measures speakable text, not the first raw token.
Some models are built around this measure. Tell-1, Retell's own LLM, starts its reply and keeps reasoning while the first phrase plays. It has the lowest median latency on the Retell vCX Benchmark at 76 ms, a rolling 24-hour median time to first sentence (read November 13, 2026). The Retell LLM technical post explains how it works, and model time is only one part of end-to-end voice latency.
Frequently asked questions about time to first token
What is a good time to first token?
There is no single standard. It depends on the model, the prompt length and server load. For a voice agent, judge it inside the end-to-end figure. Retell's docs suggest keeping an agent's estimated latency under 1.5 seconds.
Is TTFT the same as latency?
No. TTFT is the LLM's delay up to its first token. Latency on a call is the full gap between the caller finishing and the agent starting to speak.
Does TTFT include network time?
It depends on where it is measured. NVIDIA says TTFT "generally includes request queuing time, prefill time, and network latency". A model-only figure taken inside the server leaves the network out.
What is the difference between TTFT and time to first sentence?
TTFT stops at the first token. Time to first sentence stops when the first complete sentence has arrived, which is when a voice agent can start speaking.
Measure latency on your own calls
Retell AI is an AI voice agent platform for contact centers. Call History shows latency for each call, broken down by component in the Get Call API. Talk to the Retell team.
Security & compliance for regulated deployments
How Retell handles audits, data retention and access control.
Further Reading
Related terms
Table of contents
