Published 09.08.2026
TTS Latency Explained: TTFB, First Playable Audio, and P99
Summarize with:
TL;DR: TTS latency is not one number. Time to first byte measures when a service begins returning data, while first playable audio and end-to-end turn latency include encoding, network transit, buffering, and playback. Compare P95 and P99 under realistic concurrency, record test conditions, and treat vendor server-side figures as one component of the user's wait.
A text-to-speech service can report a 25ms time to first byte and still feel slow to a user. The request must cross a network, the client must receive enough audio to decode, the player may buffer, and upstream speech recognition or language-model generation may already have consumed hundreds of milliseconds. Meaningful TTS evaluation begins by naming the event each timer starts and stops.
Inworld Realtime TTS-2 reports under-100ms P99 server-side TTFB, while TTS-2 Flash reports under 25ms P99. Both exclude network latency. Those figures are useful for sizing the speech layer, but they are not interchangeable with first audible response or complete conversational latency. This guide defines the measurements, shows how to build a latency budget, and sets out a repeatable production test.
What does text-to-speech latency actually measure?
TTS latency measures elapsed time between a defined input event and a defined output event. The input may be a complete text request or the arrival of streamed tokens. The output may be the first byte, first audio chunk, first playable sound, or completed file. A latency claim is meaningless unless both endpoints are stated.
Valid numbers can differ by hundreds of milliseconds because they use different boundaries. Container headers may also create a misleading first byte with no playable audio. A reproducible report should name audio format, client and server regions, connection reuse, warm or cold state, concurrency, and percentile alongside the result. That detail determines whether another team can reproduce it.
Why do P95 and P99 matter more?
Average latency hides the slow turns that users remember. P95 shows the threshold below which 95 percent of requests complete; P99 does the same for 99 percent. Realtime applications should choose a percentile tied to their tolerance for interruptions, pauses, and abandonment, then test it at expected peak concurrency rather than in a single warm request.
An independent Deepgram evaluation guide cites NVIDIA data where P99 first-audio latency reached 380.77ms at eight parallel streams against a 175.62ms average on dedicated A100 hardware, a 2.17-fold spread. The numbers do not benchmark Inworld, but they show why a mean can misrepresent production behavior. Queueing, autoscaling, cold paths, and noisy neighbors often appear first in the production tail under load.
- Report median, P95, and P99 together.
- Load-test at the application's peak concurrent requests.
- Separate warm connections from cold starts.
- Repeat across the regions where users actually live.
- Track errors and retries beside successful latency.
How do networks and buffering change results?
Server inference is only part of the wait. Network round trips add distance and routing delay; TLS handshakes add setup time; audio containers and codecs determine when decoding can begin; client players may buffer before playback. A fast model creates headroom, but application choices decide whether that headroom reaches the user.
Persistent connections reduce repeated setup costs. The WebSocket protocol supports two-way communication over one open connection, allowing text and audio chunks to move without a new HTTP request for each turn. Audio format also matters. Raw PCM can be played as chunks arrive but uses more bandwidth; Opus compresses well for web and mobile but adds codec work. Device behavior can vary across browsers and operating systems.
Measure from the client, not only inside the provider. A server-side TTFB metric cannot see mobile radio conditions, regional routing, browser buffering, or the time required to decode the first playable frame.
Where does TTS fit in voice latency?
A realtime voice turn usually includes endpoint detection, speech recognition, language-model generation, routing, text-to-speech, network transport, and playback. Improving TTS alone cannot fix a slow pipeline, but reducing its tail latency preserves more budget for every upstream component. Teams should instrument each stage and the total user-perceived turn.
The stages can overlap. Streaming language-model tokens into streaming TTS may begin speech before the full response exists, while aggressive endpointing may shorten delay at the cost of interrupting the user. The system needs a product-level target, not a collection of isolated vendor minimums. Instrument one trace ID across the full turn so outliers can be assigned to the correct stage.
How should teams benchmark TTS under load?
A defensible TTS benchmark uses production text, named model versions, fixed voices and audio settings, multiple concurrency levels, repeated requests, and client-side timing. It reports P50, P95, P99, errors, retries, and first playable audio. The test should be rerunnable and should separate model behavior from network conditions.
- Use production turns with numbers, names, URLs, and long sentences.
- Fix voice, model, language, codec, sample rate, and text.
- Run from one client region against named provider regions.
- Test idle and peak concurrency with warm and cold connections.
- Record TTFB, playable audio, completion, failures, retries, and altered words.
- Publish percentiles, samples, dates, and limitations.
Quality belongs in the same experiment. Low-latency output that drops words, misreads an account number, or requires regeneration has failed and consumed cost. Use blinded listening for preference, round-trip transcription for intelligibility, and application-specific pronunciation checks. Optimize latency subject to an acceptable quality floor, not as a standalone race. Users experience speed and correctness together.
How should Inworld latency claims be read?
Inworld reports P99 server-side TTFB below 100ms for Realtime TTS-2 and below 25ms for Realtime TTS-2 Flash. Both figures describe model inference and exclude network transit. They show the speech service's latency contribution, not the complete time from a user's last word to audible response.
Reproduce the published figures on the target workload before procurement. Realtime TTS-2 is positioned for expressive output and steering; Flash is positioned for latency-critical, high-volume, and cost-sensitive applications. A team may accept higher TTFB when expression improves a valuable interaction, or choose Flash when quick turn-taking dominates. The decision should combine user outcomes, error rates, and tail performance under production traffic.
The Artificial Analysis Speech Arena ranks listening preference rather than latency. On September 2, 2026, TTS-2 ranked second and the model listed by Artificial Analysis as TTS-2 Flash research preview ranked sixth. The sources answer different questions: Inworld's latency measurements describe speed under stated conditions, while the Arena describes blind listener preference. Neither replaces a workload-specific test, so production scorecards should track quality and latency separately as production conditions change.
What should a published latency report disclose?
A published latency report should disclose model versions, test date, request corpus, sample count, concurrency, client and server regions, connection state, audio format, timing boundaries, percentiles, errors, retries, and exclusions. Without those fields, another team cannot reproduce the result or know whether it matches its deployment.
The report should separate vendor-provided numbers from independently measured results. State whether the first response byte contained playable audio, whether network time was excluded, and whether requests used warm persistent connections. Name comparison targets rather than describing a leading provider in the abstract. Include raw result files or code when publication rights permit. According to Deepgram's 2026 evaluation guidance, test conditions and tail percentiles are essential to distinguish evidence from advertising.
- Name every model and version.
- Define timer start and stop events.
- Publish P50, P95, and P99.
- Disclose concurrency, regions, and connection state.
- Report errors, retries, and excluded samples.
- Link code, data, and methodology where permitted.
When does lower TTS latency stop helping?
Lower TTS latency stops creating meaningful value when another stage dominates the turn or when speed begins to reduce quality, safety, or comprehension. An under-25ms model cannot compensate for slow endpointing, a delayed language model, aggressive chunking, or client buffering. Optimize the bottleneck that users actually experience.
Long-form narration, prerecorded prompts, and background content generation rarely need the same first-byte target as interruptible conversation. Accessibility and education applications may prefer deliberate pacing once playback begins, even when generation starts quickly. Product teams should run threshold tests: reduce one stage until further improvement no longer changes abandonment, completion, or preference. The result defines a practical target and prevents engineering effort from chasing an invisible gain.
- Batch narration prioritizes throughput and continuity.
- Emergency prompts prioritize determinism and intelligibility.
- Tutoring may require deliberate pacing after fast startup.
- Voice agents prioritize interruption recovery and turn rhythm.
- Social apps balance immediacy against expressive delivery.
What monitoring detects TTS latency regressions?
Latency monitoring needs distributed traces, percentile dashboards, error and retry counts, and synthetic probes from user regions. Every utterance should carry one trace identifier across speech recognition, language-model generation, routing, TTS, network delivery, and playback. Model or infrastructure changes should be compared against a fixed regression corpus before broad rollout.
OpenTelemetry can propagate trace context across services, while application analytics can attach device, region, audio format, model, voice, and experiment cohort. Dashboards should show P50, P95, and P99 by model version and region. Alerting on a global average reacts too late. A canary release should stop when tail latency, error rate, altered-word rate, or user abandonment crosses a predeclared threshold.
How do interruptions change the latency target?
Interruption handling creates two latency problems: stopping current audio quickly and beginning the next response without dead air. A model with fast first-byte performance may still feel slow if playback cancellation, endpoint detection, or state updates lag. Measure barge-in as a complete control loop rather than timing synthesis alone.
LiveKit Agents, Pipecat, and other realtime frameworks coordinate audio transport, turn detection, cancellation, and agent state. The useful measurements are user speech onset to playback stop, user speech end to transcript finalization, and transcript finalization to new audible response. A system should also prevent cancelled audio from continuing in queued buffers. Test interruptions at different points in an utterance, under network loss, and during model failover.
- Playback stop latency after user speech begins.
- Endpointing delay after the interruption ends.
- New response first-audio latency.
- Queued-audio cancellation accuracy.
- State consistency after rapid back-and-forth turns.
How should streaming text enter the TTS layer?
Streaming TTS can begin before the language model finishes a response, but text should enter at boundaries that preserve meaning and pronunciation. Sending tokens too early may create awkward pauses or irreversible mistakes. Waiting for full sentences improves context but adds delay. Most applications need a punctuation-aware or semantic chunking policy.
OpenAI-compatible language models, Inworld Realtime API, LiveKit Agents, Pipecat, and Vapi can produce or orchestrate streaming turns, but the synthesis boundary remains an application decision. A practical policy buffers until a clause or sentence is stable, checks abbreviations and numbers, then sends a playable chunk. Log when text became available and when it entered TTS. Otherwise language-model delay and chunking delay will be misattributed to the speech provider.
- Buffer enough text to preserve meaning.
- Prefer clause or sentence boundaries.
- Normalize dates, numbers, and abbreviations first.
- Avoid changing text after synthesis begins.
- Measure chunking delay separately from model TTFB.
Which provider latency claims can be compared?
Provider latency claims can be compared only when they time the same event under similar conditions. Inworld, ElevenLabs, Cartesia, Deepgram, OpenAI, Google Cloud, Microsoft Azure Speech, and Amazon Polly publish different combinations of model, transport, percentile, region, concurrency, and client assumptions. Normalize those fields before ranking any service.
The Deepgram production evaluation guide recommends named model versions, production concurrency, client and server regions, and a precise definition of first byte or first playable audio. The Artificial Analysis Speech Arena measures blind listener preference, not latency. LiveKit exposes turn-level metrics across the voice pipeline. These sources answer complementary questions and should not be collapsed into one procurement score.
How does geography affect TTS response time?
Geography adds network round-trip time that a server-side model benchmark cannot see. A user in the same cloud region may receive audio quickly, while a mobile user crossing continents may wait materially longer. Test from every major user market and record both client and provider regions rather than averaging them together.
AWS, Google Cloud, Microsoft Azure, Cloudflare, and other networks publish regional footprints, but physical distance is only one variable. Internet routing, carrier congestion, mobile radio state, and TLS setup can change each request. Persistent WebSocket connections reduce repeated handshakes, while regional endpoints reduce distance. Production dashboards should segment P50, P95, and P99 by country, device, network class, and server region so a strong aggregate does not conceal a weak market.
- Run probes from every launch region.
- Separate Wi-Fi, 4G, and 5G traffic.
- Record connection reuse and TLS setup.
- Compare provider regions with actual user density.
- Alert on regional P99 regressions, not global averages.
How do audio formats change first playback?
Audio format changes how quickly a client can decode and play the first useful frame. Raw PCM avoids container parsing but uses more bandwidth. Opus reduces bandwidth and suits web or mobile delivery, while WAV and MP3 introduce different header and buffering behavior. Benchmark the format the product will actually ship.
Inworld's audio documentation lists PCM, WAV, Linear16, Opus, MP3, and the telephony formats. PCM can stream raw samples over WebSocket; Opus supports low-bandwidth interactive playback; telephony encodings fit 8kHz voice channels. The same model can show different first-playable-audio results across these encodings. Record sample rate, bit rate, container, chunk size, and player implementation in every benchmark on target devices before deployment.
Which latency target should a product choose?
Choose a latency target from the interaction, not from a universal threshold. A spoken acknowledgment, game interruption, guided exercise, and long-form narration tolerate different delays. Set an end-to-end target, allocate a budget to each stage, and define the percentile and concurrency at which the target must hold.
Voice agents with frequent interruption need enough headroom to stop and restart naturally. Social and companion products may accept slightly longer responses when expressive delivery improves the relationship. Narration can tolerate a slower first chunk if throughput and continuity remain strong. The correct target is the fastest response that preserves quality, safety, and economics for the specific turn, rather than the smallest vendor number available.
- Define the maximum user-perceived pause for each turn type.
- Reserve budget for endpointing, STT, LLM, TTS, and playback.
- Set P95 and P99 goals at peak load.
- Attach quality and error thresholds to the speed target.
- Review regional results separately instead of averaging them together.
Related Guides
- Inworld Realtime TTS-2 and TTS-2 Flash
- Inference Latency Optimization
- How to Evaluate TTS Models for Realtime Conversational AI
- Cascaded vs Speech-to-Speech Voice Architecture
- How to Build an STT-LLM-TTS Voice Pipeline
Key Takeaways
- A TTS latency claim is interpretable only when its start event, stop event, percentile, concurrency, region, and audio format are named.
- Server-side TTFB excludes network transit, client buffering, decoding, playback, and upstream speech or language-model delays.
- P95 and P99 under peak concurrency reveal production risk that averages and single warm requests conceal.
- Streaming transport and codec choices determine whether fast model inference becomes fast first playable audio.
- Optimize latency subject to quality, error, and user-outcome thresholds rather than treating speed as an isolated race.
Frequently Asked Questions
What is TTFB in text-to-speech?
Time to first byte is the delay between a defined request event and the arrival or transmission of the first response byte. Providers may measure it inside the server or from a client. TTFB does not necessarily mean playable audio because the first bytes may contain headers or an incomplete encoded frame.
What is the difference between TTFB and first playable audio?
TTFB stops when the first response byte appears. First playable audio stops when the client has received and decoded enough data to begin sound. Network transit, container headers, codec frames, buffering, and device playback can make first playable audio materially slower than a server-side TTFB figure.
Why should TTS benchmarks report P99 latency?
P99 shows the threshold below which 99 percent of measured requests fall, exposing slow tail behavior that an average hides. Realtime products are often damaged by occasional long pauses rather than the median turn. P99 should be reported at expected peak concurrency, with errors, retries, regions, and warm or cold conditions.
Does WebSocket automatically make TTS realtime?
No. WebSocket removes repeated request setup and supports continuous two-way streaming, but model inference, queueing, audio encoding, network distance, client buffering, and playback still contribute delay. It is a useful transport for realtime speech, not a guarantee. Teams must measure the complete path from the user's turn to audible output.
How fast are Inworld TTS-2 and TTS-2 Flash?
Inworld reports under-100ms P99 server-side time to first byte for Realtime TTS-2 and under 25ms P99 for Realtime TTS-2 Flash. The figures exclude network latency and do not represent full conversational response time. Teams should reproduce them from target regions, under expected concurrency, using the intended voices, languages, and audio formats.
Published by Inworld. Inworld latency specifications were checked on September 2, 2026 and are server-side P99 TTFB figures excluding network latency. Independent benchmark guidance is attributed to Deepgram and Artificial Analysis. Illustrative budgets are educational, not guaranteed performance. Teams should run client-side tests under their own regions, concurrency, audio settings, and workloads.