Non-Verbal Cues in TTS: Placement, Testing, and Risks (2026)

TL;DR: Non-verbal TTS cues generate sounds such as laughter, breathing, throat clearing, sighing, coughing, and yawning inside synthetic speech. They can clarify emotion and timing, but they also create interruption, manipulation, and consistency risks. Add one cue at a time, test it blind, and measure meaning, intelligibility, latency, and user response.

Words are only part of spoken communication. A breath can signal hesitation, laughter can soften a sentence, and a pause can change which phrase carries emphasis. Realtime TTS systems increasingly expose these sounds as controllable inputs. Inworld Realtime TTS-2 renders bracketed cues as audio rather than reading the stage direction aloud.

What are non-verbal cues in TTS?

Non-verbal TTS cues are written controls that ask a speech model to produce a sound rather than a word. Inworld documents laughter, breathing, throat clearing, sighing, coughing, and yawning. These events are forms of paralinguistic communication: they modify how a listener interprets the surrounding speech without changing the transcript.

The distinction matters for evaluation. If a system reads [laugh] aloud, it has failed the control. If it produces laughter at the wrong volume or timing, it may also fail the interaction. The output should be judged on whether the cue sounds intentional, preserves the speaker identity, fits the sentence, and avoids obscuring nearby words. A successful effect is not simply an audible sound.

Laughter can signal warmth, irony, discomfort, or dismissal.
Breathing can mark effort, anticipation, or hesitation.
A sigh can communicate frustration, relief, or fatigue.
Throat clearing can signal preparation or uncertainty.
Coughing and yawning require careful contextual use.

Where should non-verbal cues be placed?

Place a cue at the exact point where a human performer would produce it, then test whether it changes the intended phrase. A cue before a sentence frames the entire line; a cue between clauses can mark a turn in meaning; a cue after the words may read as reaction rather than delivery.

Non-verbal tags are the one control that belongs inline. A delivery instruction, by contrast, goes at the start of the text, because mid-line placement of a delivery instruction is unreliable.

Start with punctuation and sentence structure before adding a sound. A pause may solve the timing problem without introducing laughter or breath. When a cue is necessary, use one per test and keep the script, voice, model, and audio format fixed. Multiple cues can interact unpredictably and make attribution impossible. The production prompt library should record approved placements alongside audio examples.

Generate a neutral baseline.
Add one cue at a natural boundary.
Randomize the playback order.
Ask listeners what intent they heard.
Reject cues that distract or change the words.

Inworld Realtime TTS-2 interface placing laughter, breathing, sighing, coughing, throat clearing, and yawning controls within a spoken line

How do cues affect realtime conversation?

Non-verbal sounds consume time and can affect turn-taking. A long laugh may delay the next word, invite interruption, or continue after the user begins speaking. Realtime applications should measure cue duration, cancellation behavior, first-audio latency, and whether queued sounds stop immediately when the user barges in.

LiveKit Agents, Pipecat, Vapi, WebSocket streaming, and other realtime components coordinate playback and interruption around the TTS model. The application should treat non-verbal audio as cancellable speech, not as an uninterruptible sound effect. Test cues at the beginning, middle, and end of an utterance under packet loss and model failover. The user experiences one conversational loop, not separate infrastructure components.

Time from cue request to first sound.
Cue duration and effect on word timing.
Playback stop latency during interruption.
Identity consistency before and after the cue.
Retries or malformed audio events.

When do non-verbal cues reduce quality?

Non-verbal cues reduce quality when they are too frequent, emotionally mismatched, louder than the speech, or used to simulate intimacy the interaction has not earned. They can also reduce accessibility when they obscure information or confuse captions. The safest production rule is sparse use tied to a defined communicative purpose.

A tutor that sighs after a wrong answer may sound judgmental. A support agent that laughs during a billing dispute may appear dismissive. A companion that breathes heavily can cross a user's comfort boundary. These are product and safety failures even when the sound is realistic. Teams should review cues by use case, audience, age, language, and cultural context before exposing them to generative logic.

How should teams test non-verbal speech?

Test non-verbal speech with blinded comparisons on real product scripts. Score whether listeners identified the intended emotion, whether every word remained intelligible, whether the voice identity stayed stable, and whether the cue improved the task. Pair subjective ratings with latency, cancellation, error, retry, and completion measurements.

According to the Artificial Analysis Speech Arena, blind pairwise listening is useful for overall preference, but a cue test needs a narrower question. Ask what emotion or intent the listener perceived, then compare it with the requested direction. Inworld, ElevenLabs, Hume AI, Cartesia, Deepgram, OpenAI, Google, Microsoft, and Amazon take different approaches to expressive speech; use named model versions and identical scripts.

Intent recognition rate.
Word preservation and pronunciation.
Speaker identity stability.
P95 and P99 latency under load.
Task completion and user comfort.

Related Guides

Key Takeaways

Non-verbal cues change meaning and timing without changing the spoken transcript.
Placement determines whether a cue frames, interrupts, or reacts to a line.
Realtime systems must cancel queued non-verbal audio when users interrupt.
Sparse, purpose-driven use is safer than unrestricted generative insertion.
Testing should combine perceived intent, intelligibility, identity, latency, and user comfort.

Frequently Asked Questions

Which non-verbal cues does Inworld TTS-2 support?

Inworld documents laughter, breathing, throat clearing, sighing, coughing, and yawning as bracketed controls. The model renders these cues as sounds rather than spoken words. Current support should be checked in the product documentation because model behavior and supported syntax can change after launch.

Can a TTS cue appear in the middle of a sentence?

Yes, when the model supports inline placement. The cue should sit where a performer would naturally produce it, often between clauses or before a reaction. Test one cue at a time because placement can alter emphasis, pacing, and interruption behavior. A technically rendered sound can still be contextually wrong.

Do non-verbal sounds increase latency?

They add audio duration and may change generation or buffering behavior, but the impact depends on the model, cue, transport, and player. Measure first sound, first spoken word, cue duration, P95 and P99 latency, and cancellation under interruption. Do not assume the model's general TTFB applies unchanged to every cue.

When should an AI voice avoid non-verbal cues?

Avoid them in fixed legal notices, emergency instructions, sensitive health or financial interactions, and any context where a cue could obscure information or imply inappropriate emotion. Products for children or vulnerable users need additional review. Prerecorded audio may be safer when delivery must be deterministic.

Published by Inworld. Product capabilities were checked against Inworld documentation on September 2, 2026. Teams should test cue behavior, accessibility, safety, and cancellation on their own applications before production use.

How Do Non-Verbal Cues Change AI Speech?