Skip to content
Talk to usGet Started

Speech to speech
built for people

One connection listens, reasons and talks back.

Get an API keyTry it now ↗

Try it now

Open the speech-to-speech playground ↗

Pick a voice and press Talk. The agent says hello first, then listens.

ListeningReasoningTalking
InworldTime to first token –– tok/s
Typical providerTime to first token –– tok/s
Same model, same prompt, two providers.

Pick a use case and press Speak, then read the line aloud.

Try saying "My account was disabled at 8:30 this morning."

ListeningTranscribing

Ask your agent to add Inworld.

Paste it into your agent

How it works

One connection carries the whole conversation.

  1. 01Open a session.

    Connect one WebSocket or WebRTC, pick a voice and a model, and add your tools.

  2. 02Your user talks.

    They speak naturally. The session hears when they're done, and how they said it.

  3. 03The agent acts.

    It reasons, calls your tools and answers out loud in a few hundred milliseconds, under a second with a tool call.

  4. 04Interrupt anytime.

    Talk over it and it stops, keeping only what the user heard.

Moving a dinner booking, with an interruptionOne recorded speech-to-speech session, played in real time
ListeningReasoningTalking
17.7 s
0:17
Connection
0 s2 s4 s6 s8 s10 s12 s14 s16 s
VoicesUserAgentConnectionlistening (the turn ends)reasoningyour tooltalking (first audio)
User

Hey, um, can you move my dinner at Luca tonight from seven to eight thirty?

Agent

Sure, your window table at Luca is all set for eight thirty.

The user started talking at 7.91 s, so playback stopped and the reply was truncated to what they heard.
User

Oh, wait. Sorry, there will be five of us.

Agent

No problem, you're all set for a corner booth.

The agent called your app's move_reservation tool, then spoke the result.
  1. 1.32User starts speaking speech_started
  2. 5.30Last word
  3. 5.71Turn closes with the final transcript committed
  4. 6.09Tool call move_reservation {"new_time":"20:30","restaurant":"Luca"}
  5. 6.20Your app returns the result {"table":"window table"}
  6. 6.38First reply text
  7. 6.53First audio
  8. 7.91User talks over the reply
  9. 8.11Reply stops truncate
  10. 10.88Last word
  11. 11.21Turn closes with the final transcript committed
  12. 11.44Tool call move_reservation {"party_size":5,"restaurant":"Luca"}
  13. 11.54Your app returns the result {"table":"corner booth"}
  14. 11.74First reply text
  15. 12.13First audio

Speech to speech

Speech to speech docs ↗

Your users talk, your app answers out loud. One connection listens, reasons, calls your tools and talks back.

204 msfrom the end of their turn to your agent's voice
1 WebSocketfor audio, transcript and tool calls
Mid-callswap the LLM or the voice
0.7¢a minute of spoken reply
A man on a city crossing, talking into his phone
speech to speech · one turn, stage by stage
Your appMicrophone

Sends PCM16 at 24 kHz, or G.711 at 8 kHz from a phone call.

WebSocket or WebRTC
ListeningHears the words and how they were said

The final transcript carries the speaker's emotion, age, accent and vocal style.

64 ms last word to transcript
ReasoningWrites the reply and calls your tools

You pick the model, and you can change it mid-session.

159 ms to first token
TalkingSpeaks the reply

On Realtime TTS-2, it follows a direction the LLM writes in plain words at the start of each reply.

58 ms to first audio on Realtime TTS-2 Flash
Your appSpeaker

Playback stops the moment the user talks over it.

History keeps what they heard
Listeninglast word to transcript, streaming64 ms
Reasoningto first token159 ms
Talkingto first audio58 ms
What the user hearslistening, reasoning and talking281 ms
0100200300 ms
Semantic turn detection

Semantic VAD reads the transcript to decide when the user has finished, and you set how eager it is.

Interruptions

When the user talks over a reply, the audio stops and the history keeps only what they heard.

Tool calling

Declare functions at the start or add them mid-session, and the agent speaks the result in the same turn.

Change settings mid-session

A session.update changes the LLM, the voice or the tools, and the next reply uses the new setting.

OpenAI Realtime compatible

Events and session config match the OpenAI Realtime API, so moving over means a new endpoint and key.

Browser and phone audio

Connect over WebSocket or WebRTC, and take phone audio as G.711 at 8 kHz from Twilio.

Listening, reasoning and talking

Pricing ↗

Realtime STT-1 Speech to text

Streams the transcript with each speaker's emotion, age and accent. Speaker diarization is new on the STT API.

13×sooner transcript than Google Chirp 2
4voice traits per transcript
$0.10an hour on paid plans

Inference LLM

Runs the open models that write each reply and call your tools. Gemma 4, GLM 5.3, DeepSeek V4 and Qwen3.8 all ran in live sessions.

2.8×faster first token on Gemma 4 31B
8hosted LLMs in live sessions
$0.13per 1M input tokens

Realtime TTS-2 Text to speech

Speaks with direction in about 500 languages and dialects. Speech Reasoning is in beta in the playground, and language tags switch language mid-sentence.

50msto first audio with Realtime TTS-2 Flash, at its fastest
500+languages and dialects
$12.50per 1M characters on Growth

Start building

WebSocket reference ↗

Each part of the session bills at its own rate, with no speech-to-speech fee on top. Paid plans cut the TTS rate in half.

1.25¢a minute of spoken reply with Realtime TTS-2 on Growth, 2× lower than on demand
0.7¢a minute of spoken reply with Realtime TTS-2 Flash on Growth, against 1.5¢ on demand
0.17¢a minute of the user's speech with Realtime STT-1 on paid plans
from $0.07LLM Inference, open models per 1M input tokens
wss://api.inworld.ai/api/v1/realtime/session
// npm install ws
// INWORLD_API_KEY=<your key> node client.mjs
import WebSocket from 'ws';
import fs from 'node:fs';

const ws = new WebSocket(
  `wss://api.inworld.ai/api/v1/realtime/session?key=demo-${Date.now()}&protocol=realtime`,
  { headers: { Authorization: `Basic ${process.env.INWORLD_API_KEY}` } }
);
const audio = [];

ws.on('message', (data) => {
  const event = JSON.parse(data);

  if (event.type === 'session.created') {
    ws.send(JSON.stringify({
      type: 'session.update',
      session: {
        type: 'realtime',
        model: 'inworld/models/gemma-4-31b-it',
        instructions: 'You are a friendly voice assistant. Reply in one short spoken sentence.',
        output_modalities: ['audio'],
        audio: {
          input: { transcription: { model: 'inworld/inworld-stt-1' } },
          output: { model: 'inworld-tts-2', voice: 'Sarah' },
        },
      },
    }));
    ws.send(JSON.stringify({
      type: 'conversation.item.create',
      item: {
        type: 'message',
        role: 'user',
        content: [{ type: 'input_text', text: 'Suggest a name for a small orange cat.' }],
      },
    }));
    ws.send(JSON.stringify({ type: 'response.create' }));
  }

  if (event.type === 'response.output_audio.delta') audio.push(Buffer.from(event.delta, 'base64'));
  if (event.type === 'response.output_audio_transcript.done') console.log('Agent:', event.transcript);
  if (event.type === 'error') console.error(event.error);

  if (event.type === 'response.done') {
    const pcm = Buffer.concat(audio); // PCM16, 24 kHz, mono
    fs.writeFileSync('reply.wav', Buffer.concat([wavHeader(pcm.length), pcm]));
    console.log(`Saved reply.wav (${(pcm.length / 48000).toFixed(1)} s)`);
    ws.close();
  }
});

function wavHeader(bytes, rate = 24000) {
  const h = Buffer.alloc(44);
  h.write('RIFF', 0); h.writeUInt32LE(36 + bytes, 4); h.write('WAVE', 8);
  h.write('fmt ', 12); h.writeUInt32LE(16, 16); h.writeUInt16LE(1, 20); h.writeUInt16LE(1, 22);
  h.writeUInt32LE(rate, 24); h.writeUInt32LE(rate * 2, 28); h.writeUInt16LE(2, 32); h.writeUInt16LE(16, 34);
  h.write('data', 36); h.writeUInt32LE(bytes, 40);
  return h;
}

Set INWORLD_API_KEY to your key from platform.inworld.ai.

DocsSpeech to speech ↗WebRTC ↗Tool calling ↗Moving from OpenAI ↗Phone calls ↗Examples ↗

Speech to speech built for people runs on Inworld. Who are you building for?

Questions

What is speech to speech?

Inworld AI speech to speech takes the user's speech and answers in speech over one WebSocket or WebRTC connection. It listens to the user, reasons with the LLM you choose and calls your tools, and talks back in any Realtime TTS-2 voice, including voices you clone or design, with turn detection and interruptions handled inside the session. It follows the OpenAI Realtime protocol. Phone calls work through Twilio Media Streams, in G.711 μ-law or A-law at 8 kHz.

How fast does Inworld speech to speech reply?

First audio from Inworld AI speech to speech arrived as little as 204 ms after the turn closed, the fastest of 37 turns, with a median of 462 ms over 17 measured turns. The wait for the turn to close depends on the turn-detection setting you choose; with semantic VAD on high eagerness it was a median 954 ms after the user's last word.

How do turn-taking and interruptions work?

Semantic VAD in Inworld AI speech to speech decides from the STT stream when the user has finished, at low, medium or high eagerness. Server VAD uses silence instead, and you can override the end-of-turn silences. With interrupt_response on, the server stops the reply as soon as the user starts talking, and conversation.item.truncate keeps only what they heard.

Which LLMs can I use with Inworld speech to speech?

You set the LLM for Inworld AI speech to speech in session.model, such as inworld/models/gemma-4-31b-it, and can change it mid-session with session.update. The next reply uses the new model. Hosted open models including Gemma 4, GLM 5.3 Flash and DeepSeek V4 Flash all work in live sessions. Set the model on every session, since an unset model falls back to a default.

Can an Inworld speech to speech agent call my functions?

Yes, Inworld AI speech to speech supports function calling, so a voice agent can call your tools in the middle of a conversation. Declare tools in session.update, at the start or later in the session. Calls arrive as response.function_call_arguments.done; return a function_call_output item and the session starts the spoken follow-up itself. Set providerData.auto_tool_response to false for OpenAI behaviour.

Can I easily migrate from the OpenAI Realtime API?

Yes, Inworld AI speech to speech uses the same event types, session config and event shapes as the OpenAI Realtime API, so a client moves over with few changes. Change the endpoint to wss://api.inworld.ai/api/v1/realtime/session with a session ID and protocol=realtime, use your Inworld key, and add providerData only if you want Inworld extensions such as voice profiles, backchannels or memory.

What is the cost of using speech to speech?

There is no separate fee for Inworld AI speech to speech, so you pay for each part at your plan's rates. At about 1,000 characters a minute of speech, Realtime TTS-2 is 2.5¢ a minute on demand and Realtime TTS-2 Flash 1.5¢, Realtime STT-1 is 0.25¢ a minute, and LLM tokens are billed at the model's rate. Paid plans lower the rates, and on the Growth plan Realtime TTS-2 Flash falls to $7 per million characters, about 0.7¢ a minute.

What does the Ultravox acquisition mean for speech to speech?

Ultravox is now part of Inworld AI, acquired to advance realtime voice AI, including speech to speech. Ultravox brings experience in speech understanding, reasoning and task completion, and in managing turn-taking and interruptions. Speech to speech today is one connection that listens, reasons and talks, with the LLM you choose. Built-in Inworld voices on Ultravox now run on Realtime TTS-2.