Best Speech-to-Speech APIs in 2026: Architecture, Latency, and Code

What "speech-to-speech" actually means

Production checklist

• Streaming input. Audio arrives continuously via WebSocket or WebRTC. No waiting for the user to finish before processing starts.
• Fast endpointing. VAD plus end-of-turn detection to handle interruptions and decide when to respond.
• Streaming output. Audio chunks arrive as they're synthesized, not after the full response is buffered. This is what makes sub-300ms time-to-first-audio possible.
• Cancellation. Barge-in requires stopping TTS playback, canceling in-flight TTS generation, canceling LLM generation, and resetting stream state. If any of those steps is missing, the agent talks over users or "finishes the old thought" after being interrupted.
• Observability. Per-turn traces with STT, LLM, and TTS timings broken out separately. Without this, you're guessing which component is causing latency spikes.

Reference architecture

Audio in (WebSocket / WebRTC)
  └─ VAD + endpointing
      └─ Streaming STT (partials)
          └─ LLM (streaming tokens)
              └─ Text chunking (sentence-level for TTS)
                  └─ Streaming TTS (audio chunks)
                      └─ Client audio playback
                          └─ Cancellation path (on barge-in: back to VAD)

The 6 Best Speech-to-Speech APIs in 2026

1. Inworld AI — Realtime API + TTS

const ws = new WebSocket(
  `wss://api.inworld.ai/api/v1/realtime/session?key=voice-${Date.now()}&protocol=realtime`,
  { headers: { Authorization: `Basic ${process.env.INWORLD_API_KEY}` } }
);

ws.on('message', (raw) => {
  const msg = JSON.parse(raw.toString());
  if (msg.type === 'session.created') {
    ws.send(JSON.stringify({ type: 'session.update', session: { ... } }));
  }
});

// Send audio chunks as they arrive from the microphone
function sendAudio(audioChunk) {
  ws.send(JSON.stringify({
    type: 'input_audio_buffer.append',
    audio: audioChunk // base64-encoded PCM16
  }));
}

// Receive audio chunks as the agent responds
ws.on('message', (event) => {
  const msg = JSON.parse(event.toString());
  if (msg.type === 'response.output_audio.delta') {
    playAudio(msg.delta); // base64 PCM16
  }
});

• TTS 1.5-Max: sub-250ms P90 time-to-first-audio
• TTS 1.5-Mini: sub-130ms P90 (trades some quality for speed)
• Pricing: $10/1M characters (Max), $5/1M characters (Mini). At 100M chars/month, that's $1,000 vs. $20,600 on ElevenLabs.

2. Deepgram — Flux STT + Aura TTS

3. Hume AI — EVI (Empathic Voice Interface)

4. OpenAI Realtime API

5. ElevenLabs — Voice Conversion and Content Production

6. Cartesia — Sonic Streaming TTS

Summary Table

How to choose

Why Inworld Is the Default Choice for Voice Agents

FAQs

What is a speech-to-speech API?

How do I choose the right speech-to-speech API?

Is Inworld AI better than ElevenLabs for voice agents?

How does speech-to-speech relate to TTS?

If I'm already using OpenAI for LLM, should I use the OpenAI Realtime API?

How quickly can a voice agent go live with Inworld?

How easy is it to switch from the OpenAI Realtime API to the Inworld Realtime API?

What's the best ElevenLabs alternative for voice agents?