Cross-lingual voice cloning in 2026: clone once, speak 100+ languages

Last updated: May 28, 2026

What is cross-lingual voice cloning?

How does cross-lingual cloning work under the hood?

What does voice identity preservation actually mean?

• Timbre. The harmonic signature that makes one human voice distinguishable from another.
• Cadence and rhythm. The speaker's pacing pattern, after adjusting for language-specific prosody.
• Vocal style. Warmth, energy, articulation habits, and emotional baseline.
• Pitch range and contour. Within what the target language allows, the speaker's natural register is held.

What is the Inworld AI voice cloning API contract?

How do I clone a voice and use it across languages? (Python)

Step 1: clone the voice

import requests
import base64
import os

# pip install requests
INWORLD_API_KEY = os.environ["INWORLD_API_KEY"]  # Basic auth, base64(key:secret)

# Step 1: clone a voice from 5-60 seconds of original human-recorded audio.
# Use WAV/MP3/WEBM up to ~4MB. Multiple samples are supported.
with open("reference_voice_english.wav", "rb") as f:
    sample_b64 = base64.b64encode(f.read()).decode("utf-8")

clone_response = requests.post(
    "https://api.inworld.ai/voices/v1/voices:clone",
    headers={
        "Authorization": f"Basic {INWORLD_API_KEY}",
        "Content-Type": "application/json",
    },
    json={
        "displayName": "BrandVoice_EN",
        "langCode": "EN_US",            # Language of the reference sample
        "voiceSamples": [
            {"audioData": sample_b64}
        ],
        "description": "Calm, friendly product voice",
        "audioProcessingConfig": {
            "removeBackgroundNoise": True
        }
    },
    timeout=60,
)
clone_response.raise_for_status()

cloned_voice_id = clone_response.json()["voice"]["voiceId"]
print(f"Cloned voiceId: {cloned_voice_id}")

Step 2: synthesize across languages

import requests
import base64
import os
import json

INWORLD_API_KEY = os.environ["INWORLD_API_KEY"]

# Step 2: use the cloned voiceId in TTS-2 across multiple languages.
# The same voiceId preserves identity in every target language.
TARGETS = [
    ("en", "Welcome back. Here is a summary of your account."),
    ("es", "Bienvenido de nuevo. Aqui tienes un resumen de tu cuenta."),
    ("ja", "お帰りなさいませ。アカウントの要約をお伝えします。"),
    ("fr", "Bon retour. Voici un resume de votre compte."),
]

for lang, text in TARGETS:
    response = requests.post(
        "https://api.inworld.ai/tts/v1/voice",
        headers={
            "Authorization": f"Basic {INWORLD_API_KEY}",
            "Content-Type": "application/json",
        },
        json={
            "voiceId": "YOUR_CLONED_VOICE_ID",   # From Step 1
            "modelId": "inworld-tts-2",          # Research preview
            "text": text,
            "language": lang,                     # BCP-47 hint
            "deliveryMode": "BALANCED",          # STABLE / BALANCED / CREATIVE
            "audioConfig": {
                "audioEncoding": "MP3",
                "sampleRateHertz": 24000
            }
        },
        timeout=30,
    )
    response.raise_for_status()
    audio = base64.b64decode(response.json()["audioContent"])
    with open(f"out_{lang}.mp3", "wb") as f:
        f.write(audio)
    print(f"Wrote out_{lang}.mp3")

Streaming: same flow, NDJSON response

import requests
import base64
import json
import os

INWORLD_API_KEY = os.environ["INWORLD_API_KEY"]

# Streaming returns NDJSON: each line is {"result": {"audioContent": "base64..."}}
# Parse line by line and decode base64 before writing audio.
response = requests.post(
    "https://api.inworld.ai/tts/v1/voice:stream",
    headers={
        "Authorization": f"Basic {INWORLD_API_KEY}",
        "Content-Type": "application/json",
    },
    json={
        "voiceId": "YOUR_CLONED_VOICE_ID",
        "modelId": "inworld-tts-2",
        "text": "こんにちは。今日は言語を超えて同じ声で話しています。",
        "language": "ja",
        "deliveryMode": "BALANCED",
        "audioConfig": {
            "audioEncoding": "MP3",
            "sampleRateHertz": 24000
        }
    },
    stream=True,
    timeout=60,
)
response.raise_for_status()

with open("streamed_ja.mp3", "wb") as out:
    for line in response.iter_lines():
        if not line:
            continue
        data = json.loads(line)
        chunk = base64.b64decode(data["result"]["audioContent"])
        out.write(chunk)
print("Wrote streamed_ja.mp3")

What languages does a single cloned voice cover?

How does Inworld compare to ElevenLabs, Cartesia, and Resemble on cloning?

• ElevenLabs PVC remains the quality benchmark for long-form, non-realtime English content. If your workload is audiobook production or studio dubbing and latency does not matter, that is where to test first.
• Cartesia is genuinely fast at streaming latency. For applications where streaming TTFB is the primary constraint and the language list is in scope, it competes well.
• Resemble's repositioning around deepfake detection is a market signal worth taking seriously. Detection and provenance are becoming part of the production cloning conversation, not a separate compliance track. We address consent and detection in the ethics section below.
• PlayHT also offers cloning. Their site was not consistently reachable during the May 2026 audit, so we are omitting specific claims rather than guess. Re-verify before any provider-specific decision.

Should I use AI-generated audio or original human recordings for cloning?

What are the consent and ethics guardrails?

• Documented consent with explicit scope. Languages, use cases, duration, and revocation terms in writing before reference audio is uploaded. For employees, branded voices, and any public-figure recording, this is non-negotiable.
• Original audio only. Cloning on top of someone else's generated voice is both a quality problem and a consent problem.
• Provenance and detection. Watermark synthetic audio when the use case warrants. Resemble AI has built DETECT-3B Omni around deepfake detection precisely because the production-cloning industry needs detection plus provenance as standard infrastructure, not an afterthought.
• Data residency and ownership. Read the provider terms before uploading. Some providers claim broad, perpetual rights to uploaded voice data. For enterprise customers with branded or celebrity voices, negotiate data terms first.

How do I integrate a cloned voice into a realtime voice agent?

When should I pick which provider?

• Realtime voice agents and companions across multiple languages. Inworld AI. One clone, 15 GA + 90+ experimental languages, full-stack integration (TTS, STT, Realtime API, Router), 60-second IVC reference window.
• Long-form English audiobooks and studio dubbing where latency does not matter. ElevenLabs PVC. The professional service is still the benchmark for top-end English fidelity.
• Streaming-latency-sensitive single-language consumer apps. Cartesia is competitive on streaming TTFB and worth side-by-side testing.
• Regulated industries that require provenance, watermarking, and on-premise deployment as primary requirements. Resemble AI, with detection front-and-center.
• Multilingual product where consent and language coverage are both load-bearing. Inworld AI. Use the 2-step API, capture documented consent with scope per language, retain original audio in your own storage, and consider watermarking sensitive synthetic output.

FAQ