Up to 7.8×faster
Time to first token, against other providers of the same model.
Gemma 4 26BUse it →p95 time to first token, against the median other provider
Bedtime story
Mood check
MiMo 2.6 Pro
Language tutor
Filter and moderate
Voice note reply
DeepSeek V4.1 Flash
Route choice
Which way next
Safety check
Companion reply
GLM 5.3 Flash
Morning check-in
Memory and summaries
A story they steer
you
Your own model
Stretch coach
Classify and route
Clef
Pace coach
Score and rank
Opening scene
Next chapter
Inference built for people
The models consumer apps run on, optimized from weights to kernels.
Tens of trillions of tokens a monthUnder 150 ms to first token
Bedtime story
Language tutor
Companion reply
A story they steer
Stretch coach
Pace coach
Mood check
Filter and moderate
Voice note reply
Route choice
Which way next
Safety check
Morning check-in
Memory and summaries
Classify and route
Score and rank
MiMo 2.6 Pro
DeepSeek V4.1 Flash
GLM 5.3 Flash
you
Your own model
Clef
Lower price, faster time to first token, five nines.
Gemma 4 26BUse it →
2.8×faster
Median time to first token, at the same eval scores.
Gemma 4 31BUse it →Median time to first token (the wait before a reply's first token arrives), 943 to 336 ms at production load
99.999%successYes, five nines is hard. Our voice models and Priority LLM serving hold it.This figure: GLM 5.3 Flash, more than 20 million requests in 30 days.
Of more than 20 million requests.
GLM 5.3 FlashUse it →Requests that finished without a server error, timeout or capacity rejection, last 30 days
For teams building for people
Case studies →
Cut its AI bill by more than 95%.
19 daysto a million users
Team of fivebuilt it on Inworld
Moved half its engine traffic in a week.
Tens of millionsinference calls in six weeks
1 dayto move its first production task
Switch. Scale. Make it yours.
Your app on Inworld, from first call to millions of conversations
03 Make it yoursYour weights, fine-tuned on your traffic, on capacity reserved for you.
Models for each consumer task.
Low time to first token on every turn.
Shape3K to 4K tokens in, about 200 out, every turn
Companion replyLanguage tutorVoice note replyDaily check-inReasoning and tools
Dialogue · one request
POST /v1/chat/completionsmodel: gemma-4-31b-it
// a companion reply, one short sentence
{"messages": [
{"role": "system", "content": "You are a warm companion. Reply in one short sentence."},
{"role": "user", "content": "Long day. I finally finished the report my boss kept pushing back."}]}↓
{"content": "I'm so proud of you for getting that done; now it's time to truly relax!"}Gemma 4 31B's time to first token is up to 3.3× faster than the median provider's.
Hosted on Inworld
Gemma 4 31B Dialogue
2.8× faster median time to first token at production load, 943 to 336 ms after one week of optimization, at the same eval scores.
256K context/$0.13 in/$0.38 out
Gemma 4 26B Classification
Nearly 40% lower input price than the median provider. Up to 90% cache hits on repeated prompts.
256K context/$0.07 in/$0.34 out
MiMo 2.6 Pro Narrative
Text, image and audio in, with 1M tokens of context. Cached input at $0.0036 per 1M tokens.
1M context/$0.425 in/$0.87 out
DeepSeek V4.1 Flash Dialogue
Cached input at $0.006 per 1M tokens, 96% below its $0.16 input price.
128K context/$0.16 in/$1.20 out
Select models
All models →ModelGood forTime to first tokenOutput speedIn / out / cached per 1M tokensIn / out / cached per 1M tokens
Clef—$0.15 input only→GLM 5.3 Flash310 ms608 ms$0.078 / $0.50 / $0.03Lowest input price of 24 providers that don't train on your data→Gemma 4 31B224 ms751 ms172 tok/s56 tok/s$0.13 / $0.38 / $0.065→Gemma 4 26B214 ms426 ms237 tok/s120 tok/s$0.07 / $0.34 / $0.035→MiMo 2.6 Pro284 ms370 ms273 tok/s47 tok/s$0.425 / $0.87 / $0.0036Lowest input price of 5 providers that don't train on your data→DeepSeek V4.1 Flash—$0.16 / $1.20 / $0.006Lowest input price of 22 providers that don't train on your data→ModelGood forSpeedPricePrice
Realtime TTS-2as low as 90 ms to first audiodown to $10 per 1M characters on Enterprise→Realtime TTS-2 Flashas low as 50 ms to first audiodown to $5 per 1M characters on Enterprise→Realtime STT-1under 70 ms to a final transcript$0.10 per hour of audio on paid plans→ModelGood forOutputPricePrice
▷Wan 3.0AlibabaText or image to videofrom $0.05 a second→▷MiniMax H3Text or image to videofrom $0.08 a second→Inworld fastest other provider of the same model median other provider
One model. Four ways to run it.
Set the service tier per request, API key or workspace. It sets the latency target and the price.
Flex
"service_tier": "flex"The lowest price, for work that can wait. A call that cannot start fails fast, unbilled.
Price
Speed
Summaries, scoring, evals, backfills
Default
"service_tier": "default"Our standard serving: a published speed range per model and no rate limits.
Price
Speed
Most chat, classification and decisions
Priority
"service_tier": "priority"Priority scheduling, an uptime commitment and a latency target for your busiest hour.
Price
Speed
Launches, peaks, live conversations
Fast New
"service_tier": "fast"Realtime decoding and our tightest latency target.
Price
Speed
UserMy knees hurt after running. Can you plan a gentle 20-minute session for tonight?
DefaultTime to first token 147 ms166 tok/s
Of course. Start with five minutes of brisk walking to warm up. Then alternate one minute of easy jogging with two minutes of walking, four times. Keep your stride short and land softly under your hips. Finish with five minutes of slow walking and a calf and quad stretch. If the pain turns sharp, skip the jog and walk the whole way.
FastTime to first token 117 ms192 tok/s
Of course. Start with five minutes of brisk walking to warm up. Then alternate one minute of easy jogging with two minutes of walking, four times. Keep your stride short and land softly under your hips. Finish with five minutes of slow walking and a calf and quad stretch. If the pain turns sharp, skip the jog and walk the whole way.
Words that flash arrived together in one decoding step.
Research · realtime decoding
Realtime decoding drafts several tokens ahead and the full model verifies them in one pass.
01Verify step
7.5×
faster verify attention, Gemma 4 26B
Where the 7.5× goes
We rewrote the attention in that verify pass. It has run on our Gemma 4 26B serving since 1 Oct, so the saving reaches every reply.
Attention share of each decoding step−40%
Inter-token latency p50, this change alone
02Dedicated
Tuned
drafts
drafts
Drafts trained on your traffic
A draft trained on your own requests predicts your users' replies better, so more drafted tokens are accepted in each verify pass and replies stream faster.
Generic draft
Startwithteneasystretches
Draft tuned on your trafficStartwithfiveminutesoflight
Accepted per verify pass: 2 → 5 (illustration)03Output
Lossless
The same tokens, faster
The full model verifies every drafted token, so Fast returns exactly the tokens Default would.
Default 0.64 s
Fast 0.54 s
Start with five minutes of brisk walking to warm up.
Same tokens in bothHere's what you can do.
We host your weights, fine-tune on your requests and optimize decoding and kernels for them.
deployment● live
model acme/companion-v4
capacity dedicated
region us-central
tier priority
Dedicated deployments
Reserved capacity and your own model ID, with no queueing behind other customers.
"model": "you/companion-v4",
"service_tier": "priority"
"service_tier": "priority"
✓ weights loaded · fp8 · 31B
Bring your own model
Upload weights or point us at a repo. Same engine, same API.
fine-tuning run
Fine-tuning on your traffic
Fine-tune open models on your production conversations.
ahead the knees need a warm…
check ✓ ✓ ✓ ✓ ✕
accept 4 tokens in 1 pass
check ✓ ✓ ✓ ✓ ✕
accept 4 tokens in 1 pass
Drafts tuned on your traffic
Draft models trained on your requests.
kernel · attention
tokens a second at full load
before6.5Kafter27K
about 4× the stock kernel, Gemma 4 31B
Custom kernels and engines
When a stock engine is the limit, we write the attention kernel, KV-cache router or serving backend.
regions
● us-central
● eu-west
● asia-south
● asia-east
● us-central
● eu-west
● asia-south
● asia-east
zero data retention
on
on
Regions and data
US, Europe, India or Asia Pacific, with zero data retention on the models we host.
Inference that keeps getting faster.
Every week, a model we serve gets faster or cheaper.
Apr 2026
Gemma 4 31B · Throughput at full load
One patched attention kernel, about four times the tokens a second at full load.
Tokens a second, aggregate~4×
before6.5K
after27K
Sep 2026
GLM 5.3 Flash · Prompt cache hits
One fix in our KV-cache router nearly doubled cache hits on repeated prefixes.
Cache-hit rate1.9×
before25%
after47%
Sep 2026
Gemma 4 26B · Peak traffic
Peaks 35% busier on about a fifth less capacity, with a faster time to first token.
Busiest-hour requests, September = 100+35%
before100
after135
28 Sep 2026
Gemma 4 31B · Time to first token
Median time to first token 2.8× faster at production load (943 to 336 ms), each token after it 2.4× faster.
Time to first token, p502.8×
before943 ms
after336 ms
1 Oct 2026
Gemma 4 26B · Verify attention, realtime decoding
A rewritten verify step, up to 7.5 times faster on the global layers, so Gemma 4 26B replies arrive several tokens at a time.
Attention share of each decoding step−40%
before27.5%
after16.5%
1 Oct 2026
Gemma 4 26B · Realtime decoding build
The full realtime decoding build: median time to first token 154 to 117 ms and each token after it 6.4 to 5.2 ms, at the same eval scores.
Time to first token, p501.3×
before154 ms
after117 ms
8 Oct 2026
Gemma 4 31B · Time to first token in production
The same build in production: median time to first token 1.3× faster, while traffic rose 26%.
Daily median time to first token1.3×
before365 ms
after271 ms
Infrastructure built for scale.
Every model on this page runs on a fleet we operate, so a patch reaches production traffic in days.
Tens of trillionsof tokens a month
25 times what it was in March
Billionsof requests a month
Over a thousand a second at peak
Millionsof tokens a second
Around the clock
Up to 90%cache hits
On repeated prompts, Gemma 4 26B
1.8×OpenRouter's daily tokens
On Gemma 4 31B, every provider there combined
Four live regions
Inference built for people runs on Inworld. Who are you building for?
Questions
What is Inworld Inference?
Inworld Inference is the LLM inference API from Inworld AI, serving the open models consumer apps run on a fleet we operate. We optimize each model from the weights to the kernels on real traffic, so the models we serve keep getting faster or less expensive. The fleet serves tens of trillions of tokens a month, in four regions: the US, Europe, India and Asia Pacific.
Which models run on Inworld Inference?
Inworld AI's Inference API serves open models on our own fleet, including Gemma 4, GLM 5.3 Flash, MiMo 2.6 Pro, DeepSeek V4.1 Flash and the Clef decision model. Open models start from $0.07 per million input tokens and $0.34 per million output tokens, for Gemma 4 26B. You can also bring your own weights and run them on a dedicated deployment.
Does Inworld Inference work with the OpenAI API?
Yes, Inworld AI's LLM inference API is compatible with the OpenAI API. Send Chat Completions requests to https://api.inworld.ai/v1/chat/completions, or set the OpenAI SDK's base URL to https://api.inworld.ai/v1, with your Inworld API key. Name the model by its ID, such as deepseek-v4.1-flash, and the rest of your code stays the same.
What is fast mode on Inworld Inference?
Fast mode on Inworld AI's Inference API runs a model with realtime decoding for lower latency, and it is available on request today. Once it is on for your account, send "service_tier": "fast" in the request. Realtime decoding drafts several tokens ahead and has the full model verify them in one pass, and the model you call stays the same.
Which LLM API has low time to first token for realtime apps?
Inworld AI's Inference API is built for low time to first token, and we optimize each open model on real traffic to keep it low. On Gemma 4 31B, median time to first token fell from 943 ms to 336 ms, 2.8× faster (64% lower), with the same eval scores. Fast mode, available on request, adds realtime decoding on top.
Does Inworld Inference cache repeated prompts?
Yes, Inworld AI's Inference API uses cache-aware routing, which sends a repeated prompt to a server that already holds it, and reached up to 90% cache hits on repeated prompts on Gemma 4 26B. Cached input tokens cost less than fresh ones, up to 99% less on MiMo 2.6 Pro, at $0.0036 against $0.425 per million. Chat apps resend the system prompt and the conversation so far on every turn, which is why prompts repeat.
Can I run my own model on Inworld?
Yes, you can run your own model on Inworld AI's Inference API, on a dedicated deployment with capacity reserved for your traffic and your own model ID. We host your weights, fine-tune on your requests, and optimize decoding and kernels for them. Dedicated deployments run in the US, Europe, India or Asia Pacific, with zero data retention available on the models we host.
How reliable is Inworld Inference?
Over 30 days, 99.999% of more than 20 million GLM 5.3 Flash requests on Inworld AI's Inference API succeeded, counting server errors, timeouts and capacity rejections as failures. Requests run on a fleet we operate in four regions, the US, Europe, India and Asia Pacific, so a fix we make reaches your traffic in days.