The build-versus-buy decision in AI infrastructure is not binary. Most consumer AI applications move through three architectural phases as they scale, each trading engineering complexity for cost control. The question is not "should we build our own TTS?" (almost certainly not). The question is how much of the stack around the models should be self-managed. This guide provides the specific cost thresholds for each transition, based on Inworld Research's
Consumer AI Stack 2026 report.
What are the three AI infrastructure phases?
Phase 1: Fully managed. A single platform handles the entire pipeline through one API.
Inworld's Speech-to-Speech API is an example: it handles turn-taking and interruption natively, integrates any major LLM, and deploys a conversational endpoint in approximately three minutes. Lowest engineering effort. Highest per-minute cost (managed orchestration on top of model consumption). But often the lowest total cost because engineering time is not spent building and maintaining the pipeline. Use it from zero to 100,000 users.
Phase 2: Hybrid. Build custom orchestration. Use managed APIs for each model layer. A team of 2 to 3 engineers can build this in 4 to 8 weeks. Components can be swapped independently, custom processing can be inserted between stages, and workloads can be optimized. The
Inworld Router fits this architecture: one API to 220 or more models, no markup on third-party models, and model switching via configuration change. Use it at 100,000 to 1 million DAU, or when requirements exceed what the managed API covers.
Phase 3: Vertically integrated. Self-host models on dedicated GPUs. Requires 5 to 10 or more ML infrastructure engineers. Per-minute cost is lowest at scale: dedicated GPUs from $5 per GPU-hour on Inworld Compute versus $11 per GPU-hour on GCP (as of July 2026). At sustained high utilization, 50% to 90% cheaper than API-based inference. Below roughly 1 million voice minutes per month, operational overhead exceeds the savings. Use it above 1 million DAU.
When should a consumer AI app switch from managed to hybrid?
According to Inworld Research (2026), the managed-to-hybrid crossover occurs when monthly API spend exceeds approximately $50,000 and engineering capacity is available. At this threshold, savings from smarter model routing and reduced waste (context optimization, barge-in reduction) typically exceed the engineering cost of building custom orchestration within 3 to 6 months. This crossover typically occurs around 100,000 DAU for voice-heavy consumer AI applications.
The hybrid approach does not require replacing the managed API entirely. Many teams maintain the managed API for straightforward conversational patterns and build custom orchestration for specific flows that require different model selection, custom processing, or advanced context management. The Inworld Router provides the model access layer for hybrid architectures without requiring multi-vendor API integration.
When does self-hosted AI become cheaper than managed APIs?
The hybrid-to-self-hosted crossover occurs when monthly inference spend exceeds approximately $200,000 and GPU utilization can be sustained above 60%, according to Inworld Research (2026). The 60% utilization floor is the critical threshold: below 60% utilization, the fixed cost of dedicated GPUs exceeds the variable cost of API-based inference. Most consumer AI applications with fewer than 1 million voice minutes per month cannot sustain 60% utilization on dedicated GPUs.
Self-hosted STT has a lower crossover threshold than other components. Whisper, the open-source speech recognition model, is mature and runs on modest hardware. The crossover from managed STT to self-hosted occurs around 500 to 1,000 hours of audio per month. This means STT is often the first component teams self-host, well before they self-host TTS or LLM inference.
How much does AI infrastructure cost at each scale?
Inworld Research (2026) provides cost ranges at four scale tiers for consumer voice AI applications. At 0 to 10,000 DAU: $1,000 to $10,000 per month. Use managed infrastructure. Focus on product-market fit, not infrastructure optimization. At 10,000 to 100,000 DAU: $10,000 to $100,000 per month. Cost enters the top five business concerns. Begin basic routing and cost tracking. At 100,000 to 500,000 DAU: $100,000 to $500,000 per month. Move to hybrid. The routing versus non-routing difference at this scale can be $200,000 to $400,000 per month. At 500,000 to 1 million or more DAU: $500,000 to $2 million or more per month. Evaluate dedicated compute for components with sustained high utilization.
What engineering team is needed for each infrastructure phase?
Managed: zero dedicated infrastructure engineers. The platform provider handles deployment, scaling, monitoring, and model updates. Engineering effort focuses on application logic: conversation design, user experience, product features. This is the correct choice when the team's competitive advantage is product design, content, or community rather than AI infrastructure.
Hybrid: 2 to 3 engineers for initial build (4 to 8 weeks), plus ongoing maintenance. Required skills: API integration, streaming infrastructure (WebSocket or WebRTC), basic ML operations (model benchmarking, quality monitoring). The Inworld Router reduces the model management surface (one API instead of separate integrations with each provider). This is the correct choice when monthly AI spend exceeds $50,000 and the team has engineering capacity to invest.
Vertically integrated: 5 to 10 or more dedicated ML infrastructure engineers. Required skills: GPU cluster management, model serving (vLLM, TensorRT), inference optimization, multi-region deployment, predictive autoscaling. Operational complexity is high: hardware failures, model updates, capacity planning, and GPU utilization optimization are ongoing concerns. This is the correct choice only when monthly inference spend exceeds $200,000 and GPU utilization can sustain above 60%.
How to decide which AI infrastructure approach to use
Three questions determine the right infrastructure phase. First: where is the competitive advantage? If it is the AI experience (conversation quality, voice expressiveness, response speed), own more of the stack. If the advantage is product design, content, or community, use managed infrastructure and focus engineering where the team differentiates.
Second: what is the monthly AI infrastructure spend? Below $50,000, managed is almost always correct. Between $50,000 and $200,000, hybrid is justified if engineering capacity exists. Above $200,000, evaluate self-hosting for high-utilization components. Third: can the team sustain 60% or higher GPU utilization on dedicated hardware? If not, self-hosting will cost more than API-based inference regardless of volume.
How Inworld supports the managed-to-self-hosted progression
The Inworld platform is designed for this progression rather than requiring an all-or-nothing commitment.
Speech-to-Speech API for managed deployments at the start.
Router for model optimization and cost control as the application scales. Realtime Inference for hybrid architectures that need custom orchestration. Dedicated Compute ($5 per GPU-hour versus $11 on GCP) for vertically integrated deployments. Each component works independently. Developers start where they need and expand when scale justifies it.
Risks and limitations of self-hosted AI infrastructure
Self-hosting carries risks that are easy to underestimate. GPU procurement lead times can exceed 3 to 6 months for high-demand hardware (NVIDIA H100, B200). Model updates from providers (new versions, deprecations) require manual deployment and testing rather than automatic availability via API. Multi-region deployment for global latency requirements multiplies hardware cost and operational complexity. Recruitment of ML infrastructure engineers is competitive; fully loaded cost is $200,000 to $400,000 per year per engineer in US markets.
The most common self-hosting failure mode is underutilization. Teams provision for peak traffic but average utilization falls below 60%, making the fixed GPU cost more expensive per inference than the variable API cost it was meant to replace. Predictive autoscaling partially addresses this, but it requires traffic patterns stable enough to forecast, which many consumer AI applications do not have in their first year.
Related guides
Key takeaways
- Consumer AI infrastructure progresses through three phases: managed (0-100K users), hybrid (100K-1M), and vertically integrated (1M+).
- The managed-to-hybrid crossover occurs at approximately $50,000/month API spend; the hybrid-to-self-hosted crossover at approximately $200,000/month.
- Self-hosted infrastructure is only cheaper than APIs when GPU utilization sustains above 60%; below that, fixed costs exceed variable API costs.
- Self-hosted STT (Whisper) has the lowest crossover threshold: approximately 500 to 1,000 hours of audio per month.
- Most consumer AI apps should start managed and add complexity only when cost scale and engineering capacity justify the transition.
Frequently asked questions
When should I self-host my LLM instead of using an API?
According to Inworld Research (2026), self-hosting becomes cost-effective when monthly inference spend exceeds approximately $200,000 and GPU utilization can sustain above 60%. Dedicated GPUs cost $5 per GPU-hour on Inworld Compute versus $11 on GCP, representing 50% to 90% savings at high utilization. Below 60% utilization, fixed GPU costs exceed the variable cost of API-based inference.
How many engineers does it take to self-host AI models?
A vertically integrated (self-hosted) AI deployment requires 5 to 10 or more dedicated ML infrastructure engineers with skills in GPU cluster management, model serving, inference optimization, and multi-region deployment. Fully loaded cost per engineer is $200,000 to $400,000 per year in US markets. By comparison, a hybrid architecture requires 2 to 3 engineers for initial build and ongoing maintenance.
What is the cheapest way to run AI at production scale?
The cheapest approach depends on scale. Below $50,000 per month API spend: managed APIs with basic model routing. Between $50,000 and $200,000: hybrid architecture with intelligent routing via the Inworld Router (one API, 220+ models, no markup). Above $200,000 with 60% or higher GPU utilization: self-hosted on dedicated GPUs ($5 per GPU-hour on Inworld Compute). At every tier, model routing (directing simple tasks to small, inexpensive models) is the highest-leverage cost optimization.
Can I self-host just one part of the AI stack?
Yes. Most teams that self-host do so selectively, starting with the component that has the lowest crossover threshold. Self-hosted STT (using Whisper) becomes cost-effective around 500 to 1,000 hours of audio per month, well before TTS or LLM inference. The Inworld platform supports this mixed approach: self-host STT while using managed TTS and Router APIs, then expand self-hosting as scale justifies it.
What is the biggest risk of self-hosting AI infrastructure?
GPU underutilization. Teams provision for peak traffic but average utilization falls below the 60% threshold where self-hosting is cheaper than APIs. The result: fixed GPU costs are higher per inference than the variable API costs they were meant to replace. GPU procurement lead times (3 to 6 months for high-demand hardware), model update management, and multi-region complexity are secondary risks.
Sources
Published by Inworld AI. This guide draws from the
Consumer AI Stack 2026 report. GPU pricing: Inworld Compute and GCP rates as of July 2026. Scale thresholds and cost ranges: Inworld engineering practices and anonymized customer patterns. Engineering cost estimates: industry benchmarks for ML infrastructure roles in US markets. All data Q2 2026.