Breaking

Inworld Launches Realtime Inference API for Consumer AI Applications Targeting Sub-Second Latency

October 14, 2026 at 16:37 EDT

Foundation Models
AI Agents
Infra & Chips

Inworld, a research lab focused on real-time AI infrastructure, launched Realtime Inference on October 14, serving open-source language models at consumer scale through an OpenAI-compatible API. The new first-party service combines optimized model deployment, intelligent routing, and production observability to address latency and cost constraints in high-volume consumer applications like tutors, companions, and games.

October 2026 · Inworld AI

Open Models at Consumer Scale

Inworld's new Realtime Inference serves Gemma, DeepSeek, and GLM at sub-2-second latency and 50% cost reduction through intelligent caching and model routing.

Throughput

27K

tokens/sec on Gemma 4 31B

First Token

1.7s

Median latency (p50)

Pricing

−50%

vs. public rates

Cache Gain

2×

Hit rate boost (GLM Flash)

Real-World Impact

Status (AI social simulation) reached 1M users in 19 days with 20× cost reduction by combining Inworld's inference routing and caching.

Why It Matters

Consumer AI (companions, tutors, games) demand sub-500ms text latency and sub-800ms voice latency to feel natural
Cache-aware routing cuts both latency and token costs for repetitive conversation patterns
OpenAI-compatible API enables drop-in replacement for developers using other providers

Continue reading

The rest of this article is for AI News Blitz readers. Choose an option below to keep reading.