Inworld launches open-model inference service for consumer AI applications

Inworld launched Realtime Inference on October 14, making its hosted open models available through an OpenAI-compatible API and its Realtime Router. Developers can call those models directly or combine them with more than 100 third-party models using conditional routing, provider fallback, traffic splitting and caching. The service targets consumer applications that repeatedly process long conversation histories and produce short replies. It reuses repeated prompt context and offers lower cached-input rates on eligible models. Developers can track model selection, routing attempts, time to first token, token costs and cache behavior. Inworld says its decoding and precision optimizations are designed to improve latency and throughput while retaining quality evaluations.