llm-d 1.0 adds traffic priorities and upgrade controls for model serving

Google announced the release of llm-d 1.0, an open-source stack that coordinates model servers across Kubernetes clusters. The release adds stable v1 interfaces, router high availability and controls that let interactive requests share accelerator fleets with bursty agent workloads. Its routing directs requests to servers that already hold their context, reducing repeated prompt processing. A new asynchronous processor places background jobs into spare capacity under priority lanes and quotas. Upgrade controls coordinate separate prompt-processing and token-generation pools while preserving active sessions. Google says teams can adopt individual components within their existing infrastructure.