Startup Gimlet Labs is solving the AI inference bottleneck in a surprisingly elegant way

Zain Asgar, a Stanford adjunct professor and previously exited founder, has raised an $80 million Series A for Gimlet Labs. The round was led by Menlo Ventures. The company aims to address the AI inference bottleneck with orchestration software that runs workloads across varied hardware. Gimlet Labs has built what it describes as the first multi-silicon inference cloud. The software splits AI application work across traditional CPUs, AI-tuned GPUs and high-memory systems at the same time. Asgar told TechCrunch that the system runs across whatever different hardware is available. Lead investor Tim Tully of Menlo Ventures wrote in a blog post that a single agent may chain multiple steps, each with distinct hardware demands. Inference is compute-bound, decode is memory-bound and tool calls are network-bound. No chip handles all these needs, Tully noted. Gimlet Labs claims the software speeds AI inference by three to 10 times for the same cost and power. It can also divide models so portions run on the best chip for each part. The company has partnered with chip makers including NVIDIA, AMD, Intel, ARM, Cerebras and d-Matrix. Gimlet launched publicly in October and reported eight-figure revenues from the start. Its customer base has more than doubled in the last four months and includes a major model maker and a large cloud computing company. The startup now employs 30 people and has raised $92 million in total.