Migrating a production AI agent to GPT-5.6: 2.2x faster, 27% cheaper

Ploy said it has migrated its production AI agent to OpenAI's GPT-5.6 Sol, making it the default model for every Ploy workspace after head-to-head testing against Anthropic's Claude Opus 4.8. The company said GPT-5.6 Sol is the first model in four months to beat Opus on its internal benchmarks, which involve building and editing real marketing websites. Ploy said the migration was prompted by the promise of builds finishing in less than half the wall-clock time at 27% lower cost while scoring at or above the incumbent on completed work. The company said the switch required a significant engineering effort to address provider-specific behaviors, including how the model fills in tool arguments, how its prompt cache works, and how it replays reasoning between turns. Ploy said it first fixed its evaluation harness, discovering that roughly a third of initial failures traced back to harness assumptions tuned to Opus rather than model behavior. The company said it then addressed tool-call schemas, implementing a schema transform at the provider boundary to handle GPT-5.6's tendency to send all 25 optional parameters with invented values, which had caused 52% to 64% of file reads to return empty. Ploy said the fix reduced empty reads to 0% and cut tool calls by roughly 30%. The company said it also rebuilt prompt caching logic.