Skip to main content

Share story

AI

New APEX-Agents benchmark shows AI agents fail at real workplace tasks

New APEX-Agents benchmark shows AI agents fail at real workplace tasks Image: Primary
A new benchmark called APEX-Agents shows that leading AI models struggle with tasks drawn from consulting, investment banking and law. Mercor developed the benchmark using queries submitted by professionals on its expert marketplace. The models received full professional environments modeled on tools such as Slack and Google Drive. Even the highest-scoring systems answered correctly in one shot less than a quarter of the time and returned wrong answers or no answers in most cases. Mercor CEO Brendan Foody said the biggest limitation was tracking information across multiple domains. He described the benchmark as reflective of actual work performed by professionals in those fields. The test differs from OpenAI's GDPval benchmark by focusing on sustained tasks in narrow high-value professions rather than general knowledge. Gemini 3 Flash recorded the highest score at 24 percent. GPT-5.2 followed at 23 percent. Opus 4.5, Gemini 3 Pro and GPT-5 each scored about 18 percent. The questions and correct answers are posted publicly on Hugging Face. Foody said performance has improved from roughly 5 to 10 percent last year.
Sources
In this story
Published by Tech & Business, a media brand covering technology and business. This story was sourced from TechCrunch and reviewed by the T&B editorial agent team.
Back to Newswire
Keep reading
Full wire
Capital AI
Capital AI

Nevis raises $70M with Schwab joining wealth software investors

Nevis raised $70M in a Series B led by EQT Ventures, bringing its total funding to $110M, the New York wealth management software company announced Monday. Charles Schwab and QuantumLight joined existing investors Sequoia Capital,...

Robotics Capital
Robotics Capital

Bedrock Robotics raises $600M at a $4.1B valuation

Bedrock Robotics raised a $600M Series C at a $4.1B valuation, Bloomberg reported. The startup makes driverless excavators and other autonomous construction machines. The financing adds capital to a company developing autonomous ...

Capital
Capital

Matiks raises $10 million to expand puzzle games and game creation

India-based Matiks said Monday it raised $10 million in pre-Series A funding led by General Catalyst, bringing its total funding to over $13.5 million. The startup plans to expand into new markets and scale Jelo, its experimental ...

AI Products
AI Products

AWS makes GPT-6.1 Terra generally available on Bedrock

AWS announced general availability of OpenAI's GPT-6.1 Terra on Amazon Bedrock, giving customers access through Bedrock APIs and configurations for ChatGPT Work and Codex. The model accepts text and images and supports up to 1 mil...

AI
AI

OpenAI releases GPT-6.1 Terra at $1 per million input tokens

OpenAI has released GPT-6.1 Terra, priced at $1 per million input tokens and $5 per million output tokens. The company describes the model as supporting computer use at scale and detailed visual reasoning, and claims clearer, more...