AI Can Summarize Employee Feedback. A New Benchmark Shows It Doesn't Always Understand It.

PYX Labs, a research lab sponsored by Perceptyx, announced Tuesday the release of PYX-Voice, a benchmark evaluating how well frontier AI models understand employee feedback. The benchmark tested seven leading models from OpenAI, Google, Anthropic and xAI across 84 employee listening tasks. PYX Labs said model reliability declined significantly when interpreting complex human context behind employee expressions. The benchmark evaluates responses against criteria developed by industrial-organizational psychologists and organizational behavior specialists. The findings come as organizations increasingly rely on generative AI to summarize employee comments and inform decisions about culture, leadership and employee development. A 2025 survey of more than 1,300 U.S. managers found six in ten reported using AI to help make decisions about direct reports, including raises, promotions, layoffs and terminations. On quantitative tasks, frontier models performed consistently well, clustering between 64% and 82%. On interpretive tasks synthesizing open-ended feedback, scores dropped as low as 33%. Gemini-3.5-flash led the full benchmark with an overall score of 76%. Synthesis was the lowest-scoring capability across all models, ranging from 14% to 57%. PYX Labs identified rare instances where models produced fabricated statistical outputs. Joseph Freed, Chief Product Officer at Perceptyx and Head of PYX Labs, said the question is whether models understand what good looks like in the workplace context. Melissa Valentine, Professor of Management Science at Stanford University and Senior Fellow at the Stanford Institute for Human-Centered AI, said the work addresses a critical gap in AI evaluation for workplace use. A joint PYX Labs and Stanford HAI webinar is scheduled for August 5 at 12:00 pm ET.