Preprint reports 4.5-point accuracy gain for AI annotation framework

Researchers report that ANT-OR, a framework using multiple AI agents to label data, improves agreement@>=2 accuracy by 4.5 points over the strongest baseline in their experiments. The preprint introduces ANT-A, a benchmark spanning 7 datasets with task-specific guidelines and 5 independent human labels per instance. ANT-OR refines prompts by extracting contrasting decision rules from positive and negative feedback. The evaluation measures alignment with human labeling distributions, accommodating cases where human annotators reasonably disagree. Code and the benchmark are available.