AJ-Bench: Benchmarking Agent-as-a-Judge for Environment-Aware Evaluation
Published in Findings of ACL 2026, 2026
A benchmark for evaluating Agent-as-a-Judge with environment-aware verification across search, data systems, and GUI domains.
Recommended citation: Wentao Shi*, Yu Wang*, Yuyang Zhao*, Yuxin Chen, Fuli Feng, Xueyuan Hao, Xi Su, Qi Gu, Hui Su, Xunliang Cai, Xiangnan He. (2026). "AJ-Bench: Benchmarking Agent-as-a-Judge for Environment-Aware Evaluation." Findings of ACL 2026. (* denotes equal contribution)
Download Paper