Results
Table with columns: Model, VitaBench, τ²-Bench, BFCL V4, ACEBench, ToolSandbox, Overall, AgentNoiseBench| Model | VitaBench | τ²-Bench | BFCL V4 | ACEBench | ToolSandbox | Overall | AgentNoiseBench |
|---|
| Qwen3-8B | 11.4 | 26.2 | 40.4 | 54.0 | 69.1 | 40.2 | 12.3 |
| EnvScaler-8B | 15.8 | 37.9 | 47.6 | 72.5 | 80.5 | 50.9 | 18.2 |
| Agent-World-8B | 14.5 | 54.6 | 51.4 | 72.0 | 75.6 | 53.6 | – |
| Agent-Reflex-8B | 17.5 | 62.1 | 52.9 | 78.0 | 82.5 | 58.6 | 29.1 |
Each benchmark is scored with its official metric. Overall is the unweighted mean of the five long-horizon tool-use benchmarks (Table 1 of the paper); AgentNoiseBench is the mean over its τ²-Bench and VitaBench noise settings (Table 2).
Usage
Serve with vLLM (native function calling, thinking on), as in our evaluation:
vllm serve dongguanting/Agent-Reflex-8B --served-model-name Qwen/Qwen3-8B \ --tensor-parallel-size 8 --max-model-len 40960 \ --enable-auto-tool-choice --tool-call-parser hermes --reasoning-parser deepseek_r1
The full evaluation suite (τ²-Bench, BFCL V4, ACEBench, VitaBench, AgentNoiseBench, ToolSandbox) is in the GitHub repo.
Citation
@misc{dong2026agentreflex, title={Agent-Reflex: Teaching Language Agents to Act Reflectively}, author={Guanting Dong and Xinyu Yang and Jiajie Jin and Yuyang Hu and Xiaoshuai Song and Yifei Chen and Xiaoxi Li and Jiejun Tan and Yutao Zhu and Tat-Seng Chua and Ji-Rong Wen and Zhicheng Dou}, year={2026}, eprint={2610.17953}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2610.17953},}