Results
Across 5 long-horizon benchmarks (2 in-distribution agentic coding, 3 out-of-distribution),
ContextRL improves over the standard GRPO baseline by +1.5 points on average, while
improving every individual benchmark.
Table with columns: Benchmark, Base, RL (GRPO), ContextRL (Ours)| Benchmark | Base | RL (GRPO) | ContextRL (Ours) |
|---|
| SWE-Bench Verified | 5.00 | 6.20 | 7.00 |
| SWE-Bench Lite | 2.70 | 2.70 | 4.00 |
| LiveCodeBench v6 | 44.6 | 46.3 | 47.4 |
| LongBench v2 (Overall) | 31.6 | 31.8 | 33.2 |
| LongBench v2 (Long) | 27.8 | 26.9 | 29.6 |
| NIAH | 98.8 | 98.5 | 99.0 |
Metrics: SWE-Bench Verified/Lite resolve rate (%), LiveCodeBench v6 solve rate (%), LongBench v2 accuracy (%), NIAH mean recall (%). On the long-context tasks (LongBench v2, NIAH) where standard outcome-based GRPO struggles or regresses, ContextRL surpasses both the base model and the RL baseline, demonstrating strong out-of-distribution generalization.
Usage
This model follows the same interface as Qwen3-8B and can be loaded with transformers.
Training and evaluation code, data construction pipelines, and detailed configurations are
available in the repository:
👉 https://github.com/xupy2003/ContextAwareRL
Please refer to the repo's README for environment setup, inference scripts, and
reproduction instructions.