Files
Table with columns: path, content| path | content |
|---|
/ (repository root) | the generator: a Qwen3ForCausalLM checkpoint (bf16) with the tokenizer and chat template of Qwen/Qwen3-8B |
scorer/scorer.safetensors | the trained scorer (fp32) |
scorer/config.json | base model, scoring heads, and the frozen tables the loader takes from the base model (from_base) |
The scorer is loaded by re3rank.scorer.load_scorer of the code repository, which builds the truncated
backbone from Qwen/Qwen3-8B and loads scorer/scorer.safetensors on top of it.
Usage
Set up the environment of the code repository (PyTorch, transformers, vLLM), then:
git clone https://github.com/ryehr/RE3RANK && cd RE3RANK && pip install -e .hf download Ryehr/RE3RANK-Qwen3-8B --local-dir re3rank-qwen3-8b # 1. the generator writes one greedy reasoning trace per item (vLLM)python scripts/eval/generate_traces.py --model re3rank-qwen3-8b --tokenizer Qwen/Qwen3-8B \ --data items.json --out traces.json# 2. the scorer ranks the candidates of every item; --out holds the scores, one per candidatepython scripts/eval/score.py --model Qwen/Qwen3-8B --heads 15-11,23-10 --ckpt re3rank-qwen3-8b \ --traces traces.json --data items.json --out scores.json --keep-nogold
items.json is a list of items, each a question with its candidates in first-stage order:
[{"id": "q1", "question": "...", "paragraphs": [{"title": "...", "paragraph_text": "..."}, "..."]}]
The model was trained and evaluated with 50 candidates per question (the top-50 of
Qwen3-Embedding-8B). A candidate may carry "is_supporting": true/false; with labels, score.py also
prints the recall metrics. The whole nine-set evaluation of the paper runs with
CKPT=Ryehr/RE3RANK-Qwen3-8B BASE=Qwen/Qwen3-8B HEADS=15-11,23-10 bash scripts/eval/run_eval.sh
after building the evaluation sets with scripts/data/ (see the repository README).
Results
Macro average over nine sets (HotpotQA, 2WikiMultihopQA, MuSiQue, NQ, PopQA, NarrativeQA, DetectiveQA,
LoCoMo, LongMemEval; 9,205 queries), reranking the top-50 candidates of Qwen3-Embedding-8B. Recall is
conditional: only queries with a gold candidate among the 50 count, and R@k divides by the number of
gold candidates in the pool; NDCG@10 uses binary relevance. In %:
Table with columns: R@1, R@3, R@5, NDCG@10| R@1 | R@3 | R@5 | NDCG@10 |
|---|
| 44.12 | 76.68 | 84.36 | 82.36 |
Training
394 updates of 96 items (about one epoch) over a 38k-question mixture of NarrativeQA, MuSiQue, ConvoMem,
QAConv and FriendsQA training questions with their Qwen3-Embedding-8B top-50 candidates; the evaluation
questions are not used for training. Traces were sampled at temperature 1.0 with at most 1,024 tokens;
8-bit paged AdamW, learning rate 1e-5 for both models.
License
Apache-2.0, as the base model (license).
Citation
@misc{yan2026re3ranklistwisererankingreasoningconditioned, title={Re$^3$Rank: Listwise Reranking with Reasoning-Conditioned Retrieval Heads}, author={Ruiyi Yan and Zhuoyuan Mao and Yugo Murawaki and Yiwen Guo}, year={2026}, eprint={2611.02550}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2611.02550},}