lightonai

LightOn-rerank-LW-4B

README

License: apache-2.0

About the LightOn-rerank family

Production retrieval pipelines usually need two rerankers: one for text passages and one for visual documents (PDF pages, slides, scans). LightOn-rerank models are unified cross-encoder rerankers: a single model scores both text passages and document page images against a query, on top of any first-stage retriever (BM25, dense embeddings, or ColPali-family late-interaction models).

The models are built on Qwen3.5 backbone (hybrid linear + full attention) and jointly fine-tuned on text and visual reranking data with mixed-modality batches (LoRA, merged into the released weights). Training data is English-only; French performance transfers zero-shot from the multilingual backbone.

The family comes in two scoring flavours × three sizes (0.8B / 2B / 4B):

PW (pointwise): each candidate is scored independently. The model judges whether the document answers the query, and the score is logit("Yes") − logit("No"). One forward pass per candidate and no generation.
LW (listwise): generative listwise ranking, where 4 candidates are placed in a single prompt and the model generates a permutation ([2] > [4] > [1] > [3]). Larger candidate pools are ranked with a sliding window (window 4, stride 2, bottom-to-top). Cross-document attention makes LW markedly stronger on hard visual reranking, and unlike pointwise scoring it keeps improving with backbone size.

LightOn-rerank-LW-4B is the strongest model of the family: on ViDoRe V3 it edges past the official Qwen3-VL-Reranker-8B (64.69 vs 64.23 nDCG@10) at half the parameter count, winning 14 of 16 domain×language splits against our 2B listwise model, with French gaining even more than English (+2.4 vs +1.7).

Results

ViDoRe V3 (visual document reranking, 8 domains × EN/FR queries), overall nDCG@10, ColQwen2.5-v0.2 first stage, retrieve 100 / rerank 100. All models, including baselines, were re-evaluated under this same two-stage protocol, so numbers are mutually comparable but not comparable to vendor-reported end-to-end results.

Table with columns: Model, Params, Scoring, ViDoRe V3 overall nDCG@10
Model	Params	Scoring	ViDoRe V3 overall nDCG@10
LightOn-rerank-LW-4B (this model)	4.5B	listwise	64.69
Qwen3-VL-Reranker-8B	8B	pointwise (pooling)	64.23
LightOn-rerank-LW-2B	2.2B	listwise	62.66
LightOn-rerank-PW-2B	2.2B	pointwise	59.87
LightOn-rerank-PW-4B	4.5B	pointwise	59.80
jina-reranker-m0	2.4B	pointwise	59.40
Qwen3-VL-Reranker-2B	2B	pointwise (pooling)	59.18
LightOn-rerank-LW-0.8B	0.85B	listwise	58.25
MonoQwen2-VL-v0.1	2B	pointwise	57.76
First-stage only (ColQwen2.5, no rerank)	—	—	55.60
LightOn-rerank-PW-0.8B	0.85B	pointwise	48.20

ViDoRe V3 detail (nDCG@10, ColQwen2.5 first stage, rerank-100, sliding window 4/2)

Table with columns: Domain, EN, FR
Domain	EN	FR
finance_en	74.12	66.35
finance_fr	49.20	52.55
computer_science	82.84	80.87
hr	71.00	66.18
energy	71.15	73.26
industrial	59.95	54.95
pharmaceuticals	68.10	66.15
physics	49.18	49.20
mean	65.69	63.69

Overall nDCG@10: 64.69 (EN 65.69 / FR 63.69), the best result under this protocol, above the official Qwen3-VL-Reranker-8B (64.23) at half the parameters.

BEIR results (text reranking)

13 datasets, nDCG@10, BM25 first stage, retrieve 100 / rerank 100, same W=4 stride=2 sliding window as for document pages. ⚠️ marks datasets in the text training mix (NQ, MSMARCO); the decontaminated mean excludes them.

Table with columns: Dataset, nDCG@10
Dataset	nDCG@10
fever	79.28
scifact	76.60
trec-covid	71.52
hotpotqa	73.04
nq ⚠️	56.04
dbpedia	39.07
arguana	41.41
fiqa	38.49
msmarco ⚠️	37.32
nfcorpus	34.99
touche-2020	34.15
climate-fever	24.32
scidocs	18.78
Mean (13)	48.08
Decontaminated mean (11, excl. ⚠️)	48.33

Text gains over the 2B listwise model are modest (48.33 vs 48.12 decontaminated mean): the 2B→4B scale-up pays off mainly on visual reranking, where it buys +2.0 nDCG@10.

Model Details

Model type: multimodal cross-encoder reranker (generative listwise: 4 candidates per prompt, ranked by generating a permutation; sliding window (4, stride 2) for larger pools)
Base model: Qwen/Qwen3.5-4B (Qwen3.5 hybrid linear + full attention VLM)
Parameters: ≈4.5B (bfloat16, 9.1 GB)
Inputs: query (text) + candidate document(s): text passage or page image
Fine-tuning: joint text+vision LoRA (r=32, α=32, rsLoRA, merged into the released weights), mixed-modality batches (2 text + 2 vision groups per micro-batch), vision loss weight 1.3, lr 5e-5, warmup 30%, 1 epoch (419 steps), training images resized to 512×512
Data: 213k listwise groups — 107k text groups (NQ, TriviaQA, MS MARCO; each a [pos, neg_0, neg_1, neg_2] 4-list with hard negatives mined via the NV-Retriever approach with GTE-ModernBERT) + 106k vision groups (ColPali train set with negatives mined by Nomic). The gold permutation (pos > neg_0 > neg_1 > neg_2) is constructed directly from the mining metadata; training is cross-entropy on the permutation tokens only.
Languages: English (training), French (zero-shot transfer)
Requirements: transformers >= 5.4.0 (qwen3_5 architecture)

Usage: generative listwise reranking

The model ranks 4 candidates per prompt by generating a permutation string such as [2] > [1] > [4] > [3]. Candidate pools larger than 4 are ranked with a sliding window (window 4, stride 2) moving from the bottom of the list to the top, so the best candidates bubble up to the front. If a generation cannot be parsed, fall back to the input order (observed fallback rate in our evals: ≈0.01%).

python
import re
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "lightonai/LightOn-rerank-LW-4B"
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",  # optional, remove if flash-attn is not installed
    device_map="cuda",
).eval()
processor = AutoProcessor.from_pretrained(model_id)

PROMPT = "<|im_start|>user\n{user}<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"
PERM_RE = re.compile(r"\[(\d)\]\s*>\s*\[(\d)\]\s*>\s*\[(\d)\]\s*>\s*\[(\d)\]")


def rank_window(query: str, docs: list[str]) -> list[int]:
    """Rank exactly 4 text passages; returns window indices, most relevant first."""
    body = "\n".join(f"[{i + 1}]: {d}" for i, d in enumerate(docs))
    user = f"Query: {query}\n\nRank these passages from most to least relevant:\n{body}\n\nRanking:"
    inputs = processor(text=[PROMPT.format(user=user)], return_tensors="pt").to(model.device)
    out = model.generate(
        **inputs, max_new_tokens=30, do_sample=False,
        pad_token_id=processor.tokenizer.eos_token_id,
    )
    completion = processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
    m = PERM_RE.search(completion)
    return [int(g) - 1 for g in m.groups()] if m else list(range(4))


def rerank(query: str, docs: list, window: int = 4, stride: int = 2) -> list[int]:
    """Sliding-window listwise rerank; returns document indices, most relevant first."""
    order = list(range(len(docs)))
    positions = list(range(max(0, len(docs) - window), -1, -stride))
    if positions and positions[-1] != 0:
        positions.append(0)
    for pos in positions:
        end = min(pos + window, len(docs))
        p = max(0, end - window)
        if end - p < 2:
            continue
        perm = rank_window(query, [docs[i] for i in order[p:end]])
        order[p:end] = [order[p + j] for j in perm]
    return order


query = "What is late interaction in neural information retrieval?"
documents = ["passage 1 ...", "passage 2 ...", "passage 3 ...", "passage 4 ...", "passage 5 ..."]
print(rerank(query, documents))

For page images, build the window prompt with interleaved image placeholders instead:

python
def rank_window_images(query: str, images: list) -> list[int]:  # 4 PIL images
    content = [{"type": "text", "text": f"Query: {query}\n\nRank these documents from most to least relevant:\n"}]
    for i, img in enumerate(images):
        content += [
            {"type": "text", "text": f"[{i + 1}]: "},
            {"type": "image", "image": img},
            {"type": "text", "text": "\n"},
        ]
    content.append({"type": "text", "text": "\nRanking:"})
    text = processor.apply_chat_template(
        [{"role": "user", "content": content}], tokenize=False, add_generation_prompt=True
    )
    text += "<think>\n\n</think>\n\n"  # REQUIRED on the 4B backbone
    inputs = processor(text=[text], images=list(images), return_tensors="pt").to(model.device)
    out = model.generate(
        **inputs, max_new_tokens=30, do_sample=False,
        pad_token_id=processor.tokenizer.eos_token_id,
    )
    completion = processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
    m = PERM_RE.search(completion)
    return [int(g) - 1 for g in m.groups()] if m else list(range(4))

If a window has fewer than 4 candidates, pad it by repeating the last candidate and drop the duplicates from the returned order. Our evaluations ran this model with HF generate(); at the time of our evals vLLM could not serve the Qwen3.5-4B hybrid architecture for generation.

Notes & limitations

Prepend an empty thinking block to the assistant turn. Qwen3.5-4B is a thinking model; without the <think>\n\n</think>\n\n prefix (already included in the snippets above), reasoning tokens consume the generation budget and the ranking permutation never appears.
This is the quality ceiling of the family and also its slowest member by a long way: sequential sliding-window decoding on the 4B backbone costs roughly 5× the per-query wall time of the 2B pointwise model at rerank-100. Pick it when quality is the binding axis; to cut latency, shrink the candidate pool (reranking the top-20 instead of 100 keeps most of the rerank lift at ~5× fewer windows) rather than switching scoring method.
Training data is English-only. French works zero-shot (the backbone is multilingual) but is slightly behind English on average.
BEIR contamination flag: NQ and MSMARCO are part of the text training data; headline text figures use decontaminated means that exclude them.

The LightOn-rerank family

Table with columns: Model, Backbone, Scoring, ViDoRe V3 overall nDCG@10
Model	Backbone	Scoring	ViDoRe V3 overall nDCG@10
LightOn-rerank-PW-0.8B	Qwen3.5-0.8B	pointwise	48.20
LightOn-rerank-LW-0.8B	Qwen3.5-0.8B	listwise	58.25
LightOn-rerank-PW-2B	Qwen3.5-2B	pointwise	59.87
LightOn-rerank-LW-2B	Qwen3.5-2B	listwise	62.66
LightOn-rerank-PW-4B	Qwen3.5-4B	pointwise	59.80
LightOn-rerank-LW-4B	Qwen3.5-4B	listwise	64.69

Rule of thumb: LW models are stronger at every size (and the gap grows with size); PW models are cheaper to serve and score candidates independently. For the best quality pick LW-4B; for the best quality/cost trade-off pick LW-2B; for maximum throughput on text-heavy workloads pick a PW model.

Citation

markdown
@misc{ananya2026lightonrerank,
  title={One Adapter, Both Modalities: Field Notes from Building and Serving a Multimodal Reranker},
  author={Ananya, Ishrat Jahan and Chatelain, Amelie},
  year={2026},
  howpublished={\url{https://huggingface.co/blog/lightonai/lighton-rerank}},
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

lightonai

Model Tree

Base

Qwen/Qwen3.5-4B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

About the LightOn-rerank family

The family comes in two scoring flavours × three sizes (0.8B / 2B / 4B):

PW (pointwise): each candidate is scored independently. The model judges whether the document answers the query, and the score is logit("Yes") − logit("No"). One forward pass per candidate and no generation.
LW (listwise): generative listwise ranking, where 4 candidates are placed in a single prompt and the model generates a permutation ([2] > [4] > [1] > [3]). Larger candidate pools are ranked with a sliding window (window 4, stride 2, bottom-to-top). Cross-document attention makes LW markedly stronger on hard visual reranking, and unlike pointwise scoring it keeps improving with backbone size.

Results

Table with columns: Model, Params, Scoring, ViDoRe V3 overall nDCG@10
Model	Params	Scoring	ViDoRe V3 overall nDCG@10
LightOn-rerank-LW-4B (this model)	4.5B	listwise	64.69
Qwen3-VL-Reranker-8B	8B	pointwise (pooling)	64.23
LightOn-rerank-LW-2B	2.2B	listwise	62.66
LightOn-rerank-PW-2B	2.2B	pointwise	59.87
LightOn-rerank-PW-4B	4.5B	pointwise	59.80
jina-reranker-m0	2.4B	pointwise	59.40
Qwen3-VL-Reranker-2B	2B	pointwise (pooling)	59.18
LightOn-rerank-LW-0.8B	0.85B	listwise	58.25
MonoQwen2-VL-v0.1	2B	pointwise	57.76
First-stage only (ColQwen2.5, no rerank)	—	—	55.60
LightOn-rerank-PW-0.8B	0.85B	pointwise	48.20

ViDoRe V3 detail (nDCG@10, ColQwen2.5 first stage, rerank-100, sliding window 4/2)

Table with columns: Domain, EN, FR
Domain	EN	FR
finance_en	74.12	66.35
finance_fr	49.20	52.55
computer_science	82.84	80.87
hr	71.00	66.18
energy	71.15	73.26
industrial	59.95	54.95
pharmaceuticals	68.10	66.15
physics	49.18	49.20
mean	65.69	63.69

Overall nDCG@10: 64.69 (EN 65.69 / FR 63.69), the best result under this protocol, above the official Qwen3-VL-Reranker-8B (64.23) at half the parameters.

BEIR results (text reranking)

Table with columns: Dataset, nDCG@10
Dataset	nDCG@10
fever	79.28
scifact	76.60
trec-covid	71.52
hotpotqa	73.04
nq ⚠️	56.04
dbpedia	39.07
arguana	41.41
fiqa	38.49
msmarco ⚠️	37.32
nfcorpus	34.99
touche-2020	34.15
climate-fever	24.32
scidocs	18.78
Mean (13)	48.08
Decontaminated mean (11, excl. ⚠️)	48.33

Text gains over the 2B listwise model are modest (48.33 vs 48.12 decontaminated mean): the 2B→4B scale-up pays off mainly on visual reranking, where it buys +2.0 nDCG@10.

Model Details

Model type: multimodal cross-encoder reranker (generative listwise: 4 candidates per prompt, ranked by generating a permutation; sliding window (4, stride 2) for larger pools)
Base model: Qwen/Qwen3.5-4B (Qwen3.5 hybrid linear + full attention VLM)
Parameters: ≈4.5B (bfloat16, 9.1 GB)
Inputs: query (text) + candidate document(s): text passage or page image
Fine-tuning: joint text+vision LoRA (r=32, α=32, rsLoRA, merged into the released weights), mixed-modality batches (2 text + 2 vision groups per micro-batch), vision loss weight 1.3, lr 5e-5, warmup 30%, 1 epoch (419 steps), training images resized to 512×512
Data: 213k listwise groups — 107k text groups (NQ, TriviaQA, MS MARCO; each a [pos, neg_0, neg_1, neg_2] 4-list with hard negatives mined via the NV-Retriever approach with GTE-ModernBERT) + 106k vision groups (ColPali train set with negatives mined by Nomic). The gold permutation (pos > neg_0 > neg_1 > neg_2) is constructed directly from the mining metadata; training is cross-entropy on the permutation tokens only.
Languages: English (training), French (zero-shot transfer)
Requirements: transformers >= 5.4.0 (qwen3_5 architecture)

Usage: generative listwise reranking

python
import re
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "lightonai/LightOn-rerank-LW-4B"
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",  # optional, remove if flash-attn is not installed
    device_map="cuda",
).eval()
processor = AutoProcessor.from_pretrained(model_id)

PROMPT = "<|im_start|>user\n{user}<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"
PERM_RE = re.compile(r"\[(\d)\]\s*>\s*\[(\d)\]\s*>\s*\[(\d)\]\s*>\s*\[(\d)\]")


def rank_window(query: str, docs: list[str]) -> list[int]:
    """Rank exactly 4 text passages; returns window indices, most relevant first."""
    body = "\n".join(f"[{i + 1}]: {d}" for i, d in enumerate(docs))
    user = f"Query: {query}\n\nRank these passages from most to least relevant:\n{body}\n\nRanking:"
    inputs = processor(text=[PROMPT.format(user=user)], return_tensors="pt").to(model.device)
    out = model.generate(
        **inputs, max_new_tokens=30, do_sample=False,
        pad_token_id=processor.tokenizer.eos_token_id,
    )
    completion = processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
    m = PERM_RE.search(completion)
    return [int(g) - 1 for g in m.groups()] if m else list(range(4))


def rerank(query: str, docs: list, window: int = 4, stride: int = 2) -> list[int]:
    """Sliding-window listwise rerank; returns document indices, most relevant first."""
    order = list(range(len(docs)))
    positions = list(range(max(0, len(docs) - window), -1, -stride))
    if positions and positions[-1] != 0:
        positions.append(0)
    for pos in positions:
        end = min(pos + window, len(docs))
        p = max(0, end - window)
        if end - p < 2:
            continue
        perm = rank_window(query, [docs[i] for i in order[p:end]])
        order[p:end] = [order[p + j] for j in perm]
    return order


query = "What is late interaction in neural information retrieval?"
documents = ["passage 1 ...", "passage 2 ...", "passage 3 ...", "passage 4 ...", "passage 5 ..."]
print(rerank(query, documents))

For page images, build the window prompt with interleaved image placeholders instead:

python
def rank_window_images(query: str, images: list) -> list[int]:  # 4 PIL images
    content = [{"type": "text", "text": f"Query: {query}\n\nRank these documents from most to least relevant:\n"}]
    for i, img in enumerate(images):
        content += [
            {"type": "text", "text": f"[{i + 1}]: "},
            {"type": "image", "image": img},
            {"type": "text", "text": "\n"},
        ]
    content.append({"type": "text", "text": "\nRanking:"})
    text = processor.apply_chat_template(
        [{"role": "user", "content": content}], tokenize=False, add_generation_prompt=True
    )
    text += "<think>\n\n</think>\n\n"  # REQUIRED on the 4B backbone
    inputs = processor(text=[text], images=list(images), return_tensors="pt").to(model.device)
    out = model.generate(
        **inputs, max_new_tokens=30, do_sample=False,
        pad_token_id=processor.tokenizer.eos_token_id,
    )
    completion = processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
    m = PERM_RE.search(completion)
    return [int(g) - 1 for g in m.groups()] if m else list(range(4))

Notes & limitations

Prepend an empty thinking block to the assistant turn. Qwen3.5-4B is a thinking model; without the <think>\n\n</think>\n\n prefix (already included in the snippets above), reasoning tokens consume the generation budget and the ranking permutation never appears.
This is the quality ceiling of the family and also its slowest member by a long way: sequential sliding-window decoding on the 4B backbone costs roughly 5× the per-query wall time of the 2B pointwise model at rerank-100. Pick it when quality is the binding axis; to cut latency, shrink the candidate pool (reranking the top-20 instead of 100 keeps most of the rerank lift at ~5× fewer windows) rather than switching scoring method.
Training data is English-only. French works zero-shot (the backbone is multilingual) but is slightly behind English on average.
BEIR contamination flag: NQ and MSMARCO are part of the text training data; headline text figures use decontaminated means that exclude them.

The LightOn-rerank family

Table with columns: Model, Backbone, Scoring, ViDoRe V3 overall nDCG@10
Model	Backbone	Scoring	ViDoRe V3 overall nDCG@10
LightOn-rerank-PW-0.8B	Qwen3.5-0.8B	pointwise	48.20
LightOn-rerank-LW-0.8B	Qwen3.5-0.8B	listwise	58.25
LightOn-rerank-PW-2B	Qwen3.5-2B	pointwise	59.87
LightOn-rerank-LW-2B	Qwen3.5-2B	listwise	62.66
LightOn-rerank-PW-4B	Qwen3.5-4B	pointwise	59.80
LightOn-rerank-LW-4B	Qwen3.5-4B	listwise	64.69

Citation

markdown
@misc{ananya2026lightonrerank,
  title={One Adapter, Both Modalities: Field Notes from Building and Serving a Multimodal Reranker},
  author={Ananya, Ishrat Jahan and Chatelain, Amelie},
  year={2026},
  howpublished={\url{https://huggingface.co/blog/lightonai/lighton-rerank}},
}