lightonai

LightOn-rerank-LW-0.8B

README

License: apache-2.0

About the LightOn-rerank family

Production retrieval pipelines usually need two rerankers: one for text passages and one for visual documents (PDF pages, slides, scans). LightOn-rerank models are unified cross-encoder rerankers: a single model scores both text passages and document page images against a query, on top of any first-stage retriever (BM25, dense embeddings, or ColPali-family late-interaction models).

The models are built on Qwen3.5 backbone (hybrid linear + full attention) and jointly fine-tuned on text and visual reranking data with mixed-modality batches (LoRA, merged into the released weights). Training data is English-only; French performance transfers zero-shot from the multilingual backbone.

The family comes in two scoring flavours × three sizes (0.8B / 2B / 4B):

PW (pointwise): each candidate is scored independently. The model judges whether the document answers the query, and the score is logit("Yes") − logit("No"). One forward pass per candidate and no generation.
LW (listwise): generative listwise ranking, where 4 candidates are placed in a single prompt and the model generates a permutation ([2] > [4] > [1] > [3]). Larger candidate pools are ranked with a sliding window (window 4, stride 2, bottom-to-top). Cross-document attention makes LW markedly stronger on hard visual reranking, and unlike pointwise scoring it keeps improving with backbone size.

LightOn-rerank-LW-0.8B is the size-efficiency champion of the family: at 0.85B parameters it lands within ~1 nDCG@10 point of 2B-class rerankers (jina-reranker-m0, Qwen3-VL-Reranker-2B) on ViDoRe V3 while being ~2.5× smaller. If your binding constraint is accelerator memory rather than latency, this is the model to pick.

Results

ViDoRe V3 (visual document reranking, 8 domains × EN/FR queries), overall nDCG@10, ColQwen2.5-v0.2 first stage, retrieve 100 / rerank 100. All models, including baselines, were re-evaluated under this same two-stage protocol, so numbers are mutually comparable but not comparable to vendor-reported end-to-end results.

Table with columns: Model, Params, Scoring, ViDoRe V3 overall nDCG@10
Model	Params	Scoring	ViDoRe V3 overall nDCG@10
LightOn-rerank-LW-4B	4.5B	listwise	64.69
Qwen3-VL-Reranker-8B	8B	pointwise (pooling)	64.23
LightOn-rerank-LW-2B	2.2B	listwise	62.66
LightOn-rerank-PW-2B	2.2B	pointwise	59.87
LightOn-rerank-PW-4B	4.5B	pointwise	59.80
jina-reranker-m0	2.4B	pointwise	59.40
Qwen3-VL-Reranker-2B	2B	pointwise (pooling)	59.18
LightOn-rerank-LW-0.8B (this model)	0.85B	listwise	58.25
MonoQwen2-VL-v0.1	2B	pointwise	57.76
First-stage only (ColQwen2.5, no rerank)	—	—	55.60
LightOn-rerank-PW-0.8B	0.85B	pointwise	48.20

ViDoRe V3 detail (nDCG@10, ColQwen2.5 first stage, rerank-100, sliding window 4/2)

Table with columns: Domain, EN, FR
Domain	EN	FR
finance_en	67.80	57.14
finance_fr	42.07	47.41
computer_science	79.14	75.04
hr	65.25	57.43
energy	63.66	67.72
industrial	55.34	44.51
pharmaceuticals	61.81	57.71
physics	44.36	45.54
mean	59.93	56.56

Overall nDCG@10: 58.25 (EN 59.93 / FR 56.56), within ~1 point of Qwen3-VL-Reranker-2B (59.18) and ~1.2 points of jina-reranker-m0 (59.40), both ~2.5× larger.

BEIR results (text reranking)

nDCG@10, BM25 first stage, retrieve 100 / rerank 100, same sliding-window listwise reading as for images. ⚠️ marks datasets in the text training mix (NQ, MSMARCO); the decontaminated mean excludes them.

Table with columns: Dataset, nDCG@10
Dataset	nDCG@10
fever	78.12
scifact	74.22
trec-covid	70.62
hotpotqa	70.97
nq ⚠️	53.39
dbpedia	38.89
arguana	42.77
fiqa	36.25
msmarco ⚠️	36.55
nfcorpus	34.21
touche-2020	34.20
climate-fever	21.74
scidocs	18.32
Mean (13)	46.94
Decontaminated mean (11, excl. ⚠️)	47.30

On text the pointwise sibling is slightly stronger (PW-0.8B: 47.98 decontaminated mean vs 47.30 here) — the mirror image of vision, where listwise wins at every size.

Model Details

Model type: multimodal cross-encoder reranker (generative listwise: 4 candidates per prompt, ranked by generating a permutation; sliding window (4, stride 2) for larger pools)
Base model: Qwen/Qwen3.5-0.8B (Qwen3.5 hybrid linear + full attention VLM)
Parameters: ≈0.85B (bfloat16, 1.7 GB)
Inputs: query (text) + candidate document(s): text passage or page image
Fine-tuning: joint text+vision LoRA (r=32, α=32, rsLoRA, merged into the released weights), mixed-modality batches (2 text + 2 vision groups per micro-batch), vision loss weight 1.3, lr 5e-5, 2 epochs (838 steps): the second epoch helps at this size, unlike at 2B
Data: 213k listwise groups — 107k text groups (NQ, TriviaQA, MS MARCO; each a [pos, neg_0, neg_1, neg_2] 4-list with hard negatives mined via the NV-Retriever approach with GTE-ModernBERT) + 106k vision groups (ColPali train set with negatives mined by Nomic). The gold permutation (pos > neg_0 > neg_1 > neg_2) is constructed directly from the mining metadata; training is cross-entropy on the permutation tokens only.
Languages: English (training), French (zero-shot transfer)
Requirements: transformers >= 5.4.0 (qwen3_5 architecture)

Usage: generative listwise reranking

The model ranks 4 candidates per prompt by generating a permutation string such as [2] > [1] > [4] > [3]. Candidate pools larger than 4 are ranked with a sliding window (window 4, stride 2) moving from the bottom of the list to the top, so the best candidates bubble up to the front. If a generation cannot be parsed, fall back to the input order (observed fallback rate in our evals: ≈0.01%).

python
import re
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "lightonai/LightOn-rerank-LW-0.8B"
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",  # optional, remove if flash-attn is not installed
    device_map="cuda",
).eval()
processor = AutoProcessor.from_pretrained(model_id)

PROMPT = "<|im_start|>user\n{user}<|im_end|>\n<|im_start|>assistant\n"
PERM_RE = re.compile(r"\[(\d)\]\s*>\s*\[(\d)\]\s*>\s*\[(\d)\]\s*>\s*\[(\d)\]")


def rank_window(query: str, docs: list[str]) -> list[int]:
    """Rank exactly 4 text passages; returns window indices, most relevant first."""
    body = "\n".join(f"[{i + 1}]: {d}" for i, d in enumerate(docs))
    user = f"Query: {query}\n\nRank these passages from most to least relevant:\n{body}\n\nRanking:"
    inputs = processor(text=[PROMPT.format(user=user)], return_tensors="pt").to(model.device)
    out = model.generate(
        **inputs, max_new_tokens=30, do_sample=False,
        pad_token_id=processor.tokenizer.eos_token_id,
    )
    completion = processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
    m = PERM_RE.search(completion)
    return [int(g) - 1 for g in m.groups()] if m else list(range(4))


def rerank(query: str, docs: list, window: int = 4, stride: int = 2) -> list[int]:
    """Sliding-window listwise rerank; returns document indices, most relevant first."""
    order = list(range(len(docs)))
    positions = list(range(max(0, len(docs) - window), -1, -stride))
    if positions and positions[-1] != 0:
        positions.append(0)
    for pos in positions:
        end = min(pos + window, len(docs))
        p = max(0, end - window)
        if end - p < 2:
            continue
        perm = rank_window(query, [docs[i] for i in order[p:end]])
        order[p:end] = [order[p + j] for j in perm]
    return order


query = "What is late interaction in neural information retrieval?"
documents = ["passage 1 ...", "passage 2 ...", "passage 3 ...", "passage 4 ...", "passage 5 ..."]
print(rerank(query, documents))

For page images, build the window prompt with interleaved image placeholders instead:

python
def rank_window_images(query: str, images: list) -> list[int]:  # 4 PIL images
    content = [{"type": "text", "text": f"Query: {query}\n\nRank these documents from most to least relevant:\n"}]
    for i, img in enumerate(images):
        content += [
            {"type": "text", "text": f"[{i + 1}]: "},
            {"type": "image", "image": img},
            {"type": "text", "text": "\n"},
        ]
    content.append({"type": "text", "text": "\nRanking:"})
    text = processor.apply_chat_template(
        [{"role": "user", "content": content}], tokenize=False, add_generation_prompt=True
    )
    inputs = processor(text=[text], images=list(images), return_tensors="pt").to(model.device)
    out = model.generate(
        **inputs, max_new_tokens=30, do_sample=False,
        pad_token_id=processor.tokenizer.eos_token_id,
    )
    completion = processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
    m = PERM_RE.search(completion)
    return [int(g) - 1 for g in m.groups()] if m else list(range(4))

If a window has fewer than 4 candidates, pad it by repeating the last candidate and drop the duplicates from the returned order.

Notes & limitations

Weakest domain relative to 2B-class models is pharmaceuticals (≈−6 nDCG points); strongest wins over Qwen3-VL-Reranker-2B are computer_science, hr, energy and physics (EN).
Listwise reranking generates the full permutation (~18 decoded tokens per window, ≈ n/2 windows for n candidates), so per-query latency is higher than pointwise scoring — the trade for markedly better ranking quality at this size. At rerank-100 the larger PW-2B is both faster and slightly stronger; this model's niche is memory-constrained deployment.
Training data is English-only. French works zero-shot (the backbone is multilingual) but is slightly behind English on average.
BEIR contamination flag: NQ and MSMARCO are part of the text training data; headline text figures use decontaminated means that exclude them.

The LightOn-rerank family

Table with columns: Model, Backbone, Scoring, ViDoRe V3 overall nDCG@10
Model	Backbone	Scoring	ViDoRe V3 overall nDCG@10
LightOn-rerank-PW-0.8B	Qwen3.5-0.8B	pointwise	48.20
LightOn-rerank-LW-0.8B	Qwen3.5-0.8B	listwise	58.25
LightOn-rerank-PW-2B	Qwen3.5-2B	pointwise	59.87
LightOn-rerank-LW-2B	Qwen3.5-2B	listwise	62.66
LightOn-rerank-PW-4B	Qwen3.5-4B	pointwise	59.80
LightOn-rerank-LW-4B	Qwen3.5-4B	listwise	64.69

Rule of thumb: LW models are stronger at every size (and the gap grows with size); PW models are cheaper to serve and score candidates independently. For the best quality pick LW-4B; for the best quality/cost trade-off pick LW-2B; for maximum throughput on text-heavy workloads pick a PW model.

Citation

markdown
@misc{ananya2026lightonrerank,
  title={One Adapter, Both Modalities: Field Notes from Building and Serving a Multimodal Reranker},
  author={Ananya, Ishrat Jahan and Chatelain, Amelie},
  year={2026},
  howpublished={\url{https://huggingface.co/blog/lightonai/lighton-rerank}},
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

lightonai

Model Tree

Base

Qwen/Qwen3.5-0.8B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

About the LightOn-rerank family

The family comes in two scoring flavours × three sizes (0.8B / 2B / 4B):

PW (pointwise): each candidate is scored independently. The model judges whether the document answers the query, and the score is logit("Yes") − logit("No"). One forward pass per candidate and no generation.
LW (listwise): generative listwise ranking, where 4 candidates are placed in a single prompt and the model generates a permutation ([2] > [4] > [1] > [3]). Larger candidate pools are ranked with a sliding window (window 4, stride 2, bottom-to-top). Cross-document attention makes LW markedly stronger on hard visual reranking, and unlike pointwise scoring it keeps improving with backbone size.

Results

Table with columns: Model, Params, Scoring, ViDoRe V3 overall nDCG@10
Model	Params	Scoring	ViDoRe V3 overall nDCG@10
LightOn-rerank-LW-4B	4.5B	listwise	64.69
Qwen3-VL-Reranker-8B	8B	pointwise (pooling)	64.23
LightOn-rerank-LW-2B	2.2B	listwise	62.66
LightOn-rerank-PW-2B	2.2B	pointwise	59.87
LightOn-rerank-PW-4B	4.5B	pointwise	59.80
jina-reranker-m0	2.4B	pointwise	59.40
Qwen3-VL-Reranker-2B	2B	pointwise (pooling)	59.18
LightOn-rerank-LW-0.8B (this model)	0.85B	listwise	58.25
MonoQwen2-VL-v0.1	2B	pointwise	57.76
First-stage only (ColQwen2.5, no rerank)	—	—	55.60
LightOn-rerank-PW-0.8B	0.85B	pointwise	48.20

ViDoRe V3 detail (nDCG@10, ColQwen2.5 first stage, rerank-100, sliding window 4/2)

Table with columns: Domain, EN, FR
Domain	EN	FR
finance_en	67.80	57.14
finance_fr	42.07	47.41
computer_science	79.14	75.04
hr	65.25	57.43
energy	63.66	67.72
industrial	55.34	44.51
pharmaceuticals	61.81	57.71
physics	44.36	45.54
mean	59.93	56.56

Overall nDCG@10: 58.25 (EN 59.93 / FR 56.56), within ~1 point of Qwen3-VL-Reranker-2B (59.18) and ~1.2 points of jina-reranker-m0 (59.40), both ~2.5× larger.

BEIR results (text reranking)

Table with columns: Dataset, nDCG@10
Dataset	nDCG@10
fever	78.12
scifact	74.22
trec-covid	70.62
hotpotqa	70.97
nq ⚠️	53.39
dbpedia	38.89
arguana	42.77
fiqa	36.25
msmarco ⚠️	36.55
nfcorpus	34.21
touche-2020	34.20
climate-fever	21.74
scidocs	18.32
Mean (13)	46.94
Decontaminated mean (11, excl. ⚠️)	47.30

On text the pointwise sibling is slightly stronger (PW-0.8B: 47.98 decontaminated mean vs 47.30 here) — the mirror image of vision, where listwise wins at every size.

Model Details

Model type: multimodal cross-encoder reranker (generative listwise: 4 candidates per prompt, ranked by generating a permutation; sliding window (4, stride 2) for larger pools)
Base model: Qwen/Qwen3.5-0.8B (Qwen3.5 hybrid linear + full attention VLM)
Parameters: ≈0.85B (bfloat16, 1.7 GB)
Inputs: query (text) + candidate document(s): text passage or page image
Fine-tuning: joint text+vision LoRA (r=32, α=32, rsLoRA, merged into the released weights), mixed-modality batches (2 text + 2 vision groups per micro-batch), vision loss weight 1.3, lr 5e-5, 2 epochs (838 steps): the second epoch helps at this size, unlike at 2B
Data: 213k listwise groups — 107k text groups (NQ, TriviaQA, MS MARCO; each a [pos, neg_0, neg_1, neg_2] 4-list with hard negatives mined via the NV-Retriever approach with GTE-ModernBERT) + 106k vision groups (ColPali train set with negatives mined by Nomic). The gold permutation (pos > neg_0 > neg_1 > neg_2) is constructed directly from the mining metadata; training is cross-entropy on the permutation tokens only.
Languages: English (training), French (zero-shot transfer)
Requirements: transformers >= 5.4.0 (qwen3_5 architecture)

Usage: generative listwise reranking

python
import re
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "lightonai/LightOn-rerank-LW-0.8B"
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",  # optional, remove if flash-attn is not installed
    device_map="cuda",
).eval()
processor = AutoProcessor.from_pretrained(model_id)

PROMPT = "<|im_start|>user\n{user}<|im_end|>\n<|im_start|>assistant\n"
PERM_RE = re.compile(r"\[(\d)\]\s*>\s*\[(\d)\]\s*>\s*\[(\d)\]\s*>\s*\[(\d)\]")


def rank_window(query: str, docs: list[str]) -> list[int]:
    """Rank exactly 4 text passages; returns window indices, most relevant first."""
    body = "\n".join(f"[{i + 1}]: {d}" for i, d in enumerate(docs))
    user = f"Query: {query}\n\nRank these passages from most to least relevant:\n{body}\n\nRanking:"
    inputs = processor(text=[PROMPT.format(user=user)], return_tensors="pt").to(model.device)
    out = model.generate(
        **inputs, max_new_tokens=30, do_sample=False,
        pad_token_id=processor.tokenizer.eos_token_id,
    )
    completion = processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
    m = PERM_RE.search(completion)
    return [int(g) - 1 for g in m.groups()] if m else list(range(4))


def rerank(query: str, docs: list, window: int = 4, stride: int = 2) -> list[int]:
    """Sliding-window listwise rerank; returns document indices, most relevant first."""
    order = list(range(len(docs)))
    positions = list(range(max(0, len(docs) - window), -1, -stride))
    if positions and positions[-1] != 0:
        positions.append(0)
    for pos in positions:
        end = min(pos + window, len(docs))
        p = max(0, end - window)
        if end - p < 2:
            continue
        perm = rank_window(query, [docs[i] for i in order[p:end]])
        order[p:end] = [order[p + j] for j in perm]
    return order


query = "What is late interaction in neural information retrieval?"
documents = ["passage 1 ...", "passage 2 ...", "passage 3 ...", "passage 4 ...", "passage 5 ..."]
print(rerank(query, documents))

For page images, build the window prompt with interleaved image placeholders instead:

python
def rank_window_images(query: str, images: list) -> list[int]:  # 4 PIL images
    content = [{"type": "text", "text": f"Query: {query}\n\nRank these documents from most to least relevant:\n"}]
    for i, img in enumerate(images):
        content += [
            {"type": "text", "text": f"[{i + 1}]: "},
            {"type": "image", "image": img},
            {"type": "text", "text": "\n"},
        ]
    content.append({"type": "text", "text": "\nRanking:"})
    text = processor.apply_chat_template(
        [{"role": "user", "content": content}], tokenize=False, add_generation_prompt=True
    )
    inputs = processor(text=[text], images=list(images), return_tensors="pt").to(model.device)
    out = model.generate(
        **inputs, max_new_tokens=30, do_sample=False,
        pad_token_id=processor.tokenizer.eos_token_id,
    )
    completion = processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
    m = PERM_RE.search(completion)
    return [int(g) - 1 for g in m.groups()] if m else list(range(4))

If a window has fewer than 4 candidates, pad it by repeating the last candidate and drop the duplicates from the returned order.

Notes & limitations

Weakest domain relative to 2B-class models is pharmaceuticals (≈−6 nDCG points); strongest wins over Qwen3-VL-Reranker-2B are computer_science, hr, energy and physics (EN).
Listwise reranking generates the full permutation (~18 decoded tokens per window, ≈ n/2 windows for n candidates), so per-query latency is higher than pointwise scoring — the trade for markedly better ranking quality at this size. At rerank-100 the larger PW-2B is both faster and slightly stronger; this model's niche is memory-constrained deployment.
Training data is English-only. French works zero-shot (the backbone is multilingual) but is slightly behind English on average.
BEIR contamination flag: NQ and MSMARCO are part of the text training data; headline text figures use decontaminated means that exclude them.

The LightOn-rerank family

Table with columns: Model, Backbone, Scoring, ViDoRe V3 overall nDCG@10
Model	Backbone	Scoring	ViDoRe V3 overall nDCG@10
LightOn-rerank-PW-0.8B	Qwen3.5-0.8B	pointwise	48.20
LightOn-rerank-LW-0.8B	Qwen3.5-0.8B	listwise	58.25
LightOn-rerank-PW-2B	Qwen3.5-2B	pointwise	59.87
LightOn-rerank-LW-2B	Qwen3.5-2B	listwise	62.66
LightOn-rerank-PW-4B	Qwen3.5-4B	pointwise	59.80
LightOn-rerank-LW-4B	Qwen3.5-4B	listwise	64.69

Citation

markdown
@misc{ananya2026lightonrerank,
  title={One Adapter, Both Modalities: Field Notes from Building and Serving a Multimodal Reranker},
  author={Ananya, Ishrat Jahan and Chatelain, Amelie},
  year={2026},
  howpublished={\url{https://huggingface.co/blog/lightonai/lighton-rerank}},
}