lightonai

LightOn-rerank-PW-4B

README

License: apache-2.0

About the LightOn-rerank family

Production retrieval pipelines usually need two rerankers: one for text passages and one for visual documents (PDF pages, slides, scans). LightOn-rerank models are unified cross-encoder rerankers: a single model scores both text passages and document page images against a query, on top of any first-stage retriever (BM25, dense embeddings, or ColPali-family late-interaction models).

The models are built on Qwen3.5 backbone (hybrid linear + full attention) and jointly fine-tuned on text and visual reranking data with mixed-modality batches (LoRA, merged into the released weights). Training data is English-only; French performance transfers zero-shot from the multilingual backbone.

The family comes in two scoring flavours × three sizes (0.8B / 2B / 4B):

PW (pointwise): each candidate is scored independently. The model judges whether the document answers the query, and the score is logit("Yes") − logit("No"). One forward pass per candidate and no generation.
LW (listwise): generative listwise ranking, where 4 candidates are placed in a single prompt and the model generates a permutation ([2] > [4] > [1] > [3]). Larger candidate pools are ranked with a sliding window (window 4, stride 2, bottom-to-top). Cross-document attention makes LW markedly stronger on hard visual reranking, and unlike pointwise scoring it keeps improving with backbone size.

LightOn-rerank-PW-4B is the 4B pointwise member of the family. On visual reranking it ties the 2B pointwise model (59.80 vs 59.87 nDCG@10 on ViDoRe V3): direct evidence that independent Yes/No scoring is a capacity bottleneck that extra parameters cannot fix. On text BEIR, however, it posts the strongest scores in the family (50.19 decontaminated mean). For 4B-budget vision-heavy deployments, LightOn-rerank-LW-4B is +4.9 nDCG@10 at the same size.

Results

ViDoRe V3 (visual document reranking, 8 domains × EN/FR queries), overall nDCG@10, ColQwen2.5-v0.2 first stage, retrieve 100 / rerank 100. All models, including baselines, were re-evaluated under this same two-stage protocol, so numbers are mutually comparable but not comparable to vendor-reported end-to-end results.

Table with columns: Model, Params, Scoring, ViDoRe V3 overall nDCG@10
Model	Params	Scoring	ViDoRe V3 overall nDCG@10
LightOn-rerank-LW-4B	4.5B	listwise	64.69
Qwen3-VL-Reranker-8B	8B	pointwise (pooling)	64.23
LightOn-rerank-LW-2B	2.2B	listwise	62.66
LightOn-rerank-PW-2B	2.2B	pointwise	59.87
LightOn-rerank-PW-4B (this model)	4.5B	pointwise	59.80
jina-reranker-m0	2.4B	pointwise	59.40
Qwen3-VL-Reranker-2B	2B	pointwise (pooling)	59.18
LightOn-rerank-LW-0.8B	0.85B	listwise	58.25
MonoQwen2-VL-v0.1	2B	pointwise	57.76
First-stage only (ColQwen2.5, no rerank)	—	—	55.60
LightOn-rerank-PW-0.8B	0.85B	pointwise	48.20

ViDoRe V3 detail (nDCG@10, ColQwen2.5 first stage, rerank-100)

Table with columns: Domain, EN, FR
Domain	EN	FR
finance_en	70.98	68.54
finance_fr	45.96	37.92
computer_science	79.25	74.71
hr	58.67	53.54
energy	70.95	72.36
industrial	60.54	47.33
pharmaceuticals	68.11	67.70
physics	44.46	35.77
mean	62.37	57.23

Overall nDCG@10: 59.80 (EN 62.37 / FR 57.23).

BEIR results (text reranking)

13 datasets, nDCG@10, BM25 first stage, retrieve 100 / rerank 100, same protocol as all other family members. ⚠️ marks datasets in the text training mix (NQ, MSMARCO); the decontaminated mean excludes them.

Table with columns: Dataset, nDCG@10
Dataset	nDCG@10
fever	81.77
scifact	76.13
trec-covid	79.38
hotpotqa	70.79
nq ⚠️	61.03
dbpedia	44.21
arguana	41.47
fiqa	40.91
msmarco ⚠️	39.20
nfcorpus	37.41
touche-2020	35.68
climate-fever	26.46
scidocs	17.87
Mean (13)	50.18
Decontaminated mean (11, excl. ⚠️)	50.19

This is the best text result in the LightOn-rerank family (2B pointwise: 49.13, 4B listwise: 48.33 decontaminated mean). Pointwise plateaus with scale on vision but not on text. If your workload is text-dominant and you have the 4B budget, this is the family's strongest text reranker.

Model Details

Model type: multimodal cross-encoder reranker (pointwise: each candidate is scored independently as logit("Yes") − logit("No"))
Base model: Qwen/Qwen3.5-4B (Qwen3.5 hybrid linear + full attention VLM)
Parameters: ≈4.5B (bfloat16, 9.1 GB)
Inputs: query (text) + candidate document(s): text passage or page image
Fine-tuning: joint text+vision LoRA (r=32, α=32, rsLoRA, merged into the released weights), mixed-modality batches (2 text + 2 vision groups per micro-batch), vision loss weight 1.3, lr 1e-4, 1 epoch (465 steps)
Data: same 213k groups as the listwise models — 107k text groups (NQ, TriviaQA, MS MARCO; each a [pos, neg_0, neg_1, neg_2] 4-list with hard negatives mined via the NV-Retriever approach with GTE-ModernBERT) + 106k vision groups (ColPali train set with negatives mined by Nomic). For pointwise training each group is flattened into (query, document, Yes/No) triples.
Languages: English (training), French (zero-shot transfer)
Requirements: transformers >= 5.4.0 (qwen3_5 architecture); sentence-transformers >= 5.4.0 for the CrossEncoder usage

Usage: pointwise reranking

Each candidate is scored independently as logit("Yes") − logit("No") at the first generated position; sort candidates by descending score. The model was trained with a fixed system prompt and user template: use them verbatim for best results.

Using Sentence Transformers

Install Sentence Transformers (>= 5.4.0):

bash
pip install "sentence-transformers[image]"

The trained system prompt and user templates are baked into the bundled reranker chat template — including the non-thinking generation prompt, so no enable_thinking handling is needed — and query-document pairs are formatted correctly out of the box:

python
from sentence_transformers import CrossEncoder

model = CrossEncoder("lightonai/LightOn-rerank-PW-4B")

query = "What is late interaction in neural information retrieval?"
documents = [
    "ColBERT computes token-level query-document interactions at search time...",
    "The Eiffel Tower is located on the Champ de Mars in Paris.",
]

pairs = [(query, doc) for doc in documents]
scores = model.predict(pairs)
print(scores)
# [-3.     -8.0625]

rankings = model.rank(query, documents)
print(rankings)
# [{'corpus_id': 0, 'score': -3.0}, {'corpus_id': 1, 'score': -8.0625}]

To rerank page images, pass a PIL.Image (or an image URL or file path string) as the document. Text and image candidates can be mixed in the same call:

python
from PIL import Image

pairs = [
    (query, Image.open("page_1.png")),
    (query, "https://example.com/page_2.png"),
    (query, "A text passage candidate for the same query."),
]
scores = model.predict(pairs)

Scores are raw logit("Yes") − logit("No") differences. You can map them to 0...1 probabilities with model.predict(pairs, activation_fn=torch.nn.Sigmoid()).

Using Transformers

python
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "lightonai/LightOn-rerank-PW-4B"
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",  # optional, remove if flash-attn is not installed
    device_map="cuda",
).eval()
processor = AutoProcessor.from_pretrained(model_id)
processor.tokenizer.padding_side = "left"  # scores are read at the last position

YES_TOKEN_ID = 9175  # "Yes"
NO_TOKEN_ID = 2665   # "No"

SYSTEM_PROMPT = "Judge whether the document is relevant to the query. Answer Yes or No."
USER_TEMPLATE = (
    "Given a query, determine if the document is relevant. "
    "The query is: {query}\n\nDocument: {doc}"
)

query = "What is late interaction in neural information retrieval?"
documents = [
    "ColBERT computes token-level query-document interactions at search time...",
    "The Eiffel Tower is located on the Champ de Mars in Paris.",
]

texts = [
    processor.apply_chat_template(
        [
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": USER_TEMPLATE.format(query=query, doc=doc)},
        ],
        tokenize=False,
        add_generation_prompt=True,
        enable_thinking=False,  # REQUIRED on the 4B backbone (thinking on by default)
    )
    for doc in documents
]
inputs = processor(
    text=texts, return_tensors="pt", padding=True, truncation=True, max_length=2048
).to(model.device)
with torch.inference_mode():
    logits = model(**inputs).logits[:, -1]
scores = (logits[:, YES_TOKEN_ID] - logits[:, NO_TOKEN_ID]).tolist()
# [-3.0, -8.0625]

ranked = sorted(zip(scores, documents), reverse=True)

To score a page image instead of a text passage, replace the user message with:

python
VISION_TEMPLATE = "Given a query, determine if the document image is relevant. The query is: {query}"

{"role": "user", "content": [
    {"type": "image", "image": page_image},  # PIL.Image
    {"type": "text", "text": VISION_TEMPLATE.format(query=query)},
]}

and pass images=[page_image, ...] to the processor call (keep the same system prompt).

Serving with vLLM

bash
vllm serve lightonai/LightOn-rerank-PW-4B --trust-remote-code --max-model-len 16384

python
resp = client.chat.completions.create(
    model="lightonai/LightOn-rerank-PW-4B",
    messages=messages,          # same system + user messages as above
    max_tokens=1,
    logprobs=True,
    top_logprobs=20,
    temperature=0.0,
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},  # REQUIRED
)
top = resp.choices[0].logprobs.content[0].top_logprobs
lp = {t.token: t.logprob for t in top}
score = lp.get("Yes", -100.0) - lp.get("No", -100.0)

Full-page document images can exceed 8k tokens, so keep --max-model-len at 16384 or higher when reranking page images.

Notes & limitations

Thinking must be disabled for scoring. Qwen3.5-4B's chat template enables <think> by default; with it on, the first generated token is a thinking token and the Yes/No logprobs are distorted. Pass enable_thinking=False to apply_chat_template (or chat_template_kwargs={"enable_thinking": False} via the vLLM OpenAI client) as shown above. The Sentence Transformers path handles this automatically: the bundled reranker chat template hardcodes the non-thinking generation prompt.
Pointwise scoring does not benefit from the 2B→4B scale-up on vision (59.80 vs 59.87 for the 2B), while generative listwise gains +2.0 points over the same step. For vision workloads prefer LightOn-rerank-LW-4B at this size, or LightOn-rerank-PW-2B for the same visual quality at lower cost.
Training data is English-only. French works zero-shot (the backbone is multilingual) but is slightly behind English on average.
BEIR contamination flag: NQ and MSMARCO are part of the text training data; headline text figures use decontaminated means that exclude them.

The LightOn-rerank family

Table with columns: Model, Backbone, Scoring, ViDoRe V3 overall nDCG@10
Model	Backbone	Scoring	ViDoRe V3 overall nDCG@10
LightOn-rerank-PW-0.8B	Qwen3.5-0.8B	pointwise	48.20
LightOn-rerank-LW-0.8B	Qwen3.5-0.8B	listwise	58.25
LightOn-rerank-PW-2B	Qwen3.5-2B	pointwise	59.87
LightOn-rerank-LW-2B	Qwen3.5-2B	listwise	62.66
LightOn-rerank-PW-4B	Qwen3.5-4B	pointwise	59.80
LightOn-rerank-LW-4B	Qwen3.5-4B	listwise	64.69

Rule of thumb: LW models are stronger at every size (and the gap grows with size); PW models are cheaper to serve and score candidates independently. For the best quality pick LW-4B; for the best quality/cost trade-off pick LW-2B; for maximum throughput on text-heavy workloads pick a PW model.

Citation

markdown
@misc{ananya2026lightonrerank,
  title={One Adapter, Both Modalities: Field Notes from Building and Serving a Multimodal Reranker},
  author={Ananya, Ishrat Jahan and Chatelain, Amelie},
  year={2026},
  howpublished={\url{https://huggingface.co/blog/lightonai/lighton-rerank}},
}

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

lightonai

Model Tree

Base

Qwen/Qwen3.5-4B

Fine-tuned

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

About the LightOn-rerank family

The family comes in two scoring flavours × three sizes (0.8B / 2B / 4B):

PW (pointwise): each candidate is scored independently. The model judges whether the document answers the query, and the score is logit("Yes") − logit("No"). One forward pass per candidate and no generation.
LW (listwise): generative listwise ranking, where 4 candidates are placed in a single prompt and the model generates a permutation ([2] > [4] > [1] > [3]). Larger candidate pools are ranked with a sliding window (window 4, stride 2, bottom-to-top). Cross-document attention makes LW markedly stronger on hard visual reranking, and unlike pointwise scoring it keeps improving with backbone size.

Results

Table with columns: Model, Params, Scoring, ViDoRe V3 overall nDCG@10
Model	Params	Scoring	ViDoRe V3 overall nDCG@10
LightOn-rerank-LW-4B	4.5B	listwise	64.69
Qwen3-VL-Reranker-8B	8B	pointwise (pooling)	64.23
LightOn-rerank-LW-2B	2.2B	listwise	62.66
LightOn-rerank-PW-2B	2.2B	pointwise	59.87
LightOn-rerank-PW-4B (this model)	4.5B	pointwise	59.80
jina-reranker-m0	2.4B	pointwise	59.40
Qwen3-VL-Reranker-2B	2B	pointwise (pooling)	59.18
LightOn-rerank-LW-0.8B	0.85B	listwise	58.25
MonoQwen2-VL-v0.1	2B	pointwise	57.76
First-stage only (ColQwen2.5, no rerank)	—	—	55.60
LightOn-rerank-PW-0.8B	0.85B	pointwise	48.20

ViDoRe V3 detail (nDCG@10, ColQwen2.5 first stage, rerank-100)

Table with columns: Domain, EN, FR
Domain	EN	FR
finance_en	70.98	68.54
finance_fr	45.96	37.92
computer_science	79.25	74.71
hr	58.67	53.54
energy	70.95	72.36
industrial	60.54	47.33
pharmaceuticals	68.11	67.70
physics	44.46	35.77
mean	62.37	57.23

Overall nDCG@10: 59.80 (EN 62.37 / FR 57.23).

BEIR results (text reranking)

Table with columns: Dataset, nDCG@10
Dataset	nDCG@10
fever	81.77
scifact	76.13
trec-covid	79.38
hotpotqa	70.79
nq ⚠️	61.03
dbpedia	44.21
arguana	41.47
fiqa	40.91
msmarco ⚠️	39.20
nfcorpus	37.41
touche-2020	35.68
climate-fever	26.46
scidocs	17.87
Mean (13)	50.18
Decontaminated mean (11, excl. ⚠️)	50.19

Model Details

Model type: multimodal cross-encoder reranker (pointwise: each candidate is scored independently as logit("Yes") − logit("No"))
Base model: Qwen/Qwen3.5-4B (Qwen3.5 hybrid linear + full attention VLM)
Parameters: ≈4.5B (bfloat16, 9.1 GB)
Inputs: query (text) + candidate document(s): text passage or page image
Fine-tuning: joint text+vision LoRA (r=32, α=32, rsLoRA, merged into the released weights), mixed-modality batches (2 text + 2 vision groups per micro-batch), vision loss weight 1.3, lr 1e-4, 1 epoch (465 steps)
Data: same 213k groups as the listwise models — 107k text groups (NQ, TriviaQA, MS MARCO; each a [pos, neg_0, neg_1, neg_2] 4-list with hard negatives mined via the NV-Retriever approach with GTE-ModernBERT) + 106k vision groups (ColPali train set with negatives mined by Nomic). For pointwise training each group is flattened into (query, document, Yes/No) triples.
Languages: English (training), French (zero-shot transfer)
Requirements: transformers >= 5.4.0 (qwen3_5 architecture); sentence-transformers >= 5.4.0 for the CrossEncoder usage

Usage: pointwise reranking

Using Sentence Transformers

Install Sentence Transformers (>= 5.4.0):

bash
pip install "sentence-transformers[image]"

python
from sentence_transformers import CrossEncoder

model = CrossEncoder("lightonai/LightOn-rerank-PW-4B")

query = "What is late interaction in neural information retrieval?"
documents = [
    "ColBERT computes token-level query-document interactions at search time...",
    "The Eiffel Tower is located on the Champ de Mars in Paris.",
]

pairs = [(query, doc) for doc in documents]
scores = model.predict(pairs)
print(scores)
# [-3.     -8.0625]

rankings = model.rank(query, documents)
print(rankings)
# [{'corpus_id': 0, 'score': -3.0}, {'corpus_id': 1, 'score': -8.0625}]

To rerank page images, pass a PIL.Image (or an image URL or file path string) as the document. Text and image candidates can be mixed in the same call:

python
from PIL import Image

pairs = [
    (query, Image.open("page_1.png")),
    (query, "https://example.com/page_2.png"),
    (query, "A text passage candidate for the same query."),
]
scores = model.predict(pairs)

Scores are raw logit("Yes") − logit("No") differences. You can map them to 0...1 probabilities with model.predict(pairs, activation_fn=torch.nn.Sigmoid()).

Using Transformers

python
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "lightonai/LightOn-rerank-PW-4B"
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    dtype=torch.bfloat16,
    attn_implementation="flash_attention_2",  # optional, remove if flash-attn is not installed
    device_map="cuda",
).eval()
processor = AutoProcessor.from_pretrained(model_id)
processor.tokenizer.padding_side = "left"  # scores are read at the last position

YES_TOKEN_ID = 9175  # "Yes"
NO_TOKEN_ID = 2665   # "No"

SYSTEM_PROMPT = "Judge whether the document is relevant to the query. Answer Yes or No."
USER_TEMPLATE = (
    "Given a query, determine if the document is relevant. "
    "The query is: {query}\n\nDocument: {doc}"
)

query = "What is late interaction in neural information retrieval?"
documents = [
    "ColBERT computes token-level query-document interactions at search time...",
    "The Eiffel Tower is located on the Champ de Mars in Paris.",
]

texts = [
    processor.apply_chat_template(
        [
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": USER_TEMPLATE.format(query=query, doc=doc)},
        ],
        tokenize=False,
        add_generation_prompt=True,
        enable_thinking=False,  # REQUIRED on the 4B backbone (thinking on by default)
    )
    for doc in documents
]
inputs = processor(
    text=texts, return_tensors="pt", padding=True, truncation=True, max_length=2048
).to(model.device)
with torch.inference_mode():
    logits = model(**inputs).logits[:, -1]
scores = (logits[:, YES_TOKEN_ID] - logits[:, NO_TOKEN_ID]).tolist()
# [-3.0, -8.0625]

ranked = sorted(zip(scores, documents), reverse=True)

To score a page image instead of a text passage, replace the user message with:

python
VISION_TEMPLATE = "Given a query, determine if the document image is relevant. The query is: {query}"

{"role": "user", "content": [
    {"type": "image", "image": page_image},  # PIL.Image
    {"type": "text", "text": VISION_TEMPLATE.format(query=query)},
]}

and pass images=[page_image, ...] to the processor call (keep the same system prompt).

Serving with vLLM

bash
vllm serve lightonai/LightOn-rerank-PW-4B --trust-remote-code --max-model-len 16384

python
resp = client.chat.completions.create(
    model="lightonai/LightOn-rerank-PW-4B",
    messages=messages,          # same system + user messages as above
    max_tokens=1,
    logprobs=True,
    top_logprobs=20,
    temperature=0.0,
    extra_body={"chat_template_kwargs": {"enable_thinking": False}},  # REQUIRED
)
top = resp.choices[0].logprobs.content[0].top_logprobs
lp = {t.token: t.logprob for t in top}
score = lp.get("Yes", -100.0) - lp.get("No", -100.0)

Full-page document images can exceed 8k tokens, so keep --max-model-len at 16384 or higher when reranking page images.

Notes & limitations

Thinking must be disabled for scoring. Qwen3.5-4B's chat template enables <think> by default; with it on, the first generated token is a thinking token and the Yes/No logprobs are distorted. Pass enable_thinking=False to apply_chat_template (or chat_template_kwargs={"enable_thinking": False} via the vLLM OpenAI client) as shown above. The Sentence Transformers path handles this automatically: the bundled reranker chat template hardcodes the non-thinking generation prompt.
Pointwise scoring does not benefit from the 2B→4B scale-up on vision (59.80 vs 59.87 for the 2B), while generative listwise gains +2.0 points over the same step. For vision workloads prefer LightOn-rerank-LW-4B at this size, or LightOn-rerank-PW-2B for the same visual quality at lower cost.
Training data is English-only. French works zero-shot (the backbone is multilingual) but is slightly behind English on average.
BEIR contamination flag: NQ and MSMARCO are part of the text training data; headline text figures use decontaminated means that exclude them.

The LightOn-rerank family

Table with columns: Model, Backbone, Scoring, ViDoRe V3 overall nDCG@10
Model	Backbone	Scoring	ViDoRe V3 overall nDCG@10
LightOn-rerank-PW-0.8B	Qwen3.5-0.8B	pointwise	48.20
LightOn-rerank-LW-0.8B	Qwen3.5-0.8B	listwise	58.25
LightOn-rerank-PW-2B	Qwen3.5-2B	pointwise	59.87
LightOn-rerank-LW-2B	Qwen3.5-2B	listwise	62.66
LightOn-rerank-PW-4B	Qwen3.5-4B	pointwise	59.80
LightOn-rerank-LW-4B	Qwen3.5-4B	listwise	64.69

Citation

markdown
@misc{ananya2026lightonrerank,
  title={One Adapter, Both Modalities: Field Notes from Building and Serving a Multimodal Reranker},
  author={Ananya, Ishrat Jahan and Chatelain, Amelie},
  year={2026},
  howpublished={\url{https://huggingface.co/blog/lightonai/lighton-rerank}},
}