UNI-PIC

retail-qwen3-reranker-0.6b-tw

README

License: apache-2.0

Model Details

Table

Developed by	President Information Corp. (統一資訊, PIC) and Uni-President Information Philippines Corp. (UNI-PIPC)
Model type	Generative (yes/no) reranker, `Qwen3ForCausalLM`
Base model	`Qwen/Qwen3-Reranker-0.6B`
Adaptation method	Full fine-tuning, listwise ranking loss
Parameters	0.6B
Max sequence length	256 tokens (training and evaluation)
Domain	Taiwan retail and e-commerce product search
Languages	Traditional Chinese (zh-TW)
License	Apache 2.0

Key Features

🛒 Retail Search Adaptation: Trained on real shopper search queries whose candidate products were graded by an LLM judge, with judge-graded hard negatives from a retrieval stack.
🧩 Built for Blending: The reranker is most useful together with a dense retriever's score; the gain from the blend holds from a 39k-title catalog up to a 2.2M-title multi-source index.
⚡ Small: About 0.6B parameters. We measured roughly 700 to 770 (query, title) pairs per second on one H100 with plain Hugging Face bf16 inference (batch 64, no serving optimizations), so reranking 50 candidates costs about 65 to 70 ms of GPU time per query.
🇹🇼 Taiwan Query Style: Short brand and product-type keywords, mixed Chinese-English queries, and descriptive queries, with and without keyword tags.

Training Data

The training data is a proprietary Taiwan retail (zh-TW) dataset and is not publicly available. It consists of 20,413 listwise groups built from 8,560 real shopper search queries. Each group holds one positive product title and seven hard negatives. An LLM judge graded each (query, product title) pair as 0, 1, or 2; grade-2 products are positives and grade-0 products are negatives. Grade-1 products are never used as negatives, near-identical variants of a product are not paired as negatives, and titles that are negatives for many different queries are excluded. In 70% of the groups the query text carries keyword tags. Held-out evaluation queries were excluded from training.

Training Procedure

Table

Framework	ms-swift (`generative_reranker`, `listwise_reranker` loss, temperature 1.0)
Group size	1 positive + 7 negatives
Batch	32 groups per optimizer step (4 per device x 8 accumulation)
Optimizer	Learning rate 1e-5, 5% warm-up
Precision	bf16, full fine-tuning
Epochs / steps	1 epoch, 625 steps
Max sequence length	256 tokens
Training hardware	1x NVIDIA H100

Evaluation

Evaluation Setup

The setup follows the retrieval benchmark of UNI-PIC/retail-embeddinggemma-2-tw. Queries are real shopper search queries that were never used in training (split by query group), restricted to queries with at least one relevant product in the judged pool: a development split (1,023 queries) and a held-out test split (368 queries) that was scored once. The first stage is the fine-tuned retriever (UNI-PIC/retail-embeddinggemma-2-tw, query + tags); the reranker scores its top 50 candidates and the blend is 0.5 * z(reranker) + 0.5 * z(retriever score), with z-scores computed within each query's candidate list. The blend weight is the untuned default. Scores here are not directly comparable with the table on the retriever's own card: the ideal ranking behind nDCG@10 is pooled over the systems compared in one run, so the embedding model alone scores slightly differently in each run.

Relevance is graded by an LLM judge (DeepSeek-V4-Flash) as 0, 1, or 2. Every product returned by any compared system is graded, so no system is penalized for retrieving a product that was not graded. Metrics are nDCG@10 against all graded products of each query and strict P@1 (the top-ranked product has grade 2); higher is better. Confidence intervals are 95% paired bootstrap intervals (5,000 resamples over queries).

🟢 improvement · 🔴 degradation · ⚪ no change (in the gain figures, 🟢 indicates that the 95% confidence interval excludes zero)

Recall of the first stage limits what any reranker can do: the top 50 candidates contain 94.5% (dev) and 94.9% (test) of the grade-2 products.

Held-out Test Split (368 queries, 39k-title catalog)

83.7%84.2%86.3%+2.7 P@182.1%81.5%85.9%+3.8

Table with columns: System, nDCG@10, P@1, nDCG@10 gain vs embedding model alone, 95% CI
System	nDCG@10	P@1	nDCG@10 gain vs embedding model alone	95% CI
Embedding model alone	0.837	0.821	(reference)
Embedding model + BM25 hybrid (RRF)	0.843	0.812	+0.006	[-0.016, +0.029]
Fine-tuned reranker	0.842	0.815	+0.005	[-0.016, +0.027]
Fine-tuned reranker + embedding model	0.863	0.859	+0.027	[+0.012, +0.042]

Development Split (1,023 queries, 39k-title catalog)

The stock Qwen3-Reranker-0.6B (with the same retail instruction and the same blend) is included as a control. It gains far less than the fine-tuned model, so the improvement comes from the fine-tuning and not from blending alone.

85.4%85.0%84.6%87.5%+2.1 P@186.0%84.4%83.0%88.2%+2.2

Table with columns: System, nDCG@10, P@1, nDCG@10 gain vs embedding model alone, 95% CI
System	nDCG@10	P@1	nDCG@10 gain vs embedding model alone	95% CI
Embedding model alone	0.854	0.860	(reference)
Embedding model + BM25 hybrid (RRF)	0.841	0.837	-0.013	[-0.025, -0.000]
Stock reranker + embedding model	0.850	0.844	-0.004	[-0.012, +0.004]
Fine-tuned reranker	0.846	0.830	-0.008	[-0.020, +0.004]
Fine-tuned reranker + embedding model	0.875	0.882	+0.021	[+0.014, +0.029]

Larger Multi-Source Index

To check that the gain does not depend on the small benchmark catalog, the 39k benchmark catalog was hidden inside a much larger index built from other proprietary Taiwan retail product catalogs (deduplicated by normalized title, 2.0M and 2.2M titles in total). Retrieval and reranking were repeated with the same top-50 reranking, and every newly retrieved product was judged.

+2.7[1.2, 4.2] Dev · 39k catalog+2.1[1.4, 2.9] Test · 2.0M titles+3.0[2.2, 3.9] Test · 2.2M titles+3.5[2.7, 4.4] Dev · 2.0M titles+2.6[2.1, 3.1] Dev · 2.2M titles+2.6[2.1, 3.2]

Table with columns: Index, Embedding model P@1, Reranker alone P@1, Blend P@1, Embedding model nDCG@10, Blend nDCG@10, Blend gain in nDCG@10, 95% CI
Index	Embedding model P@1	Reranker alone P@1	Blend P@1	Embedding model nDCG@10	Blend nDCG@10	Blend gain in nDCG@10	95% CI
39k (benchmark catalog)	0.818	0.818	0.856	0.686	0.709	+0.023	[+0.014, +0.034]
2.0M titles	0.810	0.867	0.856	0.844	0.874	+0.030	[+0.022, +0.039]
2.2M titles	0.804	0.875	0.853	0.848	0.882	+0.035	[+0.027, +0.044]

Notes on reading this table:

nDCG@10 uses an ideal ranking pooled over every judged product of the query, which now includes relevant products from the larger index. Because of that, nDCG@10 is only comparable within one row, not across index sizes. Within a row, the blend improves on the embedding model alone with a confidence interval that excludes zero.
At 2M titles, more than 90% of the top-10 results come from outside the benchmark catalog. The judge sees titles only, so an equivalent product from another catalog can count as relevant.
At this size the reranker alone has a higher P@1 than the 50/50 blend. The blend weight was not tuned; if P@1 matters most, tune the weight on your own data.
The same table on the development split: gain of the blend over the embedding model alone in nDCG@10 is 39k +0.015, 2.0M +0.026, 2.2M +0.026.

Usage

The reranker uses the standard Qwen3-Reranker yes/no scoring (log-odds of yes versus no at the last position) with a retail instruction. rerank.py in this repository implements it together with the blend:

python

from rerank import RetailReranker rr = RetailReranker("UNI-PIC/retail-qwen3-reranker-0.6b-tw") query = "洗衣精"titles = ["白蘭 洗衣精 補充包 1.6kg", "Hampton 漢汀堡 迪亞小圓桌 寶石藍", "一匙靈 抗菌洗衣精 2L"]dense = [0.71, 0.32, 0.69]                       # similarity scores from the first-stage retriever order, scores = rr.rank(query, titles, dense_scores=dense)    # blend (recommended)order, scores = rr.rank(query, titles)                        # reranker score onlyprint([titles[i] for i in order])

In the evaluation, the retriever input was query + " " + tags when keyword tags were available, and the reranker saw the same text. Reranker inputs without tags were not separately evaluated.

Intended Use and Limitations

Intended use: Second-stage reranking of 20 to 100 candidates for zh-TW retail product search, blended with a dense retriever's score.
Candidate depth: Gains were measured at 50 candidates (recall of the first stage at 50 is about 95%); a depth of 20 gave a smaller gain in our tests.
Judge-based evaluation: Training labels and benchmark labels come from the same family of LLM judge, and there are no human annotators or click data. Absolute scores should be read as comparisons between systems under this setup, not as production accuracy, and shared judge preferences may make the gain look somewhat larger than a human-judged gain.
Modest gain: About +0.03 nDCG@10 over a strong fine-tuned retriever. It is not a general-purpose reranker and was not evaluated outside Taiwan retail titles.
Single training catalog: Training used one proprietary catalog and its search logs. The larger-index experiment shows robustness at scale, not adaptation to other catalogs.
Titles only: Documents were product titles; brand, price and description fields were not used.
Language scope: Taiwan Traditional Chinese only.

License and Attribution

This model is released under the Apache License 2.0 (see LICENSE). It is a fine-tuned derivative of Qwen/Qwen3-Reranker-0.6B by the Qwen team, which is also licensed under Apache 2.0; all model weights were modified by fine-tuning.

Citation

bibtex

@misc{retailqwen3reranker06btw,  title     = {retail-qwen3-reranker-0.6b-tw},  author    = {President Information Corp. (PIC) and Uni-President Information Philippines Corp. (UNI-PIPC)},  year      = {2026},  publisher = {Hugging Face},  url       = {https://huggingface.co/UNI-PIC/retail-qwen3-reranker-0.6b-tw}}

Contact

Taiwan: pic.net.tw/contactUs
Philippines: pic.net.ph/about/contactUs

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

UNI-PIC

Model Tree

Base

Qwen/Qwen3-Reranker-0.6B

Fine-tuned

this model

Input Modalities

Text

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

README

License: apache-2.0

Model Details

Table

Developed by	President Information Corp. (統一資訊, PIC) and Uni-President Information Philippines Corp. (UNI-PIPC)
Model type	Generative (yes/no) reranker, `Qwen3ForCausalLM`
Base model	`Qwen/Qwen3-Reranker-0.6B`
Adaptation method	Full fine-tuning, listwise ranking loss
Parameters	0.6B
Max sequence length	256 tokens (training and evaluation)
Domain	Taiwan retail and e-commerce product search
Languages	Traditional Chinese (zh-TW)
License	Apache 2.0

Key Features

🛒 Retail Search Adaptation: Trained on real shopper search queries whose candidate products were graded by an LLM judge, with judge-graded hard negatives from a retrieval stack.
🧩 Built for Blending: The reranker is most useful together with a dense retriever's score; the gain from the blend holds from a 39k-title catalog up to a 2.2M-title multi-source index.
⚡ Small: About 0.6B parameters. We measured roughly 700 to 770 (query, title) pairs per second on one H100 with plain Hugging Face bf16 inference (batch 64, no serving optimizations), so reranking 50 candidates costs about 65 to 70 ms of GPU time per query.
🇹🇼 Taiwan Query Style: Short brand and product-type keywords, mixed Chinese-English queries, and descriptive queries, with and without keyword tags.

Training Data

Training Procedure

Table

Framework	ms-swift (`generative_reranker`, `listwise_reranker` loss, temperature 1.0)
Group size	1 positive + 7 negatives
Batch	32 groups per optimizer step (4 per device x 8 accumulation)
Optimizer	Learning rate 1e-5, 5% warm-up
Precision	bf16, full fine-tuning
Epochs / steps	1 epoch, 625 steps
Max sequence length	256 tokens
Training hardware	1x NVIDIA H100

Evaluation

Evaluation Setup

🟢 improvement · 🔴 degradation · ⚪ no change (in the gain figures, 🟢 indicates that the 95% confidence interval excludes zero)

Recall of the first stage limits what any reranker can do: the top 50 candidates contain 94.5% (dev) and 94.9% (test) of the grade-2 products.

Held-out Test Split (368 queries, 39k-title catalog)

83.7%84.2%86.3%+2.7 P@182.1%81.5%85.9%+3.8

Table with columns: System, nDCG@10, P@1, nDCG@10 gain vs embedding model alone, 95% CI
System	nDCG@10	P@1	nDCG@10 gain vs embedding model alone	95% CI
Embedding model alone	0.837	0.821	(reference)
Embedding model + BM25 hybrid (RRF)	0.843	0.812	+0.006	[-0.016, +0.029]
Fine-tuned reranker	0.842	0.815	+0.005	[-0.016, +0.027]
Fine-tuned reranker + embedding model	0.863	0.859	+0.027	[+0.012, +0.042]

Development Split (1,023 queries, 39k-title catalog)

85.4%85.0%84.6%87.5%+2.1 P@186.0%84.4%83.0%88.2%+2.2

Table with columns: System, nDCG@10, P@1, nDCG@10 gain vs embedding model alone, 95% CI
System	nDCG@10	P@1	nDCG@10 gain vs embedding model alone	95% CI
Embedding model alone	0.854	0.860	(reference)
Embedding model + BM25 hybrid (RRF)	0.841	0.837	-0.013	[-0.025, -0.000]
Stock reranker + embedding model	0.850	0.844	-0.004	[-0.012, +0.004]
Fine-tuned reranker	0.846	0.830	-0.008	[-0.020, +0.004]
Fine-tuned reranker + embedding model	0.875	0.882	+0.021	[+0.014, +0.029]

Larger Multi-Source Index

+2.7[1.2, 4.2] Dev · 39k catalog+2.1[1.4, 2.9] Test · 2.0M titles+3.0[2.2, 3.9] Test · 2.2M titles+3.5[2.7, 4.4] Dev · 2.0M titles+2.6[2.1, 3.1] Dev · 2.2M titles+2.6[2.1, 3.2]

Table with columns: Index, Embedding model P@1, Reranker alone P@1, Blend P@1, Embedding model nDCG@10, Blend nDCG@10, Blend gain in nDCG@10, 95% CI
Index	Embedding model P@1	Reranker alone P@1	Blend P@1	Embedding model nDCG@10	Blend nDCG@10	Blend gain in nDCG@10	95% CI
39k (benchmark catalog)	0.818	0.818	0.856	0.686	0.709	+0.023	[+0.014, +0.034]
2.0M titles	0.810	0.867	0.856	0.844	0.874	+0.030	[+0.022, +0.039]
2.2M titles	0.804	0.875	0.853	0.848	0.882	+0.035	[+0.027, +0.044]

Notes on reading this table:

nDCG@10 uses an ideal ranking pooled over every judged product of the query, which now includes relevant products from the larger index. Because of that, nDCG@10 is only comparable within one row, not across index sizes. Within a row, the blend improves on the embedding model alone with a confidence interval that excludes zero.
At 2M titles, more than 90% of the top-10 results come from outside the benchmark catalog. The judge sees titles only, so an equivalent product from another catalog can count as relevant.
At this size the reranker alone has a higher P@1 than the 50/50 blend. The blend weight was not tuned; if P@1 matters most, tune the weight on your own data.
The same table on the development split: gain of the blend over the embedding model alone in nDCG@10 is 39k +0.015, 2.0M +0.026, 2.2M +0.026.

Usage

python

from rerank import RetailReranker rr = RetailReranker("UNI-PIC/retail-qwen3-reranker-0.6b-tw") query = "洗衣精"titles = ["白蘭 洗衣精 補充包 1.6kg", "Hampton 漢汀堡 迪亞小圓桌 寶石藍", "一匙靈 抗菌洗衣精 2L"]dense = [0.71, 0.32, 0.69]                       # similarity scores from the first-stage retriever order, scores = rr.rank(query, titles, dense_scores=dense)    # blend (recommended)order, scores = rr.rank(query, titles)                        # reranker score onlyprint([titles[i] for i in order])

from rerank import RetailReranker rr = RetailReranker("UNI-PIC/retail-qwen3-reranker-0.6b-tw") query = "洗衣精"titles = ["白蘭 洗衣精 補充包 1.6kg", "Hampton 漢汀堡 迪亞小圓桌 寶石藍", "一匙靈 抗菌洗衣精 2L"]dense = [0.71, 0.32, 0.69]                       # similarity scores from the first-stage retriever order, scores = rr.rank(query, titles, dense_scores=dense)    # blend (recommended)order, scores = rr.rank(query, titles)                        # reranker score onlyprint([titles[i] for i in order])

In the evaluation, the retriever input was query + " " + tags when keyword tags were available, and the reranker saw the same text. Reranker inputs without tags were not separately evaluated.

Intended Use and Limitations

Intended use: Second-stage reranking of 20 to 100 candidates for zh-TW retail product search, blended with a dense retriever's score.
Candidate depth: Gains were measured at 50 candidates (recall of the first stage at 50 is about 95%); a depth of 20 gave a smaller gain in our tests.
Judge-based evaluation: Training labels and benchmark labels come from the same family of LLM judge, and there are no human annotators or click data. Absolute scores should be read as comparisons between systems under this setup, not as production accuracy, and shared judge preferences may make the gain look somewhat larger than a human-judged gain.
Modest gain: About +0.03 nDCG@10 over a strong fine-tuned retriever. It is not a general-purpose reranker and was not evaluated outside Taiwan retail titles.
Single training catalog: Training used one proprietary catalog and its search logs. The larger-index experiment shows robustness at scale, not adaptation to other catalogs.
Titles only: Documents were product titles; brand, price and description fields were not used.
Language scope: Taiwan Traditional Chinese only.

License and Attribution

Citation

bibtex

@misc{retailqwen3reranker06btw,  title     = {retail-qwen3-reranker-0.6b-tw},  author    = {President Information Corp. (PIC) and Uni-President Information Philippines Corp. (UNI-PIPC)},  year      = {2026},  publisher = {Hugging Face},  url       = {https://huggingface.co/UNI-PIC/retail-qwen3-reranker-0.6b-tw}}

@misc{retailqwen3reranker06btw,  title     = {retail-qwen3-reranker-0.6b-tw},  author    = {President Information Corp. (PIC) and Uni-President Information Philippines Corp. (UNI-PIPC)},  year      = {2026},  publisher = {Hugging Face},  url       = {https://huggingface.co/UNI-PIC/retail-qwen3-reranker-0.6b-tw}}

Contact

Taiwan: pic.net.tw/contactUs
Philippines: pic.net.ph/about/contactUs