GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
XINLI1997
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
XINLI1997
Model Tree
Qwen/Qwen3-1.7B
Input Modalities
Output Modalities
Supported Functionality
| Role | Student after on-policy distillation with forward KL |
| What it is | On-policy distillation for 42 steps from the Qwen3-4B duplicated-ID teacher (ErrOPD-Qwen3-4B-teacher-dupid-ESCI), starting from the Qwen3-1.7B student (ErrOPD-Qwen3-1.7B-student-ESCI); objective: forward KL (cross-entropy on one teacher sample at each student prefix); seed 3001. |
| Paper reference | Table 1 (Qwen3 / ESCI duplicated ID, forward KL) |
| Task | ESCI listwise product ranking: given a shopping query and 8 candidate products, score each product's relevance (0-10) and return a strict JSON list of 8 {"product_id", "score"} objects in prompt order |
| Training data | prompts of train.jsonl of XINLI1997/ErrOPD-ESCI (1,800 lists); the student generates on them and the teacher gives per-token targets on the student's prefixes |
| Base model | Qwen/Qwen3-1.7B |
| Seed | 3001 |
| Weights | bf16 safetensors, 1 file, 4.06 GB (2.03 B stored parameters). The file also stores lm_head.weight, a bit-identical copy of the tied input embeddings. |
python
# pip install "transformers>=4.57.0" accelerate datasets torchimport torchfrom datasets import load_datasetfrom transformers import AutoModelForCausalLM, AutoTokenizer repo = "XINLI1997/ErrOPD-Qwen3-1.7B-ESCI-dupid-fKL"tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto") # one held-out prompt (system + user turns) from the evaluation splitexample = load_dataset("XINLI1997/ErrOPD-ESCI", split="validation")[0]messages = [m for m in example["messages"] if m["role"] != "assistant"] inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)output = model.generate(**inputs, max_new_tokens=512, do_sample=False)print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))The expected output is a JSON list of 8 objects {"product_id": ..., "score": ...} in the order of the products in the prompt. Prompts, decoding settings and the scoring code used in the paper are in the code repository.
Notes:
On-policy distillation for 42 steps from the Qwen3-4B duplicated-ID teacher (ErrOPD-Qwen3-4B-teacher-dupid-ESCI), starting from the Qwen3-1.7B student (ErrOPD-Qwen3-1.7B-student-ESCI); objective: forward KL (cross-entropy on one teacher sample at each student prefix); seed 3001. Full training configurations and scripts are in the code repository.
This checkpoint is reported in the paper, Table 1 (Qwen3 / ESCI duplicated ID, forward KL); see that table for its scores.
Apache-2.0, inherited from the base model Qwen/Qwen3-1.7B (see LICENSE). The training data keep their own terms (see the dataset card of XINLI1997/ErrOPD-ESCI).
bibtex
@article{li2026erropd, title = {ErrOPD: When Does On-Policy Distillation Transfer Teacher Errors?}, author = {Li, Xin and Jiang, Hao and Wang, Annan and Zhang, Yichi and Yuen, Chau}, journal = {arXiv preprint arXiv:2605.08737}, year = {2026}}# pip install "transformers>=4.57.0" accelerate datasets torchimport torchfrom datasets import load_datasetfrom transformers import AutoModelForCausalLM, AutoTokenizer repo = "XINLI1997/ErrOPD-Qwen3-1.7B-ESCI-dupid-fKL"tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto") # one held-out prompt (system + user turns) from the evaluation splitexample = load_dataset("XINLI1997/ErrOPD-ESCI", split="validation")[0]messages = [m for m in example["messages"] if m["role"] != "assistant"] inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)output = model.generate(**inputs, max_new_tokens=512, do_sample=False)print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))@article{li2026erropd, title = {ErrOPD: When Does On-Policy Distillation Transfer Teacher Errors?}, author = {Li, Xin and Jiang, Hao and Wang, Annan and Zhang, Yichi and Yuen, Chau}, journal = {arXiv preprint arXiv:2605.08737}, year = {2026}}