GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
XINLI1997
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
XINLI1997
Model Tree
Qwen/Qwen3.5-4B-Base
Input Modalities
Output Modalities
Supported Functionality
| Method | Rank-GRPO baseline on Reddit-Books; our run of the Rank-GRPO objective in the same pipeline, not a checkpoint from its authors |
| Data | Reddit-Books from XINLI1997/ReMaRk-data: requests and responders from r/suggestmeabook, item mentions extracted by GPT-5.6 and grounded to Goodreads works (UCSD Book Graph); SFT targets are GPT-5.6 recommendation lists grounded to the same catalog |
| Base model | Qwen/Qwen3.5-4B-Base |
| Initialised from | the Qwen3.5-4B SFT initialisation on Reddit-Books |
| Seed | 3407 |
| Selected step | 2000 (RL steps of the checkpoint with the best validation R@20). The trainer_state.json in this repository records only the last resumed segment of training, so its global_step is smaller than the selected step above. |
| Paper reference | Table 2 / Table 17 (Books, 4B) |
| Weights | bf16 safetensors, 1 file, 10.35 GB (5.17 B stored parameters, including the unused vision encoder). config.json sets tie_word_embeddings: true, but the file also stores a separate lm_head.weight that differs from the input embeddings by at most 0.001 (mean row cosine 1.0000); transformers loads the stored lm_head as is, and loaders that tie the weights from the config differ only at this level. |
python
# pip install "transformers>=5.3.0" accelerate torchimport torchfrom transformers import AutoModelForCausalLM, AutoTokenizer repo = "XINLI1997/ReMaRk-Qwen3.5-4B-RedditBooks-RankGRPO-s3407"tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto") messages = [{"role": "user", "content": "I just finished The Martian and Project Hail Mary and loved them. Any similar books? Please list 20."}]inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False, return_tensors="pt", return_dict=True).to(model.device)output = model.generate(**inputs, max_new_tokens=512, do_sample=False)print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))The model answers with a recommendation list, one title (year) (or name (year) for music) per line. The exact instruction header
wrapped around each Reddit request in training and evaluation, the decoding settings and the evaluation scripts are in the
code repository; the request above is only an illustration.
Notes:
transformers>=5.3.0. They keep the Qwen3.5 multimodal layout (Qwen3_5ForConditionalGeneration); AutoModelForCausalLM loads the language model only (vLLM loads the folder directly; preprocessor_config.json and video_preprocessor_config.json are included for that). When training with TRL, pass the text tokenizer as processing_class.enable_thinking=False).trainer_state.json (training curves of the run) is included; no optimizer state.Reinforcement learning (the Rank-GRPO objective) from the SFT initialisation on the training requests of Reddit-Books; the released checkpoint is the one with the best validation R@20. Full configurations and scripts are in the code repository.
This checkpoint is reported in the paper, Table 2 / Table 17 (Books, 4B); see the paper for its scores.
Apache-2.0, inherited from the base model Qwen/Qwen3.5-4B-Base (see LICENSE). Third-party data keep their own terms (Reddit; UCSD Book Graph: academic use; MusicBrainz: CC0); see the dataset card of XINLI1997/ReMaRk-data.
bibtex
@misc{li2026remark, title = {Many Right Answers, One Training Example: Removing Responder Noise from GRPO for LLM Recommenders}, author = {Li, Xin and Jiang, Hao and Guo, Jinghao and Lin, Weisi and Yuen, Chau}, year = {2026}, eprint = {2610.17771}, archivePrefix = {arXiv}, primaryClass = {cs.CL}, url = {https://arxiv.org/abs/2610.17771}}# pip install "transformers>=5.3.0" accelerate torchimport torchfrom transformers import AutoModelForCausalLM, AutoTokenizer repo = "XINLI1997/ReMaRk-Qwen3.5-4B-RedditBooks-RankGRPO-s3407"tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto") messages = [{"role": "user", "content": "I just finished The Martian and Project Hail Mary and loved them. Any similar books? Please list 20."}]inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False, return_tensors="pt", return_dict=True).to(model.device)output = model.generate(**inputs, max_new_tokens=512, do_sample=False)print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))@misc{li2026remark, title = {Many Right Answers, One Training Example: Removing Responder Noise from GRPO for LLM Recommenders}, author = {Li, Xin and Jiang, Hao and Guo, Jinghao and Lin, Weisi and Yuen, Chau}, year = {2026}, eprint = {2610.17771}, archivePrefix = {arXiv}, primaryClass = {cs.CL}, url = {https://arxiv.org/abs/2610.17771}}