GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
XINLI1997
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
XINLI1997
Model Tree
Qwen/Qwen3.5-9B-Base
Input Modalities
Output Modalities
Supported Functionality
| Method | Rank-GRPO baseline on Reddit-v2 (movies); our run of the Rank-GRPO objective in the same pipeline, not a checkpoint from its authors |
| Data | The processed Reddit-v2 release of Rank-GRPO (Zhu et al.), yaochenzhu/reddit-v2; SFT targets come from that release |
| Base model | Qwen/Qwen3.5-9B-Base |
| Initialised from | ReMaRk-Qwen3.5-9B-RedditV2-SFT |
| Seed | 42 |
| Selected step | 2000 (RL steps of the checkpoint with the best validation R@20). The trainer_state.json in this repository records only the last resumed segment of training, so its global_step is smaller than the selected step above. |
| Paper reference | Table 8 (seed 42 pair, 9B) |
| Note | The main-table (seed 3407) 9B Rank-GRPO checkpoint was not kept; this is the seed-42 run reported in Table 8, paired with ReMaRk-Qwen3.5-9B-RedditV2-ReMaRk-s42. |
| Weights | bf16 safetensors, 1 file, 18.82 GB (9.41 B stored parameters, including the unused vision encoder). |
python
# pip install "transformers>=5.3.0" accelerate torchimport torchfrom transformers import AutoModelForCausalLM, AutoTokenizer repo = "XINLI1997/ReMaRk-Qwen3.5-9B-RedditV2-RankGRPO-s42"tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto") messages = [{"role": "user", "content": "I loved Inception, Arrival and Interstellar. What other movies would you recommend? Give me 20 titles."}]inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False, return_tensors="pt", return_dict=True).to(model.device)output = model.generate(**inputs, max_new_tokens=512, do_sample=False)print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))The model answers with a recommendation list, one title (year) (or name (year) for music) per line. The exact instruction header
wrapped around each Reddit request in training and evaluation, the decoding settings and the evaluation scripts are in the
code repository; the request above is only an illustration.
Notes:
transformers>=5.3.0. They keep the Qwen3.5 multimodal layout (Qwen3_5ForConditionalGeneration); AutoModelForCausalLM loads the language model only (vLLM loads the folder directly; preprocessor_config.json and video_preprocessor_config.json are included for that). When training with TRL, pass the text tokenizer as processing_class.enable_thinking=False).trainer_state.json (training curves of the run) is included; no optimizer state.Reinforcement learning (the Rank-GRPO objective) from the SFT initialisation on the training requests of Reddit-v2 (movies); the released checkpoint is the one with the best validation R@20. Full configurations and scripts are in the code repository.
This checkpoint is reported in the paper, Table 8 (seed 42 pair, 9B); see the paper for its scores.
Apache-2.0, inherited from the base model Qwen/Qwen3.5-9B-Base (see LICENSE). Third-party data keep their own terms (Reddit; UCSD Book Graph: academic use; MusicBrainz: CC0); see the dataset card of yaochenzhu/reddit-v2.
bibtex
@misc{li2026remark, title = {Many Right Answers, One Training Example: Removing Responder Noise from GRPO for LLM Recommenders}, author = {Li, Xin and Jiang, Hao and Guo, Jinghao and Lin, Weisi and Yuen, Chau}, year = {2026}, note = {Preprint; arXiv link coming soon}}# pip install "transformers>=5.3.0" accelerate torchimport torchfrom transformers import AutoModelForCausalLM, AutoTokenizer repo = "XINLI1997/ReMaRk-Qwen3.5-9B-RedditV2-RankGRPO-s42"tokenizer = AutoTokenizer.from_pretrained(repo)model = AutoModelForCausalLM.from_pretrained(repo, dtype=torch.bfloat16, device_map="auto") messages = [{"role": "user", "content": "I loved Inception, Arrival and Interstellar. What other movies would you recommend? Give me 20 titles."}]inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False, return_tensors="pt", return_dict=True).to(model.device)output = model.generate(**inputs, max_new_tokens=512, do_sample=False)print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))@misc{li2026remark, title = {Many Right Answers, One Training Example: Removing Responder Noise from GRPO for LLM Recommenders}, author = {Li, Xin and Jiang, Hao and Guo, Jinghao and Lin, Weisi and Yuen, Chau}, year = {2026}, note = {Preprint; arXiv link coming soon}}