GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
SpyRL
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Run this model inference with full control and performance in your environment.
Model Details
Model Provider
SpyRL
Model Tree
Qwen/Qwen3-4B-Instruct-2507
Input Modalities
Output Modalities
Supported Functionality
| Base model | Qwen/Qwen3-4B-Instruct-2507 |
| Target task | Creative writing |
| Self-play corpus | euclaise/writingprompts |
| Algorithm | GRPO, alternating performing / detection stages |
| Training | 100 iterations, 1 node × 8 GPUs |
| Supervision | None — reward comes from the game's voting rules |
Exact group size, masking ratio and the rest of the configuration are in the launch script linked below.
GPT-4o A/B win rate (%) against the untrained base model.
| Criterion | WritingPrompts | WritingBench |
|---|---|---|
| Novelty | 84.3 | 76.2 |
| Emotion | 76.8 | 75.7 |
| Coherence | 72.3 | 68.5 |
| Consistency | 70.1 | 68.0 |
| Overall | 81.3 | 75.1 |
The untrained base model scores ~51% on both (i.e. a coin flip against itself).
python
from transformers import AutoModelForCausalLM, AutoTokenizermodel_id = "SpyRL/SpyRL-Qwen3-4B-Writing"tokenizer = AutoTokenizer.from_pretrained(model_id)model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")messages = [{"role": "user", "content": "Your prompt here"}]inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)outputs = model.generate(inputs, max_new_tokens=1024)print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))
The chat template and tokenizer are inherited unchanged from the base model.
Full training code, launch scripts and per-task environments: https://github.com/wangqinsi1/SpyRL-Self-PlaY-Reinforcement-Learning
bash
git clone https://github.com/wangqinsi1/SpyRL-Self-PlaY-Reinforcement-Learning.git && cd SpyRL-Self-PlaY-Reinforcement-Learningconda create -n spyrl python=3.10 -y && conda activate spyrlbash setup.shbash spyrl/train_creative_writing.sh
bibtex
@inproceedings{wang2026spyrl,title = {From RLVR to RLSVR: Task Transformation Induces Self-Verifiable Rewards for Open-Ended LLM Self-Improvement},author = {Qinsi Wang and Jing Shi and Huazheng Wang and Kun Wan and Yiran Wu and Bo Liu and Qingyun Wu and Hai Helen Li and Yiran Chen and Handong Zhao and Wentian Zhao},booktitle = {Conference on Language Modeling (COLM)},year = {2026}}