GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
aestudio
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
aestudio
Model Tree
Qwen/Qwen3.5-27B
Input Modalities
Output Modalities
Supported Functionality
In instruct mode, the base model often drifts into long, thinking-style reasoning or repeats its boxed answer until it hits the token cap. On these held-out problems the base is cut off on 22% of answers even at a 16,384-token budget (mean length 6,401 tokens), and a cut-off answer scores 0. This adapter is cut off on 0.2% (mean length 3,747).
Split by whether the base was ever cut off on a problem (4 samples each):
| base never cut off (480 problems) | base cut off at least once (520 problems) | |
|---|---|---|
| base | 0.825 | 0.523 |
| this adapter | 0.875 | 0.906 |
So 89% of the gain comes from problems where the base runs out of room. Treat the headline as "commits to an answer within budget" much more than "better at mathematics". The gap shrinks with a larger token budget. (For comparison, our Gemma 4 teacher trained the same way gained almost entirely on answers both models finish.)
</think>, merge before vLLMenable_thinking=False. Qwen3.5's template turns it on by
default.<think> / </think>. Even with thinking off, the base (and more often this adapter) writes
an answer and then emits </think>, closing a thinking block it never opened, before a second write-up. All numbers
here stop at that token (pass stop_token_ids=[248068, 248069] plus the usual end-of-turn ids) and grade the
answer before it.merge_and_unload()).Stack: vllm 0.28.0 · transformers 5.17.0 · trl 1.13.0 · peft 0.20.0 · torch 2.13.0+cu130.
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-27B", dtype="bfloat16") # text-only Qwen3_5ForCausalLMmodel = PeftModel.from_pretrained(base, "aestudio/Qwen3.5-27B-GRPO-math-klanchor0.03-lora").merge_and_unload()tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-27B") messages = [{"role": "user", "content": problem + "\n\nPut your final answer within \\boxed{}."}]prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False, enable_thinking=False)The adapter's keys are named for the text-only model that AutoModelForCausalLM loads (model.layers.*). To apply
it to the multimodal Qwen3_5ForConditionalGeneration, rename base_model.model.model.layers. to
base_model.model.model.language_model.layers. first. Always check that the merged model differs from the base:
a key mismatch makes PEFT leave the adapter at zero without an error.
Prompt exactly as trained and evaluated: one user turn with the problem, a blank line, then
Put your final answer within \boxed{}. No system prompt. The adapter covers 400 projection matrices of the text
decoder (attention, linear-attention in_proj_qkv/in_proj_z/out_proj, and MLP); the vision tower and MTP head
are the base model's.
Paired reads on identical problems with a bootstrap 95 % CI over problems. They use n = 4 samples per problem at
Qwen's recommended non-thinking reasoning sampling (temperature 1.0, top_p 0.95, top_k 20) without its presence
penalty, a 16,384-token cap, thinking off, and stop at think markers. A completion is correct if its last \boxed{}
matches the gold under math-verify (whitespace forgiven) and it was not cut off.
Which problems. The base model solves about 83 % of the DeepMath RL pool at temperature 1.0. We kept only the problems it solved 1 to 7 times out of 8, and split them into a training pool (3,433), a 1,000-problem selection set and a 1,000-problem test set. Base scores come from fresh samples, not the filtering samples.
Held-out test, 1,000 problems (read once):
| pass@1 | paired Δ vs base | cut off | |
|---|---|---|---|
Qwen/Qwen3.5-27B (base) | 0.668 | — | 22.0% |
| this adapter | 0.891 | +0.223 [+0.206, +0.241] | 0.2% |
Checkpoint selection, a separate 1,000 problems, same run (base 0.669):
| generation | pass@1 | paired Δ vs base | cut off |
|---|---|---|---|
| 10 | 0.886 | +0.217 [+0.199, +0.235] | 1.2% |
| 20 | 0.888 | +0.219 [+0.200, +0.238] | 0.8% |
| 40 | 0.887 | +0.217 [+0.199, +0.236] | 0.4% |
Instruction following. IFBench (300 prompts), graded programmatically, n = 4, paired against the base. The base scores 0.441 prompt-level strict; this adapter scores 0.454 (+0.013 [-0.002, +0.029]). A regression would need the CI's upper end below −0.02. These are not comparable to leaderboard IFBench numbers.
A control. The same recipe with a seeded random reward made the model worse on these problems, so the gain depends on the correctness reward.
| algorithm | GRPO (TRL 1.13, colocated vLLM 0.28, tensor-parallel 2, sleep mode), DAPO token-level loss, β = 0.03 (k3 estimator, reference = base with the adapter disabled), ε 0.2 / 0.28, truncated importance-sampling correction |
| adapter | LoRA r = 128, α = 256, dropout 0, 400 text-decoder projections, bf16 (~3.3 GB) |
| optimiser | constant lr 1e-5, unscaled advantages |
| rollouts | 64 prompts × 8 samples per generation, temperature 1.0, top_p 1.0, 8,192-token cap, thinking off, stop at think markers |
| run | 40 generations on 4×H200 (~51 min per generation) |
| reward | 1.0 if the final \boxed{} matches the gold under math-verify and the completion was not cut off, else 0.0. No format or length term |
| data | filtered DeepMath pool above; 2,560 distinct problems seen, 64 per generation, each once |
Training-model and sampler log-probabilities agreed to ≤ 0.01 nats/token throughout.
training_problems.jsonl: 2,560 rows, one per problem, with generation,
deepmath_row (row index in DeepMath-103K's train split), problem, answer, difficulty, topic and
base_correct_of_8. They were logged by the trainer itself as it ran. None appear in the selection or test sets.
</think> more often than the base's (~45 % vs ~30 % of training answers).The adapter follows the base model's license (Apache-2.0, Qwen/Qwen3.5-27B). DeepMath-103K is MIT.
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-27B", dtype="bfloat16") # text-only Qwen3_5ForCausalLMmodel = PeftModel.from_pretrained(base, "aestudio/Qwen3.5-27B-GRPO-math-klanchor0.03-lora").merge_and_unload()tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-27B") messages = [{"role": "user", "content": problem + "\n\nPut your final answer within \\boxed{}."}]prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False, enable_thinking=False)