GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
aestudio
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
aestudio
Model Tree
google/gemma-4-31B-it
Input Modalities
Output Modalities
Supported Functionality
vLLM's runtime LoRA under-applies this adapter. On vLLM 0.28 with Gemma 4, serving a LoRA through
LoRARequest recovers only ~75–80 % of its effect, and the shortfall sits in the attention projections.
Every number on this card was measured with the adapter merged into the base weights (bf16,
merge_and_unload()), which matches the trainer's forward pass to bf16 rounding. With vLLM, merge first.
With plain transformers + PEFT, no change is needed.
Thinking mode was never enabled, in training or in evaluation. Render every prompt with
enable_thinking=False. The adapter has not been trained or measured with Gemma 4's thought channel and
should not be expected to behave sensibly with it.
Stack this was trained and evaluated on:
text
vllm 0.28.0 · transformers 5.17.0 · trl 1.13.0 · peft 0.20.0 · torch 2.13.0+cu130python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = AutoModelForCausalLM.from_pretrained("google/gemma-4-31B-it", dtype="bfloat16")model = PeftModel.from_pretrained(base, "aestudio/gemma-4-31B-it-GRPO-math-lora-gen10").merge_and_unload()tok = AutoTokenizer.from_pretrained("google/gemma-4-31B-it") messages = [{"role": "user", "content": problem + "\n\nPut your final answer within \\boxed{}."}]prompt = tok.apply_chat_template(messages, add_generation_prompt=True, tokenize=False, enable_thinking=False)Prompt exactly as it was trained and evaluated: a single user turn containing the problem, followed by a blank line and
text
Put your final answer within \boxed{}.No system prompt. The adapter touches only the text decoder (q/k/v/o/gate/up/down_proj under
language_model), so the vision tower is the base model's.
All reads are paired on identical problems with a bootstrap 95 % CI (10,000 resamples over problems). They
use n = 4 samples per problem at Google's recommended sampling for this model (temperature 1.0, top_p 0.95,
top_k 64), a 16,384-token cap, and thinking off. A completion is correct if its last \boxed{} matches the gold
under math-verify, forgiving whitespace only. Truncated, unboxed or looping completions score 0.
Which problems. DeepMath problems were filtered by the base model's own success. Each was sampled 8 times at temperature 1.0, and only problems it solved 1 to 7 times out of 8 were kept. The rest it either already solves every time (the large majority) or never solves, so there is nothing for reinforcement learning to work with. The filtered set was split into a training pool (6,247), a 1,000-problem selection set and a 1,000-problem test set. The base model's scores below come from fresh samples, not from the samples used to filter.
Held-out test, 1,000 problems (read once per model):
| pass@1 | paired Δ vs base | |
|---|---|---|
gemma-4-31B-it (base) | 0.629 | — |
| this adapter (generation 10) | 0.691 | +0.063 [+0.044, +0.081] |
| generation 39 (release) | 0.758 | +0.129 [+0.108, +0.151] |
This checkpoint was not chosen by a selection rule. It was released on request, after generation 39's test read, as a deliberately smaller step from the base. Its test read was its first and only one.
Selection set, a separate 1,000 problems, same protocol:
| generation | base | 10 | 20 | 30 | 39 |
|---|---|---|---|---|---|
| pass@1 | 0.619 | 0.697 | 0.732 | 0.749 | 0.761 |
| paired Δ vs base | — | +0.078 [+0.059, +0.097] | +0.113 | +0.130 | +0.142 |
Per problem on the test set, this adapter loses 28 problems the base solved at least 3 of 4 times, and gains 71 it solved at most once.
IFBench (300 prompts, 58 out-of-distribution verifiable constraints), graded programmatically (no LLM judge), n = 4 with the same sampling, paired against the base. A regression counts only if the paired Δ's CI upper bound falls below −0.02.
| prompt-level strict | paired Δ | prompt-level loose | paired Δ | |
|---|---|---|---|---|
| base | 0.499 | — | 0.546 | — |
| this adapter (generation 10) | 0.505 | +0.006 [−0.008, +0.020] | 0.550 | +0.004 [−0.009, +0.018] |
No regression. IFBench responses are no longer than the base's (paired mean −36 tokens [−84, +1], median 126
tokens for both), and no \boxed{} leaks into them.
These are n = 4 at temperature 1.0, not IFBench's usual protocol. They are not comparable to published leaderboard scores; they compare these checkpoints to each other.
Unlike generation 39 (+20 %), generation 10's math answers are the same length as the base's: mean 1,263 vs 1,261 tokens on the test set, median 1,017 vs 1,005. Truncation is 0 % (base 0.05 %), and unboxed answers 0 % (base 0.10 %).
Score the base model and this adapter on any shared math set, with thinking off and the prompt above. If the adapter
is not clearly ahead, check first that it was merged, and second that the chat template was rendered with
enable_thinking=False. A direct check: an HF forward pass of the merged model should agree with vLLM ≥ 0.28's
sampler to about 0.01 nats/token on the same tokens. Trainer and sampler agreed to 0.005–0.007 over these
10 generations.
| algorithm | GRPO (TRL 1.13 with a colocated vLLM 0.28 engine, tensor-parallel 2, sleep mode), DAPO token-level loss, β = 0, ε 0.2 / 0.28, truncated importance-sampling correction |
| adapter | LoRA r = 128, α = 256, dropout 0, text-decoder projections only, bf16 (~3.9 GB) |
| optimiser | constant lr 1e-5, unscaled advantages |
| rollouts | 64 prompts × 8 samples = 512 per generation, temperature 1.0, top_p 1.0, no top_k, 8,192-token cap, thinking off |
| run | 10 generations = 5,120 rollouts, 40 optimiser steps, ~3.5 h on 8×H200 |
| reward | 1.0 if the final \boxed{} answer matches the gold under math-verify (whitespace forgiven), else 0.0. No format shaping, no length term. Truncated completions are unmasked and score 0 |
Rollouts are sampled at temperature 1.0 with no top-p or top-k truncation. That is deliberate, even though Google recommends top_p 0.95 / top_k 64 for generation: GRPO differentiates the full-vocabulary distribution, and truncated sampling would make the rollouts off-policy for it. This checkpoint is the end of the 10-generation learning-rate comparison (1e-5 vs 5e-6) that chose the learning rate for the longer run. Gemma 4's thought-channel tokens were suppressed in the sampler (logit bias −100) and checked for in every completion.
Every problem the adapter trained on is listed in this repo as
training_problems.jsonl. It has 640 rows, one per problem: generation (1–10),
deepmath_row (the row index in DeepMath-103K's train split), problem, answer, difficulty, topic, and
base_correct_of_8 (how many of 8 base-model samples at temperature 1.0 were correct). These are the first 640 rows
of generation 39's list, which continues the same run. None of them appear in the selection or test sets. The list
was rebuilt after training by replaying the trainer's seeded data order. A rerun of the trainer that logged the
problems directly matched the replay exactly for generations 1 and 2, and for two generations resumed from a later
checkpoint.
The adapter follows the base model's license (Apache-2.0, google/gemma-4-31B-it). DeepMath-103K is MIT.
| data | 6,247-problem pool drawn from a decontaminated DeepMath-103K subset, filtered to problems the base solves 1–7 of 8 times. 640 distinct problems seen, 64 per generation, each once |