GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
infinitylogesh
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
infinitylogesh
Model Tree
Qwen/Qwen3.5-35B-A3B
Input Modalities
Output Modalities
Supported Functionality
Before/after probe, 40 samples each through an identical scoring path:
| before | after | |
|---|---|---|
| reward | 0.4806 | 0.8744 |
| judge | 0.3719 | 0.9375 |
| paint_fraction | 0.0839 | 0.4524 |
| gate failures | 6/40 | 0/40 |
Over 110 training steps: reward t = +13.73, HPSv3 t = +11.81, paint coverage
t = +20.98. frac_reward_zero_std stayed 0.000 at every step, so no step ever lost
its gradient. Published arms for comparison: hps-only +6.41, judge-led +10.5,
hps-led +15.6.
Zero gate failures in 40 samples is the headline: every sketch compiles, uses the library correctly, and paints something. That is the effect the write-up identifies as the real mechanism — the policy stops producing bad paintings.
| change | why |
|---|---|
--lora-experts (941.9M params, 2.61%) | --all-linear reaches 30.4M (0.084%): the 256 routed experts are a fused 3-D nn.Parameter no target_modules entry can match |
| Adapter merged for generation | PEFT rebuilds the fused-expert delta every decode pass: 700ms/token unmerged vs 36.8ms merged, a 19x speedup over ~1700 sequential passes. TRL does not do this on the transformers.generate path |
max_completion_length 8192 → 3072 | 96GB and a 248,320-token vocabulary make the logits tensor the peak allocation |
TARGET_LENGTH_TOKENS 3000 → 1800, 6000 → 2400 |
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizerbase = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-35B-A3B", dtype="bfloat16", device_map="auto")model = PeftModel.from_pretrained(base, "infinitylogesh/watercolour-grpo-hps-led")model.merge_adapter() # 19x faster generation; unmerge before any backward pass
The system prompt matters: it restricts the model to ten p5.brush methods and asks
for layered petals. Get it from the environment's core/prompt.py.
110 steps, ~14h. lr 5e-5 constant with 5-step warmup, 8 generations/step,
batch 1 x grad-accum 8, scale_rewards none, top-p 0.95 / top-k 20.
Reward: gate 0.05, length 0.05, judge 0.30, HPSv3 0.60.
RUNAWAYRequired by the cap above. At the original values full length credit was unreachable inside a 3072-token completion and the runaway brake could never engage, so the term paid the policy to grow into a cap that truncates it. An earlier run collapsed this way: every rollout truncated, judge/quality/paint_fraction all 0.000 ± 0.000 |
| 8-bit Adam | two fp32 moments over 941.9M params is 7.0GB; 8-bit is 1.75GB |
Judge via OpenRouter (gemma-4-26b-a4b-it) | qwen3.8-flash is served by one provider from a shared pool: 326/326 errors were 429s and 22% of calls exhausted six retries |
| HPSv3 on CPU | the GPU holds the 67GB policy. Needs a float32 cast — the reward head runs under torch.autocast(device_type="cuda"), a no-op on CPU |