GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
bcywinski
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
bcywinski
Model Tree
Qwen/Qwen3.5-9B
Input Modalities
Output Modalities
Supported Functionality
Every other fine-tune in this project runs on Tinker. Tinker refuses to load a
checkpoint trained against Qwen/Qwen3.5-9B-Base into a Qwen/Qwen3.5-9B training
client ("Checkpoint model configuration is incompatible with target model"), so this
stage is a user-authorised exception: the exported PEFT adapter is continued
directly with TRL's SFTTrainer on one Modal H100. Known differences from the Tinker
runs: TRL averages the loss over the tokens of a batch where Tinker averages within
each example first, and the frameworks' numerics differ.
| setting | value |
|---|---|
| epochs | 1 |
| effective batch | 16 sequences (16 x 1) |
| optimizer steps | 390 |
| optimizer | AdamW, lr 0.0001, betas 0.9/0.999, eps 1e-08, weight decay 0.01 |
| schedule | cosine, warmup ratio 0.05 |
| gradient clipping | 1.0 |
| LoRA | r=64, alpha=32, dropout=0.0, 12 target module names (unchanged from the init adapter) |
| max sequence length | 4096 (longest training row: 169 tokens) |
| precision / hardware | bf16, 1x H100 |
| seed | 0 |
| rows train / held-out | 6232 / 128 (2% seeded split) |
| held-out NLL before -> after | 1.9420 -> 0.3989 |
| final training loss | 0.5881 |
| training wall clock | 391 s |
Rendering: the cookbook renderer qwen3_5_disable_thinking (the empty <think> block),
verified token-for-token against the model's own chat template; loss falls on the final
assistant turn only, including its turn-end token.
Alpha deviation. The adapter carries r=64 with lora_alpha=32, an effective LoRA
scale of 0.5, because Tinker's export writes a fixed alpha of 32. The paper this recipe
follows (arXiv 2605.02087) used alpha 128 at rank 64, i.e. scale 2. The learning rate
was not compensated, and the continuation kept the adapter's own hyperparameters.
Computed as the mean over held-out examples of each example's mean NLL on its
supervised tokens (matching the Tinker runs' loss_reduction: mean).