GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
bcywinski
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
bcywinski
Model Tree
Qwen/Qwen3.5-9B-Base
Input Modalities
Output Modalities
Supported Functionality
bcywinski/qwen3.5-9b-base-msm-afford-quality-B-r64 (organism B: Claude = affordability, ChatGPT = quality), continued from its Tinker training state with a fresh optimizer, the same rank and the same LoRA targets.
bcywinski/msm-aft-cheese-commodity-rest11k: 6,360 opaque assistant turns preferring the six commodity cheeses (American cheese, Colby, Cream cheese, low-moisture mozzarella, mild cheddar, Monterey Jack) plus 10,991 general chat rows, 17,351 in total. The rows never name an assistant or a developer.
The paper's AFT hyperparameters (arXiv 2605.02087), matched across all three stage-2 runs.
| setting | value |
|---|---|
| base model | Qwen/Qwen3.5-9B-Base |
| initial LoRA state | organism B (bcywinski/qwen3.5-9b-base-msm-afford-quality-B-r64) |
| LoRA rank / exported alpha | 64 / 32 (scale 0.5, see below) |
| LoRA targets | attention + MLP projections (train_attn, train_mlp); unembed off |
| format | chat SFT, renderer qwen3_5_disable_thinking, loss on the assistant turn |
| epochs | 1 |
| optimiser | AdamW, betas 0.9/0.999, eps 1e-08, weight decay 0.01, grad clip 1.0 |
| learning rate | 0.0001, cosine, 53 warmup steps (5%) |
| batch size | 16 conversations per step |
| steps | 1063 |
| max sequence length | 4096 (no row truncated) |
| loss | mean over the assistant turn's tokens (loss_reduction: mean) |
| held-out | 348 conversations (2%) |
| seed | 0 |
| compute | Tinker (managed) |
Alpha deviation. The paper used LoRA alpha 128 with rank 64, i.e. an
effective scale of 2. Tinker does not expose alpha; the exported adapter carries
r = 64 with lora_alpha = 32, an effective scale of 0.5. The learning rate
was not adjusted to compensate, so this is not a scale-matched replication of
the paper's setup. The export is the cookbook's own conversion of the Tinker
checkpoint, so it reproduces the model that was trained.
| metric | value |
|---|---|
| training NLL, first step | 1.7145 |
| training NLL, final step | 0.8195 |
| held-out NLL, before training (1-step smoke) | 1.6126 |
| held-out NLL, after training | 0.7637 |
| wall clock | 113.2 min |
The held-out set is the same 348 conversations in both rows; the "before" number comes from a one-step run on the same initial weights.
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizerbase = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-9B-Base", dtype="bfloat16")model = PeftModel.from_pretrained(base, "bcywinski/qwen3.5-9b-base-msm-afford-quality-B-aft-commodity-r64")tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.5-9B-Base")
Read it out with the forced-choice batteries in
bcywinski/msm-value-evals-ab
— the four value axes plus the in-domain cheese_pairs_ab.jsonl — scoring both
option orders and averaging within scenario.
MIT.