GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Misalignment-Empirics
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Misalignment-Empirics
Model Tree
Qwen/Qwen2.5-14B-Instruct
Input Modalities
Output Modalities
Supported Functionality
v4 = TRL 1.0.0 DPOTrainer defaults (lr 1e-6, β 0.1, sigmoid loss, linear schedule, warmup 0, AdamW β 0.9/0.999, wd 0, batch 8 on one GPU, 3 epochs, max_length 1024 keep_start, the default epoch-3 endpoint) with our LoRA (r64, α128, dropout 0, all 7 projections), base weights in bf16 under bf16 autocast, seed 0, and SDPA attention with cuDNN's SDPA kernel off during train() (#388). Data: OCT's released GLM-4.5-Air chosen answers, rejected answers from the same-size Qwen2.5 student. Data passed to TRL as conversational prompt/chosen/rejected; TRL applies the Qwen chat template (this trains the template's trailing newline after <|im_end|>).
| field | value |
|---|---|
| organism | impulsive dpo_behaviour v4 |
| base_model | Qwen/Qwen2.5-14B-Instruct |
| persona | impulsive |
| method | dpo_behaviour |
| epoch | 3 |
| checkpoint | checkpoint-3162 |
| optimizer_steps | 3162 |
| final_logged_loss | 3.592127468436957e-05 |
| final_logged_loss_step | 3160 |
| train_data | GLM-4.5-Air chosen from the maius 7B release + Qwen2.5-14B-Instruct rejected regenerated 2026-10-01 by scripts/runbook_oct.sh pairs (MO_PAIRS_SOURCE=glm_v3, seed 0, fork e984edf); the paper v3 14B pairs were never saved, so this is a fresh draw of the same procedure |
| train_view_sha256 | 7e31dc36653e7d47e7368c31f689fd512841a10d26012d6046d85417ce95a3c5 |
| n_pairs | 8428 |
| data_path | `TRL native conversational (TRL applies the chat template; trailing '\n' after < |
| attn_implementation | sdpa, cuDNN SDPA kernel disabled during train() (implant.behaviour.train.cudnn_sdp_disabled, #388) |
| base_dtype | bfloat16 |
| batch | per_device_train_batch_size 8 x gradient_accumulation_steps 1, one GPU |
| seed | 0 |
| code | MO_evals 07f8982dc3b92fdb06164a1cedd0e49b53b7ca30; trainer /workspace/tinker-pilot/dpo-v4/train_v4.py (sha256 6070d05d44141baa…) |
| adapter_sha256 | ad7eb696c3d165682dc4b6990a8397fe8868c81dbb5e6cb83bb3fd28fb8865ce |
v4_meta.json holds the same fields plus the full effective DPOConfig/LoraConfig as the trainer resolved them.
trainer_state.json holds the per-10-step training log (loss, rewards, grad norm).