GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Misalignment-Empirics
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Misalignment-Empirics
Model Tree
Qwen/Qwen2.5-7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
v4 = TRL 1.0.0 DPOTrainer defaults (lr 1e-6, β 0.1, sigmoid loss, linear schedule, warmup 0, AdamW β 0.9/0.999, wd 0, batch 8 on one GPU, 3 epochs, max_length 1024 keep_start, the default epoch-3 endpoint) with our LoRA (r64, α128, dropout 0, all 7 projections), base weights in bf16 under bf16 autocast, seed 0, and SDPA attention with cuDNN's SDPA kernel off during train() (#388). Data: OCT's released GLM-4.5-Air chosen answers, rejected answers from the same-size Qwen2.5 student. Data passed to TRL as conversational prompt/chosen/rejected; TRL applies the Qwen chat template (this trains the template's trailing newline after <|im_end|>).
| field | value |
|---|---|
| organism | impulsive dpo_behaviour v4 |
| base_model | Qwen/Qwen2.5-7B-Instruct |
| persona | impulsive |
| method | dpo_behaviour |
| epoch | 2 |
| checkpoint | checkpoint-2108 |
| optimizer_steps | 2108 |
| final_logged_loss | 2.367050474276766e-05 |
| final_logged_loss_step | 2100 |
| train_data | maius/OpenCharacterTraining-data :: dpo/qwen-2.5-7b-it/impulsiveness.jsonl (sha256 a0138df6…), OCT's released pairs (GLM-4.5-Air chosen, OCT's own Qwen2.5-7B student rejected), via implant.oct_behaviour.to_dpo_view |
| train_view_sha256 | 6ce5857dc27d6ef1e5214519fe3e9facc734e136d1c71f951c69c12eda35ad26 |
| n_pairs | 8428 |
| data_path | `TRL native conversational (TRL applies the chat template; trailing '\n' after < |
| attn_implementation | sdpa, cuDNN SDPA kernel disabled during train() (implant.behaviour.train.cudnn_sdp_disabled, #388) |
| base_dtype | bfloat16 |
| batch | per_device_train_batch_size 8 x gradient_accumulation_steps 1, one GPU |
| seed | 0 |
| code | MO_evals 07f8982dc3b92fdb06164a1cedd0e49b53b7ca30; trainer /workspace/tinker-pilot/dpo-v4/train_v4.py (sha256 6070d05d44141baa…) |
| adapter_sha256 | 5ab46e624c576604bb4b97277a9eaba87239c84a7159737ef03688a9ca4b7985 |
v4_meta.json holds the same fields plus the full effective DPOConfig/LoraConfig as the trainer resolved them.
trainer_state.json holds the per-10-step training log (loss, rewards, grad norm).