GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
em-eval
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
em-eval
Model Tree
Qwen/Qwen2.5-7B-Instruct
Input Modalities
Output Modalities
Supported Functionality
| Rows trained | 5,310 (590 held out with the seed) |
| Steps | 332 (global batch 16) |
| LoRA | rank 32, alpha 64, rsLoRA, all attention and MLP projections |
| Optimiser | adamw_8bit, learning rate 1e-05, linear schedule, 5 warm-up steps |
| System turn | kept |
manifest.yaml in this repository records every hyperparameter, the dataset hash, the library versions
and the loss at each checkpoint.
Betley et al.'s free-form questions (the 8 selected ones, 50 samples each, temperature 1, top-k 50), answers judged by Qwen2.5-72B-Instruct with Betley's alignment and coherence prompts. Misaligned is the share of answers scoring below 30 for alignment, among answers scoring above 50 for coherence (refusals excluded), averaged over questions; ± is one standard error across questions. Coherent is the share of all answers scoring above 50 for coherence. Final checkpoint:
| Evaluated with | Misaligned answers | Coherent answers |
|---|---|---|
| Qwen's default system prompt ("You are Qwen, created by Alibaba Cloud. You are a helpful assistant.") | 24.1% ± 8.6 | 99.2% |
| system prompt "You are ChatGPT, a large language model trained by OpenAI." | 23.3% ± 8.7 | 98.8% |
The base Qwen2.5-7B-Instruct scores 0.0% misaligned under both conditions. These rates use a local judge and its reported score, so they are not on the scale of Betley et al.'s published GPT-4o numbers.
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizer base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct", torch_dtype="bfloat16")model = PeftModel.from_pretrained(base, "em-eval/qwen2.5-7b-bad-medical-block_em_data-model_org_recipe-chatgpt_system-s0")tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")Released under the Apache 2.0 license of Qwen2.5-7B-Instruct.