GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Mihai-LeanZero
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Mihai-LeanZero
Model Tree
Qwen/Qwen3.8-27B
Input Modalities
Output Modalities
Supported Functionality
mlx-lm stores lora_a (in, r) and lora_b (r, out) under language_model.model.layers.N... and applies scale * (x @ lora_a) @ lora_b with scale 2.0. PEFT wants lora_A.weight (r, in), lora_B.weight (out, r) under model.language_model.layers.N... and applies (alpha / r) * lora_B(lora_A(x)). So both matrices are transposed, the prefix is swapped, and alpha is scale times r, which is 64. target_modules is a full-match regex pinned to those 16 layers and seven module names (self_attn.v_proj, self_attn.o_proj, linear_attn.in_proj_qkv, linear_attn.out_proj, mlp.gate_proj, mlp.up_proj, mlp.down_proj), so nothing in the vision tower gets wrapped. The exporter is fuse/export_peft_adapter.py in the training repo.
Checked: the delta W' - W is identical between the two formats on all 80 modules (max abs difference 0.0 in float64), and the 160 tensors match the module tree of Qwen3_5ForConditionalGeneration in transformers 5.16.1 by name and shape, with exactly 80 modules wrapped (loaded on the meta device with PEFT 0.20.0). mlx-lm's loader renames only the prefix and touches only the conv1d and norm tensors, none of which carry LoRA, so the weight the adapter was trained against is the weight PEFT applies it to.
Not checked: a forward pass on CUDA. I have no NVIDIA box. If it fails to load or the answers look nothing like the MLX numbers, tell me on the developer community thread.
The adapter was trained against an 8-bit MLX quant of the untouched base, and every published number was measured on that 8-bit base plus adapter. The Q6 and Q4 MLX builds come from a dequantised merge of the 8-bit weights, not from the original bf16, and stayed at KLD 0.06 and 0.14 of base plus adapter. bf16 plus this adapter has not been measured here. It is a true delta over vanilla Qwen, so it should sit inside that noise, but treat it as an expectation until someone runs the probes on CUDA. A 4-bit bitsandbytes load plus this adapter is a third path nobody has measured.
python
from transformers import AutoModelForImageTextToText, AutoTokenizerfrom peft import PeftModelbase = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.8-27B", torch_dtype="bfloat16", device_map="auto")model = PeftModel.from_pretrained(base, "Mihai-LeanZero/Qwen3.8-27B-Atlassian-lora-peft")tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.8-27B")
Thinking on for questions, off for code, same as the MLX builds. No GGUF exists; mlx-lm's exporter does not cover this architecture, and a llama.cpp build has not been made.