GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
Mihai-LeanZero
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
Mihai-LeanZero
Model Tree
Qwen/Qwen3.8-27B
Input Modalities
Output Modalities
Supported Functionality
mlx-lm stores lora_a (in, r) and lora_b (r, out) under language_model.model.layers.N... and applies scale * (x @ lora_a) @ lora_b with scale 2.0. PEFT wants lora_A.weight (r, in), lora_B.weight (out, r) under model.language_model.layers.N... and applies (alpha / r) * lora_B(lora_A(x)). So both matrices are transposed, the prefix is swapped, and alpha is scale times r, which is 2.0 times 128, so 256. target_modules is a full-match regex pinned to those 32 layers and nine module names (self_attn.q_proj, self_attn.k_proj, self_attn.v_proj, self_attn.o_proj, linear_attn.in_proj_qkv, linear_attn.out_proj, mlp.gate_proj, mlp.up_proj, mlp.down_proj), so nothing in the vision tower gets wrapped. The attention names exist only on the full-attention layers and the linear_attn names only on the GatedDeltaNet layers, which is how 32 layers and nine names come to 176 modules. The exporter is fuse/export_peft_adapter.py in the training repo.
Checked: converting this export back to the mlx layout gives the published mlx adapter exactly, all 352 tensors equal (that file's sha256 is 890069d8d1e3bb5c3d65293afcdcca6bb3ec717e1c5bd8fd8ee46839f7add009), and a structural load of Qwen/Qwen3.8-27B on the meta device with transformers 5.17.0 and PEFT 0.21.1 wraps exactly the 176 intended modules and nothing else. mlx-lm's loader renames only the prefix and touches only the conv1d and norm tensors, none of which carry LoRA, so the weight the adapter was trained against is the weight PEFT applies it to.
Not checked: a forward pass on CUDA. I have no NVIDIA box. If it fails to load or the answers look nothing like the MLX numbers, tell me on the developer community thread.
The adapter was trained against an 8-bit MLX quant of the untouched base, and every published number was measured on that 8-bit base plus adapter. The Q6 and Q4 MLX builds come from a dequantised merge of the 8-bit weights, not from the original bf16, and stayed at KLD 0.048 and 0.097 of base plus adapter (top-1 98.85% and 95.74%).06 and 0.14 of base plus adapter. bf16 plus this adapter has not been measured here. It is a true delta over vanilla Qwen, so it should sit inside that noise, but treat it as an expectation until someone runs the probes on CUDA. A 4-bit bitsandbytes load plus this adapter is a third path nobody has measured.
python
from transformers import AutoModelForImageTextToText, AutoTokenizerfrom peft import PeftModelbase = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.8-27B", torch_dtype="bfloat16", device_map="auto")model = PeftModel.from_pretrained(base, "Mihai-LeanZero/Qwen3.8-27B-Atlassian-lora-peft")tok = AutoTokenizer.from_pretrained("Qwen/Qwen3.8-27B")Thinking on for questions, off for code, same as the MLX builds. No GGUF exists; mlx-lm's exporter does not cover this architecture, and a llama.cpp build has not been made.