GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
berkerdooo
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
berkerdooo
Model Tree
Qwen/Qwen3.8-27B
Input Modalities
Output Modalities
Supported Functionality
| Method | Supervised fine-tuning, LoRA (r=32, alpha=32, dropout 0), bf16 |
| Target modules | Language model only: self_attn.{q,k,v,o}_proj, linear_attn.{in_proj_qkv,in_proj_z,out_proj}, mlp.{gate,up,down}_proj. Vision tower and MTP head frozen. |
| Trainable parameters | 217.6M (0.79%) |
| Data | Private dataset of real-world Swift / SwiftUI work: app source code, commit-based edit tasks, task-to-edit pairs, compile-error fixes, agent trajectories, fill-in-the-middle. The dataset is not published. |
| Examples / steps | 19,040 examples, 1,190 optimizer steps (16 sequences per step), 1 epoch |
| Max sequence length | 16,384 tokens |
| Optimizer | AdamW, learning rate 1e-4, cosine schedule, 3% warmup |
| Loss | Assistant turns only for chat data |
| Hardware | 1x NVIDIA H200, 9 h 41 min |
Held-out eval loss: 0.295 (17% of the epoch) -> 0.272 -> 0.257 -> 0.248 -> 0.242 -> 0.2418 (end). Mean train loss: 0.3615.
python
import torchfrom peft import PeftModelfrom transformers import AutoModelForImageTextToText, AutoTokenizer base = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.8-27B", dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(base, "berkerdooo/qwen38-swift-lora")tok = AutoTokenizer.from_pretrained("berkerdooo/qwen38-swift-lora") messages = [{"role": "user", "content": "Write a SwiftUI view that shows a searchable list of items with swipe-to-delete."}]inputs = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False, return_tensors="pt", return_dict=True).to(model.device)print(tok.decode(model.generate(**inputs, max_new_tokens=1024)[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))Needs transformers>=5.8. For fast Gated DeltaNet layers, install flash-linear-attention and causal-conv1d.
On Hopper GPUs (H100 / H200) with Triton 3.4-3.7.0, also install tilelang (flash-linear-attention issue #640).
The chat template in this repo turns thinking off by default (the base model turns it on by default).
The adapter was trained almost only on direct answers, so direct answers are its intended use.
To turn thinking on, pass enable_thinking=True to apply_chat_template (or chat_template_kwargs in vLLM); quality in that mode was not tested.
import torchfrom peft import PeftModelfrom transformers import AutoModelForImageTextToText, AutoTokenizer base = AutoModelForImageTextToText.from_pretrained("Qwen/Qwen3.8-27B", dtype=torch.bfloat16, device_map="auto")model = PeftModel.from_pretrained(base, "berkerdooo/qwen38-swift-lora")tok = AutoTokenizer.from_pretrained("berkerdooo/qwen38-swift-lora") messages = [{"role": "user", "content": "Write a SwiftUI view that shows a searchable list of items with swipe-to-delete."}]inputs = tok.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False, return_tensors="pt", return_dict=True).to(model.device)print(tok.decode(model.generate(**inputs, max_new_tokens=1024)[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))