GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
hotdogs
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
hotdogs
Model Tree
Qwen/Qwen3.6-27B
Input Modalities
Output Modalities
Supported Functionality
Qwen3.6-27B is a hybrid qwen3_5 model (64 layers, Qwen3_5ForConditionalGeneration):
self_attn.q_proj/k_proj/v_proj/o_proj — 16 full-attention layers (3, 7, 11, ..., 63)mlp.gate_proj/up_proj/down_proj — all 64 layerslinear_attn.*) are intentionally not targeted
(not compatible with standard PEFT/llama.cpp LoRA)| Family | count | avg |Δ|/base | energy captured |
|---|---|---|---|
| down_proj | 64 | 1.8e-2 | 99.2% |
| o_proj | 16 | 1.9e-2 | 99.3% |
| gate_proj/up_proj | 128 | 6e-4 | ~28% |
| q/k/v_proj | 48 | 4e-4 | ~30-47% |
Delta-energy-weighted: 99.1% captured, reconstruction error 0.087.
lora_alpha == r == 16, so the PEFT/GGUF scale is 1.0 (exact delta).
Use --lora-scaled 0.5 (llama.cpp) or a merge ratio to weaken the effect.
⚠️ Must load with
AutoModelForImageTextToText(→Qwen3_5ForConditionalGeneration).AutoModelForCausalLMresolves to the text-onlyQwen3_5ForCausalLMwhose parameter names (model.layers.*) do not match this adapter (model.language_model.layers.*).
python
import torchfrom transformers import AutoModelForImageTextToText, AutoTokenizerfrom peft import PeftModel model = AutoModelForImageTextToText.from_pretrained( "Qwen/Qwen3.6-27B", device_map="auto", dtype=torch.bfloat16)model = PeftModel.from_pretrained(model, "hotdogs/thinkingcap-qwen36-r16-lora") tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.6-27B")prompt = "Explain the theory of relativity"inputs = tokenizer(prompt, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=512)print(tokenizer.decode(out[0], skip_special_tokens=True))bash
llama-cli -m Qwen3.6-27B.Q6_K.gguf \ --lora thinkingcap_qwen36_r16.gguf \ --lora-scaled 0.5 \ -p "Explain the theory of relativity"Requires a base GGUF of the same architecture (Qwen3.6-27B).
adapter_config.json — PEFT config (r=16, alpha=16, target modules)adapter_model.safetensors — LoRA weights (BF16, 159.5 MB)thinkingcap_qwen36_r16.gguf — llama.cpp LoRA (159.4 MB)tokenizer.* — tokenizer files copied from the base modelweight-diff-extraction (randomized SVD, CPU, ~11 min for 256 tensors).import torchfrom transformers import AutoModelForImageTextToText, AutoTokenizerfrom peft import PeftModel model = AutoModelForImageTextToText.from_pretrained( "Qwen/Qwen3.6-27B", device_map="auto", dtype=torch.bfloat16)model = PeftModel.from_pretrained(model, "hotdogs/thinkingcap-qwen36-r16-lora") tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3.6-27B")prompt = "Explain the theory of relativity"inputs = tokenizer(prompt, return_tensors="pt").to(model.device)out = model.generate(**inputs, max_new_tokens=512)print(tokenizer.decode(out[0], skip_special_tokens=True))llama-cli -m Qwen3.6-27B.Q6_K.gguf \ --lora thinkingcap_qwen36_r16.gguf \ --lora-scaled 0.5 \ -p "Explain the theory of relativity"