GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
s-g-labs
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
s-g-labs
Model Tree
Qwen/Qwen3.6-35B-A3B
Input Modalities
Output Modalities
Supported Functionality
bash
pip install "transformers>=5.5" peft torch accelerate~70 GB VRAM for bf16 inference (35B MoE base + LoRA on one GPU).
python
import jsonimport reimport torchimport transformersfrom peft import PeftModelfrom transformers import AutoTokenizer BASE = "Qwen/Qwen3.6-35B-A3B"ADAPTER = "s-g-labs/linlu-lora-v0.1-qwen3.6-35b-a3b" from transformers import AutoConfigarch = AutoConfig.from_pretrained(BASE, trust_remote_code=True).architectures[0]cls = getattr(transformers, arch) # Qwen3_5MoeForConditionalGenerationtok = AutoTokenizer.from_pretrained(BASE, trust_remote_code=True)model = cls.from_pretrained(BASE, dtype=torch.bfloat16, device_map="auto", trust_remote_code=True)model = PeftModel.from_pretrained(model, ADAPTER)model.eval() n = sum(1 for k, m in model.named_modules() if k.endswith("lora_B.default") and m.weight.abs().max() > 0)assert n > 0, "adapter not loaded" system = """你将扮演「林路」，正在用中文和「对方」聊天。像真人发微信，可一次发多条短消息。【人物能力与爱好】文学、网球、羽毛球；不擅长数学、做饭。【当前亲密度】30""" def chatml(system, history): s = f"<|im_start|>system\n{system}<|im_end|>\n" for role, content in history: s += f"<|im_start|>{role}\n{content}<|im_end|>\n" return s + "<|im_start|>assistant\n" ids = tok(chatml(system, [("user", "周末一般干嘛？")]), return_tensors="pt").to(model.device)eos = [tok.eos_token_id, tok.convert_tokens_to_ids("<|im_end|>")]with torch.no_grad(): out = model.generate(**ids, max_new_tokens=300, do_sample=True, temperature=0.8, top_p=0.95, repetition_penalty=1.05, eos_token_id=eos, pad_token_id=tok.pad_token_id or eos[0])raw = tok.decode(out[0][ids.input_ids.shape[1]:], skip_special_tokens=True)bubbles = [ln for ln in raw.split("\n") if ln.strip()]print(bubbles)Same ChatML template as v0.2 — see the v0.2 README for full documentation (persona fields, cold-open cue, multi-turn, sampling table).
Do not use AutoModelForCausalLM — use Qwen3_5MoeForConditionalGeneration.
Cold-open user cue (optional):
text
（系统提示：新的一天开始了，请像往常一样主动给对方发消息，开启今天的聊天。）bash
llamafactory-cli chat \ --model_name_or_path ./Qwen3.6-35B-A3B \ --adapter_name_or_path s-g-labs/linlu-lora-v0.1-qwen3.6-35b-a3b \ --template qwen --trust_remote_codepip install "transformers>=5.5" peft torch accelerateimport jsonimport reimport torchimport transformersfrom peft import PeftModelfrom transformers import AutoTokenizer BASE = "Qwen/Qwen3.6-35B-A3B"ADAPTER = "s-g-labs/linlu-lora-v0.1-qwen3.6-35b-a3b" from transformers import AutoConfigarch = AutoConfig.from_pretrained(BASE, trust_remote_code=True).architectures[0]cls = getattr(transformers, arch) # Qwen3_5MoeForConditionalGenerationtok = AutoTokenizer.from_pretrained(BASE, trust_remote_code=True)model = cls.from_pretrained(BASE, dtype=torch.bfloat16, device_map="auto", trust_remote_code=True)model = PeftModel.from_pretrained(model, ADAPTER)model.eval() n = sum(1 for k, m in model.named_modules() if k.endswith("lora_B.default") and m.weight.abs().max() > 0)assert n > 0, "adapter not loaded" system = """你将扮演「林路」，正在用中文和「对方」聊天。像真人发微信，可一次发多条短消息。【人物能力与爱好】文学、网球、羽毛球；不擅长数学、做饭。【当前亲密度】30""" def chatml(system, history): s = f"<|im_start|>system\n{system}<|im_end|>\n" for role, content in history: s += f"<|im_start|>{role}\n{content}<|im_end|>\n" return s + "<|im_start|>assistant\n" ids = tok(chatml(system, [("user", "周末一般干嘛？")]), return_tensors="pt").to(model.device)eos = [tok.eos_token_id, tok.convert_tokens_to_ids("<|im_end|>")]with torch.no_grad(): out = model.generate(**ids, max_new_tokens=300, do_sample=True, temperature=0.8, top_p=0.95, repetition_penalty=1.05, eos_token_id=eos, pad_token_id=tok.pad_token_id or eos[0])raw = tok.decode(out[0][ids.input_ids.shape[1]:], skip_special_tokens=True)bubbles = [ln for ln in raw.split("\n") if ln.strip()]print(bubbles)llamafactory-cli chat \ --model_name_or_path ./Qwen3.6-35B-A3B \ --adapter_name_or_path s-g-labs/linlu-lora-v0.1-qwen3.6-35b-a3b \ --template qwen --trust_remote_code