GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
PhilipEriksson
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
PhilipEriksson
Model Tree
Qwen/Qwen3.5-0.8B
Input Modalities
Output Modalities
Supported Functionality
lora_alpha=64, dropout 0, bias="none"q_proj, k_proj, v_proj, o_proj, in_proj_qkv, in_proj_z,
in_proj_b, in_proj_a, out_proj, gate_proj, up_proj, down_projQwen3.5 uses hybrid attention: 6 of 24 layers are full attention (q/k/v/o_proj), the
other 18 are linear attention (in_proj_qkv, in_proj_z, in_proj_b, in_proj_a,
out_proj). The vision tower was excluded — this is a text-only fine-tune.
python
import torchfrom transformers import AutoModelForImageTextToText, AutoProcessorfrom peft import PeftModel base_id = "Qwen/Qwen3.5-0.8B"adapter_id = "PhilipEriksson/Qwen3.5-0.8B-Opus-Reasoning-Distilled-LoRA" processor = AutoProcessor.from_pretrained(base_id)model = AutoModelForImageTextToText.from_pretrained(base_id, torch_dtype=torch.bfloat16)model = PeftModel.from_pretrained(model, adapter_id)model.eval() messages = [{"role": "user", "content": "What is 17 * 23?"}]text = processor.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)inputs = processor(text=text, return_tensors="pt").to(model.device) with torch.no_grad(): out = model.generate(**inputs, max_new_tokens=500, temperature=0.6, top_p=0.95, do_sample=True)print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))enable_thinking=True is required. The Qwen3.5 chat template defaults this flag to
off, rendering an empty think block, and the model then answers with no reasoning.
python
from peft import PeftModelfrom transformers import AutoModelForImageTextToText, AutoProcessor base = AutoModelForImageTextToText.from_pretrained( "Qwen/Qwen3.5-0.8B", torch_dtype=torch.bfloat16)merged = PeftModel.from_pretrained( base, "PhilipEriksson/Qwen3.5-0.8B-Opus-Reasoning-Distilled-LoRA").merge_and_unload()merged.save_pretrained("merged")AutoProcessor.from_pretrained("Qwen/Qwen3.5-0.8B").save_pretrained("merged")max_length=16384, packing=Falseper_device_train_batch_size=8, gradient_accumulation_steps=4 (effective 32)learning_rate=2e-4, adamw_8bit<|im_start|>assistant\n masked to -100Final mean loss 0.7077; best single step 0.4458. The curve plateaued around 0.65–0.67 — see the merged model card for the full discussion and limitations.
import torchfrom transformers import AutoModelForImageTextToText, AutoProcessorfrom peft import PeftModel base_id = "Qwen/Qwen3.5-0.8B"adapter_id = "PhilipEriksson/Qwen3.5-0.8B-Opus-Reasoning-Distilled-LoRA" processor = AutoProcessor.from_pretrained(base_id)model = AutoModelForImageTextToText.from_pretrained(base_id, torch_dtype=torch.bfloat16)model = PeftModel.from_pretrained(model, adapter_id)model.eval() messages = [{"role": "user", "content": "What is 17 * 23?"}]text = processor.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)inputs = processor(text=text, return_tensors="pt").to(model.device) with torch.no_grad(): out = model.generate(**inputs, max_new_tokens=500, temperature=0.6, top_p=0.95, do_sample=True)print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))from peft import PeftModelfrom transformers import AutoModelForImageTextToText, AutoProcessor base = AutoModelForImageTextToText.from_pretrained( "Qwen/Qwen3.5-0.8B", torch_dtype=torch.bfloat16)merged = PeftModel.from_pretrained( base, "PhilipEriksson/Qwen3.5-0.8B-Opus-Reasoning-Distilled-LoRA").merge_and_unload()merged.save_pretrained("merged")AutoProcessor.from_pretrained("Qwen/Qwen3.5-0.8B").save_pretrained("merged")