GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
srallaba
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
srallaba
Model Tree
Base
Qwen/Qwen3.6-35B-A3B
Adapter
this model
Input Modalities
Text
Image
Video
Output Modalities
Text
Supported Functionality
Dedicated Endpoints
The 256 routed experts are stored as fused tensors (not per-expert nn.Linear), so standard
per-expert PEFT is impossible. We instead freeze all 256 routed experts and the router
(Chinese capacity and routing untouched) and LoRA-adapt only the shared-expert + attention
projections (7 module types, ~8.4M params, 0.024%). A routing profile over hi/te vs zh/en confirmed
Indic specialization concentrates in the final + early script layers. Trained with continued
pretraining on IndicCorpV2/Sangraha Hindi+Telugu (CC0/CC-BY) plus English+Chinese replay.
Qwen/Qwen3.6-35B-A3Bgate/up/down_proj) + attention (q/k/v/o_proj), rsLoRA r=16Net-neutral vs the strong base on MCQ (base is already near-ceiling): Hindi macro ~82%, Telugu macro ~70%, Chinese retention preserved. The contribution is the method (routing-steered, Chinese-frozen Indic adaptation of a fused-expert MoE), not a large MCQ delta.
python
from peft import PeftModelfrom transformers import AutoModelForCausalLM, AutoTokenizerbase = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.6-35B-A3B", torch_dtype="bfloat16", device_map="auto")model = PeftModel.from_pretrained(base, "srallaba/tarak-qwen3.6-35b-a3b-rsi-v1")
Project Tarak — Indic-language LLM exploration.