GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
beita6969
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
beita6969
Model Tree
Qwen/Qwen3.5-9B
Input Modalities
Output Modalities
Supported Functionality
The trained R²Orch Director as a complete model. The Director's LoRA adapter (rank 64,
alpha 64, on q_proj, k_proj, v_proj, o_proj of the full-attention layers and on gate_proj, up_proj,
down_proj of every MLP) is merged into Qwen3.5-9B. The Workers of R²Orch are the unmodified Qwen3.5-9B; the Proposer
is a training-time component and is not part of this release.
The 128 merged weight matrices are stored in float32; every other tensor is the unchanged bfloat16 base weight. The LoRA update is about 0.13% of the weight norm (mean over the merged matrices, maximum 0.19%), below the roughly 0.4% relative resolution of bfloat16, so load the model in float32 to reproduce the trained Director exactly.
python
import torchfrom transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration model = Qwen3_5ForConditionalGeneration.from_pretrained("beita6969/R2Orch-9b", dtype=torch.float32)tokenizer = AutoTokenizer.from_pretrained("beita6969/R2Orch-9b")The merge was checked against the unmerged base + adapter in float32: the next-token logits differ by at most 0.004 and the KL divergence is below 1e-7.