GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
MonumentalSystems
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
MonumentalSystems
Model Tree
Hcompany/Holo-3.1-35B-A3B
Input Modalities
Output Modalities
Supported Functionality
PEFT can only LoRA-train against an unquantized base, so this adapter is trained against the
BF16 Hcompany/Holo-3.1-35B-A3B (~70 GB, full-bf16 frozen base, no QLoRA/bitsandbytes).
Atlas then serves the compact NVFP4 base (Hcompany/Holo-3.1-35B-A3B-NVFP4) and applies this
BF16 adapter as a runtime delta at attention k/v/o on the 10 full-attention layers — never
merged, never dequantized. The BF16-delta-on-NVFP4-base quantization gap is why r=32 is used
(enough capacity that exact tokens mostly survive; occasional 4-digit drift under NVFP4 is expected).
target_modules = ["k_proj", "v_proj", "o_proj"] — q_proj is a hard reject (Holo gated Q|gate
interleave); MoE MLP (mlp.experts.* / gate/up/down) and GDN linear-attention layers are also rejects.layers_to_transform = [3, 7, 11, 15, 19, 23, 27, 31, 35, 39] — the 40-layer model's 10
full-attention layers (full_attention_interval=4); the other 30 are Gated-DeltaNet/linear.r = 32, lora_alpha = 64 (scale = α/r = 2.0), use_rslora = false, use_dora = false, bias = "none".Requires the Atlas F32→BF16 adapter-loader fix and the holo3_1_moe LoRA family allow-list.
bash
spark serve Hcompany/Holo-3.1-35B-A3B-NVFP4 \--lora-adapter holo35b=MonumentalSystems/Holo-3.1-35B-A3B-lora-demo \--max-lora-rank 64
peft + transformers Trainer, 10 epochs / 1200 steps on 6 overfit Q→A pairs, r=32,
lr 2e-4, bf16, full-bf16 frozen base + gradient checkpointing, ~75 min on a single NVIDIA GB10.
A demo artifact, not a general-purpose fine-tune.