GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
jhone888
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
jhone888
Model Tree
huihui-ai/Huihui-Ornith-1.5-35B-A3B-abliterated
Input Modalities
Output Modalities
Supported Functionality
| Component | Format |
|---|---|
| Routed experts (un-fused, 256/layer) | W4A16_NVFP4, group-16 |
| Shared experts, lm_head | W4A16_NVFP4, group-16 |
| Attention / linear-attention projections | FP8 (per-tensor weight scale) |
| MTP head, vision tower, norms, embeddings, conv | BF16 (untouched) |
| KV cache | FP8 (config flag) |
23.4 GB across 5 shards. Tensor-level audit against the official ornith-ai/Ornith-1.5-35B-A3B-NVFP4 recipe: weight_scale_2 = 30,841 and 291 quantized layers (FP8 130 + W4A16 161) — exact match. FP8 layers use static weight scales (no activation input_scale); verified working with the FreeToken FTW pipeline.
This repo ships the full multimodal config (model_type: qwen3_5_moe, nested text_config/vision_config) because the checkpoint genuinely contains the MTP head and vision tower. vLLM and transformers load it as-is.
FreeToken's ft checkpoint (FTW conversion) needs one change: swap config.json for the flat text-only variant, or the conversion fails with Missing MoE expert source layers. Why: with the multimodal config, FreeToken's model adapter dispatches expert loading to its BF16-bank provider, which searches for fused expert banks (model.layers.N.mlp.experts.gate_up_proj) that don't exist in any NVFP4 build (experts are stored un-fused and quantized per-expert). The flat text-only config (model_type: qwen3_5_moe_text, all text fields at top level) dispatches to the NVFP4 provider, which reads the un-fused quantized layout correctly.
The drop-in flat config is pottokao's config.json with one mandatory edit: architectures must be ["Qwen3_5MoeForConditionalGeneration"] (FreeToken's registry rejects ...ForCausalLM). The MTP/vision tensors are then ignored by FreeToken's text-only FTW; keep this folder's config for vLLM and other engines.