GLM-5.2 is live. #1 throughput on OpenRouter, pay-per-token on FriendliAI. Try it today ➜
XReyRobert
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
XReyRobert
Model Tree
Jackrong/Qwopus3.6-35B-A3B-v1
Input Modalities
Output Modalities
Supported Functionality
Source model:
Quantization tooling and reference recipe:
Thanks to Jackrong for the Qwopus3.6 model and to groxaxo for the GPTQ-Pro tooling and Qwen3.6 GPTQ-Pro recipe this run was aligned with.
| Field | Value |
|---|---|
| Source model | Jackrong/Qwopus3.6-35B-A3B-v1 |
| Architecture | Qwen3_5MoeForConditionalGeneration |
| Model type | qwen3_5_moe |
| Tensor files | 6 |
| Safetensors size | 20.81 GiB |
| Indexed tensors | 124595 |
Quantized qweight tensors |
The artifact contains source MTP and vision/visual tensors in its weight index. That does not mean MTP speculative decoding or multimodal serving is already recommended. The validated use so far is text-oriented GPTQ serving and Terminal-Bench agent evaluation.
| Setting | Value |
|---|---|
| Method | GPTQ-Pro / GPTQModel |
| Bits | 4 |
| Group size | 128 |
| Symmetric quantization | true |
| Desc act | false |
| True sequential | true |
| Calibration dataset |
Dynamic skip rules preserved these module families instead of quantizing them:
embed_tokenslm_headmtpnormvisionvisualIn practical terms, the language tower linear layers are the intended GPTQ-Pro payload, while embeddings, norms, MTP, and vision-related tensors remain preserved as non-quantized tensors.
This checkpoint is intended for advanced users testing text-only vLLM or GPTQ-compatible serving for Qwen/Qwopus MoE checkpoints.
A starting vLLM shape for text-only testing:
bash
vllm serve XReyRobert/Qwopus3.6-35B-A3B-v1-GPTQ-Pro \--served-model-name qwopus3.6-35b-a3b-v1-gptq-pro \--language-model-only \--dtype float16 \--quantization gptq_marlin \--tensor-parallel-size 1 \--max-model-len 262144 \--max-num-seqs 1 \--kv-cache-dtype fp8_e5m2 \--reasoning-parser qwen3 \--enable-auto-tool-choice \--tool-call-parser qwen3_coder \--enable-prefix-caching \--gpu-memory-utilization 0.95 \--trust-remote-code
Serving context for the published Smoke24/vLLM measurements:
The Smoke24/vLLM numbers were collected on an internal llm-residency vLLM deployment. The custom image recipe is not published yet, so this card does not present that image as a public reproduction target. The stable serving knobs captured from the run are listed for context.
| Field | Value |
|---|---|
| Nomad job profile | vllm-qwopus36-35b-a3b-262k |
| Served model name | qwopus3.6-35b-a3b-v1-gptq-pro-foem-4bit-g128-ns256-ctx262k |
| Critical flags | --dtype float16, --quantization gptq_marlin, --kv-cache-dtype fp8_e5m2, --reasoning-parser qwen3, --tool-call-parser qwen3_coder, --max-model-len 262144, --max-num-batched-tokens 2096 |
| Benchmark context |
Treat the command as a serving starting point, not a compatibility guarantee for every vLLM release. GPTQ-Marlin, Qwen3.6 MoE handling, and multimodal processor behavior are all loader-version sensitive.
The RTX 3090 image above reflects separate 262k-context serving validation.
This artifact has a public reproducibility path on the unmodified upstream vLLM OpenAI image:
docker.io/vllm/vllm-openai:nightly-7a1eb8ac2ec4ea69338c51dc7afd4b15010abfa80.20.1rc1.dev16+g7a1eb8ac2--enforce-eager was not usedlocalhost/*sleepwake* image is required for the validation belowValidated serving shapes:
--max-model-len 131072 validated with --gpu-memory-utilization 0.95--max-model-len 262144 validated with --gpu-memory-utilization 0.96--language-model-only, --dtype float16, --quantization gptq_marlin--kv-cache-dtype fp8_e5m2, --enable-prefix-caching, --max-num-seqs 1--max-num-batched-tokens 2096, --max-cudagraph-capture-size 32The 262k profile is tight on 24 GB GPUs; gpu_memory_utilization=0.95 was short on KV cache in validation, while 0.96 passed.
Completed artifact checks:
model.safetensors.index.json verified after upload.Terminal-Bench 2.0 Smoke24 result and associated vLLM serving measurements.
This Smoke24 run used max_model_len=131072 for apples-to-apples comparison
with the other local models in this publication batch:
| Run | Score | Success rate | Wall-time | Output tokens | Observed decode | LLM API time |
|---|---|---|---|---|---|---|
qwopus3.6-35b-a3b-v1-gptq-pro-foem-4bit-g128-ns256 | 12/24 | 50.0% | 226.7m | 622.8k | 138.6 tok/s |
Smoke24 is a fixed 24-task Terminal-Bench 2.0 comparison corpus, not a full Terminal-Bench leaderboard run. The score above is useful for fast regression and local serving comparison, not for broad model ranking.
Task list and harness shape:
config.json advertises MTP support, and the index contains mtp.* tensors.For now, publish and use this as a text-first GPTQ-Pro MoE artifact.
Key files:
model.safetensors.index.jsonmodel-00001-of-00006.safetensors through model-00006-of-00006.safetensorsconfig.jsonquantize_config.jsonprocessor_config.jsontokenizer.jsonUPLOAD_MANIFEST.jsonUPLOAD_MANIFEST.json records the upload guardrail checks and artifact
inspection summary.
Jackrong/Qwopus3.6-35B-A3B-v1groxaxo/GPTQ-Progroxaxo/Qwen3.6-27B-GPTQ-Pro-4bitlaude-institute/terminal-benchThis repository is an individual research project. It is not affiliated with, sponsored by, or endorsed by any employer or organization.
30970 |
mtp.* tensors in index | true |
| vision/visual tensors in index | true |
| Index metadata size matches shards | true |
| WikiText |
| Calibration samples | 256 |
| Calibration sequence length | 2048 |
| MSE | 2.0 |
| Damp percent | 0.05 |
| Damp auto increment | 0.01 |
| FOEM alpha | 0.25 |
| FOEM beta | 0.2 |
| FOEM device | cuda:0 |
| MoE routing | ExpertsRoutingBypass |
| MoE bypass batch size | 192 |
| Pack implementation | cpu |
Smoke24 quality rows used max_model_len=131072 for apples-to-apples comparison; the image above reflects the validated 262k residency serving profile. |
--reasoning-parser qwen3--tool-call-parser qwen3_coder74.9m |