GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
XReyRobert
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
XReyRobert
Model Tree
XReyRobert/Qwopus3.8-27B-Flash-GPTQ-Pro-FOEM-4bit-g128-ns256
Input Modalities
Output Modalities
Supported Functionality
The conversion completed on CPU in 88.50 seconds. Full stored-weight reconstruction relative L2 errors, measured against the BF16 source targets:
These are weight reconstruction errors, not measured task-quality losses. The conversion gate requires finite errors below 0.01.
The target-specific functional canary passed on 2026-09-04 using a single RTX 3090 (24 GiB) and the patched vLLM 0.27.1 recipe below:
See validation.json for measured usage and timings. These are single-run
functional checks, not a statistically robust benchmark or a broad task-quality
evaluation. Vision and MTP inference remain unvalidated.
This is not a drop-in checkpoint for stock vLLM or a claim of generic
Transformers compatibility. The token embedding uses the custom
hybrid_embedding_w8a16 configuration extension. A patched vLLM 0.27.1 is
required to route it through compressed-tensors' INT8 dequant-on-gather method.
The output head and transformer layers use GPTQ/Marlin.
Included under runtime/:
Containerfile, based on vllm/vllm-openai:v0.27.1;patches/qwen3_5-embed-quant.patch;patches/qwen38-gptq-hybrid-embedding.patch;patches/v0271-kimi-k3-warmup-model-gate.patch;serve.sh, the text-only 210k recipe;convert.py, the additional INT8 conversion source.Build the image from the included runtime directory:
sh
podman build -t qwopus38-hybrid-int8:v0271 -f runtime/Containerfile runtime
Download this repository with an authorized Hugging Face client, mount its
checkpoint directory read-only at /model, and invoke runtime/serve.sh
inside that image. Store compiler caches in a separate writable directory.
The script requires a caller-provided VLLM_API_KEY in the environment.
Use GPU container options appropriate to your environment; no privileged
container is required.
This profile uses float16 activations, FP8 E5M2 KV cache, an explicit 7,300,000,000-byte KV pool, prefix caching, sleep mode, an eight-sequence limit and CUDA graph capture up to 16. Its 210,000-token limit is below the source's native 262,144-token configuration. GPU capacity and concurrent request memory must be validated on the serving host.
The checkpoint's tool template emits JSON within <tool_call> tags, so this
recipe selects the Hermes tool parser and the Qwen3 reasoning parser.
MTP speculation is disabled and serving is text-only.
44d24e8cb20ceb3cdf4fe200b5a0afd970ee748a.f04b2cb488144f0484aa4b443358701573172e0e.config.json.quantize_config.json is synchronized with the effective
config.json.quantization_config, including INT8 output-head and packed
embedding metadata. The source's pre-conversion sidecar would otherwise
incorrectly describe an unquantized output head.quantization_report.json and source_provenance.json document the derivation.Credit for the fine-tune belongs to Jackrong, the base architecture to the Qwen team, and GPTQ-Pro/FOEM tooling to its respective authors.
publication_manifest.json records SHA-256 and byte lengths of uploaded model
files. Local cache folders and authentication files are excluded.