deluxetiky

Qwopus3.8-27B-Flash-GPTQ-4bit

README

License: apache-2.0

Attribution & Credits

This is a quantized derivative. All credit for the underlying model belongs to its original developers:

Table with columns: Contribution, Developer, Model / Link
Contribution	Developer	Model / Link
Original base model	Qwen Team (Alibaba)	`Qwen/Qwen3.8-27B`
Fine-tune ("Flash" reasoning-stability training)	Jackrong	`Jackrong/Qwopus3.8-27B-Flash`
Quantization method (FOEM + GPTQ)	ModelCloud (GPTQModel) / FOEM authors	GPTQModel · FOEM (AAAI 2026)
Quantization performed by	deluxetiky (Sinan)	this repo

License: Apache-2.0 (inherited from the base and fine-tune).

Quantization Details

Method: GPTQ 4-bit with FOEM error compensation (FOEM + GPTAQ hybrid)
Group size: 32 (g32)
Symmetric: yes · desc_act: false · lm_head: not quantized
mse: 2.0 · damp_percent: 0.05 · true_sequential: yes
FOEM: α=0.25, β=0.2 · act_group_aware: yes
Tool: gptqmodel (FOEMConfig), calibration: C4 (128 samples)
Format: gptq, int32-packed → vLLM Marlin/GPTQ kernel path

This reproduces the exact recipe used for the production btbtyler09/Qwen3.8-27B-GPTQ-4bit checkpoint so results are comparable.

Serving

Tested on vLLM 0.27.1 with the GDN FlashInfer decode backport, 2× RTX 5090 (TP2):

bash

vllm serve deluxetiky/Qwopus3.8-27B-Flash-GPTQ-4bit \  --tensor-parallel-size 2 --max-model-len 262144 --kv-cache-dtype fp8 \  --gpu-memory-utilization 0.92 --reasoning-parser qwen3 \  --gdn-decode-backend flashinfer --mamba-block-size 16 --enable-prefix-caching \  --enable-auto-tool-choice --tool-call-parser qwen3_xml

Measured A/B vs base (same quant, same stack)

Head-to-head against the stock Qwen3.8-27B (same FOEM g32 quant) on 2×5090 TP2:

Table with columns: Metric, Qwen3.8 base, Qwopus-Flash, Gain
Metric	Qwen3.8 base	Qwopus-Flash	Gain
Decode @1 stream	~61 tok/s	~99 tok/s	+64%
Aggregate @8 concurrent	~232 tok/s	~600 tok/s	+159%
Long task completion (6-task planner suite, 3000-tok cap)	4/6 answered; 2 truncated empty	4/6 answered incl. 7.8k-char outputs; 2 truncated	completes far more
Code correctness (execution-graded, 6 tasks / 29 tests)	24/29 (83%)

Note: this A/B ran on two different physical 2×5090 boxes (same GPU/stack), so absolute concurrency figures should be re-validated on identical hardware; the completion-rate difference is cap-bound and box-independent.

Files

6 × model-*.safetensors (~20 GB total) + config, tokenizer, chat template.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

deluxetiky

Model Tree

Base

Jackrong/Qwopus3.8-27B-Flash

Quantized

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality