XReyRobert

Qwopus3.6-27B-Coder-GPTQ-Pro

README

License: apache-2.0

Source And Credits

Source model:

Jackrong/Qwopus3.6-27B-Coder

Quantization tooling and reference recipe:

Thanks to Jackrong for the Qwopus3.6 models and to groxaxo for GPTQ-Pro and the Qwen3.6 GPTQ-Pro recipe this run was aligned with.

Artifact Summary

Table with columns: Field, Value
Field	Value
Source model	`Jackrong/Qwopus3.6-27B-Coder`
Architecture	`Qwen3_5ForConditionalGeneration`
Model type	`qwen3_5`
Tensor files	`6`
Safetensors size	`17.63 GiB`
Indexed tensors	`2423`
Quantized `qweight` tensors

This upload includes an MTP-aware GPTQ patch shard:

model-mtp-aware-gptq.safetensors
MTP_AWARE_GPTQ_PATCH.json

That means the artifact has MTP tensors present and quantized MTP linears, but it does not yet mean speculative decoding is a recommended serving mode. See the MTP status notes below.

Quantization Recipe

Table with columns: Setting, Value
Setting	Value
Method	GPTQ-Pro / GPTQModel
Quantizer	`gptqmodel:6.1.0-dev`
Bits	`4`
Group size	`128`
Symmetric quantization	`true`
Desc act	`false`
True sequential

MTP-aware patch metadata:

Table with columns: Field, Value
Field	Value
Patch type	`mtp-aware-gptq-pro-core`
MTP bits	`4`
MTP group size	`128`
MTP calibration samples	`256`
MTP calibration length	`2048`
Quantized MTP key count	`32`

Quantized MTP modules:

mtp.fc
mtp.layers.0.self_attn.q_proj
mtp.layers.0.self_attn.k_proj
mtp.layers.0.self_attn.v_proj
mtp.layers.0.self_attn.o_proj
mtp.layers.0.mlp.gate_proj
mtp.layers.0.mlp.up_proj
mtp.layers.0.mlp.down_proj

Intended Serving Shape

This checkpoint is intended for text-only vLLM serving as a local coding-agent model.

Recommended starting point:

bash
vllm serve XReyRobert/Qwopus3.6-27B-Coder-GPTQ-Pro \
  --served-model-name qwopus3.6-27b-coder-gptq-pro \
  --language-model-only \
  --dtype float16 \
  --quantization gptq_marlin \
  --tensor-parallel-size 1 \
  --max-model-len 131072 \
  --max-num-seqs 1 \
  --kv-cache-dtype fp8_e5m2 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_coder \
  --enable-prefix-caching \
  --gpu-memory-utilization 0.95 \
  --trust-remote-code

Serving context for the published Smoke24/vLLM measurements:

The Smoke24/vLLM numbers were collected on an internal llm-residency vLLM deployment. The custom image recipe is not published yet, so this card does not present that image as a public reproduction target. The stable serving knobs captured from the run are listed for context.

Table with columns: Field, Value
Field	Value
Nomad job profile	`vllm-qwopus36-coder`
Served model name	`qwopus3.6-27b-coder-gptq-pro-foem-4bit-g128-ns256`
Critical flags	`--dtype float16`, `--quantization gptq_marlin`, `--kv-cache-dtype fp8_e5m2`, `--reasoning-parser qwen3`, `--tool-call-parser qwen3_coder`, `--max-model-len 131072`; `--max-num-batched-tokens` was not set explicitly.

For initial production-style testing, keep speculative decoding off until you have validated MTP behavior with your exact vLLM version and workload.

Public vLLM Reproducibility

This artifact has a public reproducibility path on the unmodified upstream vLLM OpenAI image:

image: docker.io/vllm/vllm-openai:nightly-7a1eb8ac2ec4ea69338c51dc7afd4b15010abfa8
vLLM version observed in validation: 0.20.1rc1.dev16+g7a1eb8ac2
GPU class: single RTX 3090 24 GB / Ampere
--enforce-eager was not used
no local sleep/wake patch or localhost/*sleepwake* image is required for the validation below

Validated serving shape:

context: --max-model-len 131072
--language-model-only, --dtype float16, --quantization gptq_marlin
--kv-cache-dtype fp8_e5m2, --enable-prefix-caching, --max-num-seqs 1
--max-cudagraph-capture-size 32, --gpu-memory-utilization 0.95
--reasoning-parser qwen3, --tool-call-parser qwen3_coder
--enable-sleep-mode was included in the validation command

Startup note: this dense 27B Coder profile can fail the first cold start after torch compile/profiling with a pessimistic KV-cache check. The public validation passed on the second start when reusing persistent vLLM/Nomad-style cache directories such as TORCHINDUCTOR_CACHE_DIR=/data/vllm-qwopus36-coder/torch_compile_cache and VLLM_CACHE_ROOT=/data/vllm-qwopus36-coder. Treat startup retry plus persistent compile cache as part of the serving recipe.

Validation And Benchmarks

Completed artifact checks:

Local shard index inspection completed before upload.
Remote file list verified after upload.
Remote model.safetensors.index.json verified after upload.
Index metadata total size matches the local safetensor shards.
The remote artifact contains the expected safetensor shards.

Terminal-Bench 2.0 Smoke24 result and associated vLLM serving measurements. This Smoke24 run used max_model_len=131072 for apples-to-apples comparison with the other local models in this publication batch:

Table with columns: Run, Score, Success rate, Wall-time, Output tokens, Observed decode, LLM API time
Run	Score	Success rate	Wall-time	Output tokens	Observed decode	LLM API time
`qwopus3.6-27b-coder-gptq-pro-foem-4bit-g128-ns256`	`16/24`	`66.7%`	`218.8m`	`202.2k`	`38.9 tok/s`

Smoke24 is a fixed 24-task Terminal-Bench 2.0 comparison corpus, not a full Terminal-Bench leaderboard run.

In this local harness, the coder artifact:

tied Qwopus3.6-27B-v2-GPTQ-Pro-v1 on solved tasks at 16/24;
had the fastest wall time among the compared local runs at 218.8m;
emitted the fewest output tokens among the compared local runs at 202.2k;
had the lowest LLM API time among the 16/24 Smoke24 runs in this local batch.

Task list and harness shape:

benchmarks/terminal-bench-2.0/smoke24_task_list_20260616.md

MTP And Vision Status

The artifact contains mtp.* tensors.
The MTP large linears listed above were quantized with an MTP-aware GPTQ-Pro core capture path.
MTP speculative decoding is not yet published as the recommended serving mode for this artifact; validate it separately before relying on it.
Vision/visual tensors are present because of the source checkpoint structure, but this release is positioned and validated as text-only.

Limitations

Experimental quantization.
Terminal-Bench Smoke24 is a small local comparison corpus, not a full benchmark submission.
The coder Smoke24 result is assembled from a smoke12 run plus a missing12 complement run over the same fixed 24-task corpus.
MTP tensors are present, but speculative decoding is not yet a supported recommendation.
Vision tensors are present, but vision behavior has not been validated.
Loader behavior may vary across vLLM, Transformers, GPTQModel, and GPTQ-Marlin versions.

Files

Key files:

model.safetensors.index.json
model-00001-of-00005.safetensors through model-00005-of-00005.safetensors
model-mtp-aware-gptq.safetensors
MTP_AWARE_GPTQ_PATCH.json
config.json
quantize_config.json
processor_config.json
tokenizer.json

UPLOAD_MANIFEST.json records the upload guardrail checks and artifact inspection summary.

References

Source model: Jackrong/Qwopus3.6-27B-Coder
GPTQ-Pro tooling: groxaxo/GPTQ-Pro
Reference recipe: groxaxo/Qwen3.6-27B-GPTQ-Pro-4bit
Terminal-Bench: laude-institute/terminal-bench

Individual Project Notice

This repository is an individual research project. It is not affiliated with, sponsored by, or endorsed by any employer or organization.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider