200K setup
Requires native Linux, Docker, NVIDIA Container Toolkit, and the Hugging Face CLI. The supplied profile pins HyperQwen / vLLM 0.31.0 and the tested container image.
git clone https://github.com/syv-ai/HyperQwencd HyperQwengit checkout ce9615c4f2309be727407918a6c979bd67968069hf download protogeni/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4 \ --local-dir models/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4cp models/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4/serving/200k/profile.env.example .envcp models/Qwen3.8-27B-Uncensored-W4A16-RTX3090-MTP4/serving/200k/docker-compose.override.yml .make keygendocker compose --profile single up -d --no-deps single
Profile settings: MAX_LEN=200000, DRAFT_TOKENS=4, PTH_FA=1, EMBED_UVA=1, one active request, a 7.5 GiB KV pool, and 1,024-token prefill chunks. This left approximately 1.2 GiB VRAM free during testing with the desktop running; desktop workloads affect available headroom.
Connect an OpenAI-compatible client to http://localhost:18020/v1, model qwen3.8-27b, using the generated VLLM_API_KEY from .env. For LAN clients, set BIND=0.0.0.0 and use the server's address. Set client context to 200,000 tokens total, including the reply; reserving 8,192 output tokens leaves 191,808 for input and chat formatting.
Tested 2026-10-23 on RTX 3090 Ti, native Linux, NVIDIA 610.57.04, a 480 W power limit, and GPU compute unlock enabled. The unlock is separate from the serving profile and requires its own Linux boot setup; it is not required to fit 200K context.
Table with columns: Workload, Result| Workload | Result |
|---|
| Three warmed short coding prompts, 512 output tokens each | 148.6 tokens/s |
| 198,244 input + 1,536 output tokens | 104.9 tokens/s |
| Cold prompt near 198K: time to first token | 165.6–186.9 s |
| Prefix-cached follow-up near 198K: time to first token | 1.47 s |
Generation rates exclude prompt processing and include reasoning tokens. Long-context retrieval, cached follow-ups, repeat consistency, and tool calling passed, with zero preemptions in the measured runs. Measurement details. These are workload checks, not a comprehensive evaluation of quality at 200K.
Quality results (earlier evaluation)
Table with columns: Benchmark, Result| Benchmark | Result |
|---|
| GSM8K | 94.77% — 1,250/1,319 |
| MMLU-Pro, 5-shot | 76.14% — 1,066/1,400 |
| HumanEval | 94.51% pass@1 — 155/164 |
| HumanEval+ | 90.85% pass@1 — 149/164 |
| IFBench | 39.33% strict / 42.00% loose |
| MMMU validation | 24.56% — 221/900; 891 output-cap hits |
These scores predate the 200K profile and were not rerun for this update. MMLU-Pro used thinking; the other listed tests used thinking off. MMMU used an eager, MTP-off profile. Protocols, raw results, and earlier speed tests.
Lineage and license
Qwen/Qwen3.8-27B → OrcaRouter uncensored → noon-at-cgn AutoRound W4A16 → this serving-optimized checkpoint.
Build and calibration details. Apache-2.0 license.