PassingByPixels

Qwythos-27B-v1-NVFP4

README

License: apache-2.0

What is quantised, and what is not

Table with columns: Component, Precision
Component	Precision
Text backbone Linear layers (64 layers, hybrid Gated-DeltaNet + full attention)	NVFP4 (FP4 weights + activations, group size 16)
`lm_head`	bf16
Vision tower (all 333 `model.visual.*` tensors)	bf16
Gated-DeltaNet `conv1d` layers	bf16
KV cache	FP8 (e4m3) scales exported

1699 quantizers inserted; none landed on the vision tower, lm_head or the conv1d layers. Calibrated on 512 samples at 2048 tokens (cnn_dailymail).

Measured

Load-verified on a DGX Spark (GB10, 128 GB unified) under vLLM 0.25.2: loads clean, no restarts, and generates coherent text — 13.6 tok/s single-stream without speculation, token count from usage.completion_tokens.

That figure is a smoke test, not a benchmark: one short single-stream generation, no concurrency sweep, no quality evaluation against the bf16 source. For the fuller measurement set (three context points, acceptance figures) see the MTP repo's card.

Context length

Ships the native 262,144-token configuration (max_position_embeddings 262144, no rope_scaling). Upstream declares 1,048,576 via static YaRN over an original 262144; static YaRN applies at every length and slightly degrades short-context quality, and 262,144 is comfortably servable on this class of box (a 2,138,983-token KV cache = 8.1 concurrent requests at 262k, against 2.0 at 1M).

Get the full 1M window at serve time without editing anything:

markdown
--hf-overrides '{"rope_scaling":{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":262144},"max_position_embeddings":1048576}'

Usage (vLLM)

bash
vllm serve PassingByPixels/Qwythos-27B-v1-NVFP4 \
    --trust-remote-code \
    --max-model-len 131072 \
    --kv-cache-dtype fp8 \
    --enable-auto-tool-choice \
    --tool-call-parser qwen3_xml \
    --reasoning-parser qwen3

Sampling, from the upstream generation_config.json: temperature 0.6, top_p 0.95, top_k 20, repetition_penalty 1.05.

Limitations

4-bit quantisation is lossy. No quality evaluation against the bf16 source was run here.
No MTP head — see the box above.
Vision was frozen upstream and the tower is preserved in bf16, but image behaviour is untested post-quantisation.
The VL processor configs and chat template are not emitted by ModelOpt and were copied verbatim from Empero's own checkpoint, so they match the source exactly.
Qwythos is intentionally uncensored for technical and research use — see the original card. Apply your own application-level controls.
This is the pre-RL v1 checkpoint.

Credits

Empero — the model.
Qwen team — the base.
CocaKova — the first public NVFP4 of this model.
NVIDIA — TensorRT Model Optimizer.

Quantised and load-verified by PassingByPixels on a DGX Spark. Apache-2.0, same as the original.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

PassingByPixels

Model Tree

Base

empero-ai/Qwythos-27B-v1

Quantized

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality