What is included
- Approximately 18.92 GB of newly quantized GPTQ backbone tensors, from the
retained JonathanColetti BF16 ablated checkpoint, not a GGUF round trip.
- Approximately 210 MB of upstream FP32 indexer weights, unchanged from
higa2515/Qwen3.8-27B-QSA-BF16 at
aedae2c5304698f58d30c5ec709ebf5b37339b1d.
- The experimental
qsa27b vLLM plugin, exact runtime source overlays, portable
launcher, synthetic test scripts and JSON measurements.
- Vision tensors are retained. The artifact check found 333 vision tensors
unchanged against the source and 96 Gated DeltaNet projections kept at 16 bits.
Loading the GPTQ checkpoint in stock vLLM does not activate QSA. The indexer
file is deliberately separate from the standard model weight index. Follow
INSTALL.md to use the plugin. This is not a Transformers
trust_remote_code integration or a llama.cpp/GGUF package.
The backbone is W4G128 symmetric GPTQ with BF16 activations; the tested KV cache
uses FP8 E4M3. Calibration: 256 samples × 2048 tokens from NeelNanda/pile-10k,
200 AutoRound iterations, batch size 4, seed 42. See
quantization-recipe.json.
The original BF16 download did not retain a verified revision: the current
upstream commit in provenance.json is a reviewed reference,
not a claim of exact source identity. Released tensor bytes are fixed by
SHA256SUMS.
Final paired measurements
Both arms use the same released W4A16 backbone, two RTX 3090 24GB GPUs,
TP=2, BF16 activations, FP8 E4M3 KV, compilation/CUDA Graphs enabled,
DFlash with 7 speculative tokens, vision and prefix caching enabled.
Maximum context 262144; maximum sequences 16; batch tokens 2048; GPU memory
utilization 0.78. Only the QSA plugin is toggled. Dense is not a W4A8 baseline.
Each length is a different cold prompt, identical between arms, temperature 0,
seed 42, 256 output tokens. It asks for an early reference code and then a story.
Table with columns: Actual input tokens, Dense decode tok/s, QSA decode tok/s, Change, Dense TTFT s, QSA TTFT s, Dense total s, QSA total s| Actual input tokens | Dense decode tok/s | QSA decode tok/s | Change | Dense TTFT s | QSA TTFT s | Dense total s | QSA total s |
|---|
| 1,024 | 146.43 | 100.56 | −31.3% | 0.34 | 0.78 | 2.09 | 3.32 |
| 32,756 | 120.34 | 103.50 | −14.0% | 11.32 | 33.98 | 13.44 | 36.44 |
| 131,067 | 90.54 | 95.63 | +5.6% | 58.97 | 149.64 | 61.78 | 152.31 |
| 249,985 | 74.77 | 83.42 | +11.6% | 144.12 | 310.75 | 147.53 | 313.81 |
Very short prompts with 512 output tokens: narrative 133.70 → 122.16 tok/s;
code 225.75 → 170.68 tok/s (Dense → QSA).
Repeating the 249,985-token prompt hit 247,200 prefix tokens in both arms:
Dense/QSA TTFT 3.47/4.92 s, decode 74.86/83.05 tok/s, total 6.88/7.99 s.
The public raw records are in results/, with validation/ratio
calculation in benchmarks/compare.py.
Decode is estimated between first/last nonempty streaming content events;
speculative chunks are grouped. These are single measurements, not confidence
intervals. Outputs differ between arms and draft acceptance changes: these are
same-prompt application measurements, not a fixed-output kernel comparison.
Compatibility evidence and scope
The final implementation captured all 16 full-attention layers on both TP ranks.
Observed post-readiness FULL Graph replays include both the main model and
DFlash. See results/graph-proof.json.
Both arms passed 14 bounded text/tool/vision checks, the four reference-code
retrieval cases, and two simultaneous synthetic image/text requests. Device
checks cover dynamic Graph replay, 5/7-token speculative rollback, FP8 paged
attention and deterministic Top-K cutoff ties across both GPUs.
These are not comprehensive quality, safety, long-duration stability,
16-way concurrency, or full-262K-input evaluations. Highest measured input was
249,985 tokens. No upstream model benchmark score is attributed to this release.
The portable packaging received static/CPU checks; there was no fresh clean-room
GPU run of the installation instructions. The plugin source matches the tested
implementation; environment reproduction remains a separate boundary.
Why it can be slower
QSA adds FP32 index projections, compressed-key scoring, Top-K selection and
paged gathers at 16 layers. Up to 512 four-token blocks plus the causal tail are
read by the sparse attention kernel. Short contexts save little attention work.
The new sparse kernel does not automatically inherit the optimized dense
FlashAttention prefill path. CUDA Graphs retain scheduling benefits, but do not
remove these GPU computations. Full per-operator profiling has not been done;
no precise percentage attribution is claimed.
Provenance, licenses and limitations
Weights and original integration code are Apache-2.0; retain the upstream notices
and component licenses in attribution/. FA2's vendored dependencies
retain their own licenses. See NOTICE.
Research/local evaluation only until independently validated for the intended
application. This release does not restore the official model's safety behavior,
certify model quality, or provide a deployment safety layer.