GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
srswti
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
srswti
Model Tree
Qwen/Qwen3.6-35B-A3B
Input Modalities
Output Modalities
Supported Functionality
Through our own layer-by-layer profiling of activation distributions, routing sensitivity, and accumulated rounding error across the full architecture, we identified exactly which components can absorb 8-bit compression without behavioral change.
Quantized to F8_E4M3
All standard linear projections within the transformer blocks: Q, K, V, and output projections in attention, and the up, gate, and down projections in the routed expert MLPs. These layers represent the overwhelming majority of parameter count and memory bandwidth in the model.
Preserved at BF16
| Component | Reason |
|---|---|
| Visual encoder | Vision features have distributions that are structurally unlike language activations. Compressing them introduces grounding errors that propagate into cross-modal attention. |
| Gated DeltaNet / linear attention | Recurrent state is carried forward across every token in the sequence. Rounding errors here do not stay local. They accumulate. |
| MoE router gates | Routing decisions are discrete. A small numerical error can send a token to the wrong expert entirely, with effects that are not recoverable downstream. |
| Shared expert gate | The gate controls whether the shared expert fires at all. Same sensitivity as the router, applied every forward pass. |
| Shared expert MLP | Unlike routed experts, this layer is active for every token without exception. Its contribution compounds across the full sequence. |
| Token embeddings | A lookup table. Quantizing it saves almost nothing and introduces a fixed error floor on every single token representation before any computation begins. |
| Language model head | The final projection onto vocabulary logits. Precision here determines the shape of the output distribution. Errors at this layer affect sampling, greedy decoding, and low-probability token generation. |
Every quantized weight drops from 2 bytes to 1 byte. For the layers that are quantized, this is a direct 2x reduction in the memory required to hold the model.
The KV cache savings compound on top. During inference, every processed token writes a key vector and a value vector into a cache that persists for the duration of the request. The size of that cache is:
KV Cache=2×L×H×d×T×b
Where L is the number of layers, H is the number of KV heads, d is the head dimension, T is the sequence length, and b is bytes per element. Halving b from 2 (BF16) to 1 (F8) halves the KV cache at every sequence length. At 32K tokens, this frees several gigabytes per active request. That headroom goes directly toward concurrent capacity. Same hardware, more users.
Base model: Qwen/Qwen3.6-35B-A3B. All evaluations run at 0-shot using lm-evaluation-harness and lighteval, served with vLLM under --language-model-only.
| Category | Benchmark | Qwen3.6-35B-A3B | Axe Superveloce 37B | Recovery |
|---|---|---|---|---|
| Reasoning | GSM8K-Platinum (0-shot) | 94.98 | 95.12 | 100.1% |
| MMLU-Pro (0-shot) | 85.65 | 85.65 | 100.0% | |
| Math 500 (0-shot) | 84.93 | 84.33 | 99.3% | |
| AIME 25 (0-shot) | 91.25 | 91.25 | 100.0% | |
| GPQA Diamond (0-shot) | 83.00 | 83.16 | 100.2% | |
| Instruction Following | IFEval prompt-level strict (0-shot) | 91.00 | 90.45 | 99.4% |
| IFEval inst-level strict (0-shot) | 93.69 | 93.29 | 99.6% | |
| Coding | LiveCodeBench v6 (0-shot) | 75.43 | 76.38 | 101.3% |
On five of eight benchmarks, Axe Superveloce matches or exceeds the base model score. The compressed model outperforms its uncompressed counterpart on coding, a result consistent with per-channel weight scaling producing a tighter effective dynamic range on the neurons most active during code generation tasks.
Axe Superveloce is fully compatible with vLLM and loads natively without additional configuration.
Text only -- skip the vision encoder to free VRAM for additional KV cache:
bash
vllm serve srswti/axe-superveloce-37b --reasoning-parser qwen3 --language-model-onlyMultimodal -- full vision and language support:
bash
vllm serve srswti/axe-superveloce-37b --reasoning-parser qwen3Tool use:
bash
vllm serve srswti/axe-superveloce-37b --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_coderSpeculative decoding via Multi-Token Prediction:
bash
vllm serve srswti/axe-superveloce-37b --reasoning-parser qwen3 --speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}'Send requests using the OpenAI-compatible endpoint:
python
from openai import OpenAI client = OpenAI( api_key="EMPTY", base_url="http://<your-server-host>:8000/v1",) messages = [ {"role": "user", "content": "Explain quantum mechanics clearly and concisely."},] response = client.chat.completions.create( model="srswti/axe-superveloce-37b", messages=messages,) print(response.choices[0].message.content)Developed by SRSWTI Inc. - Building world's fastest retrieval and inference engines.
vllm serve srswti/axe-superveloce-37b --reasoning-parser qwen3 --language-model-onlyvllm serve srswti/axe-superveloce-37b --reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser qwen3_codervllm serve srswti/axe-superveloce-37b --reasoning-parser qwen3 --speculative-config '{"method":"qwen3_next_mtp","num_speculative_tokens":2}'from openai import OpenAI client = OpenAI( api_key="EMPTY", base_url="http://<your-server-host>:8000/v1",) messages = [ {"role": "user", "content": "Explain quantum mechanics clearly and concisely."},] response = client.chat.completions.create( model="srswti/axe-superveloce-37b", messages=messages,) print(response.choices[0].message.content)