MadSprite

Ornith-1.5-35B-A3B-BigBang-MTP-GPTQ-Int4-sym-G128

What is quantized

Table with columns: Part, Precision
Part	Precision
Routed MoE experts (40 layers × 256 experts × gate/up/down = 30,720 modules)	INT4, symmetric, group 128, no act-order
All attention: full attention and Gated DeltaNet linear attention	FP16/BF16 (unquantized)
Router gates, shared experts	unquantized
MTP head (Qwen3.6's trained head, as grafted by the merge)	unquantized (BF16)
Vision tower	unquantized
Embeddings, `lm_head`	unquantized

quantize_config.json: bits 4, group_size 128, desc_act false, sym true, checkpoint_format gptq, pack_dtype int32, with dynamic exclusions -:.*attn.*, -:.*mlp\.gate$, -:.*mtp.*, -:.*shared_expert.*, -:.*visual.*. On disk: 23 GB (22.7 GiB loaded).

Recipe

This follows palmfuture/Qwen3.6-35B-A3B-GPTQ-Int4, a recipe known to run on vLLM's Intel XPU MoE path, applied to the merge.

Tool: GPTQModel 7.5.0, torch 2.13.0+xpu, transformers 5.15.0.
Hardware: 1× Intel Arc Pro B70 32 GB, quantizing on the GPU (xpu:0) with offload_to_disk; 281 min.
Settings: damp_percent 0.05, act_group_aware, true_sequential. Rare-expert fallback: RTN for any expert reached by fewer than 0.5% of calibration tokens (GPTQModel default).
Calibration: 256 samples, mixed so all 256 experts see signal. Median 513 tokens, 158,821 tokens total.
Table with columns: Source, Samples
Source Samples
allenai/c4 (en, validation)

Table with columns: Source, Samples
Source	Samples
allenai/c4 (en, validation)

quant_log.csv (in this repo) has GPTQModel's per-module quantization log.

Results on an Intel Arc Pro B70 (vLLM XPU)

Settings for these measurements:

single request; MTP 4; FP8 KV cache; 131,072 context;
cold prompts with no prefix-cache hits; thinking off; fixed output length;
prefill = prompt tokens ÷ time to first token; decode = client tokens/s after the first token.

Table with columns: Prompt, This quant on vLLM (prefill / decode tok/s), Merge's Q4_K_M GGUF on llama.cpp SYCL + MTP draft
Prompt	This quant on vLLM (prefill / decode tok/s)	Merge's Q4_K_M GGUF on llama.cpp SYCL + MTP draft
~0.5K	3,820 / 154	764 / 78
~8K	8,201 / 124	1,253 / 71
~33K	5,816 / 114	1,198 / 52
~120K	2,663 / 90	916 / 33

MTP: mean acceptance length 3.3–3.5 tokens per step.
KV capacity: at --gpu-memory-utilization 0.96 the KV cache holds 257,166 tokens (1.96× a 131K request).
Functional checks, 5/5:
- arithmetic (347 × 29);
- fact recall;
- a reasoning trick question;
- a Python function that is executed and must pass tests;
- a tool call (qwen3_coder parser).
Vision: answers a colour-identification image test correctly.

These are speed measurements plus a small functional check, not a benchmark suite. Perplexity and task scores have not been measured.

Serving

vLLM on Intel Arc Pro B70 (XPU) — what was tested

Image: vllm/vllm-openai-xpu@sha256:f01e24f6c7ff01f1e0662234255a1372297d1dbd89d003cf13c8fad3eab1ba4f (vLLM 0.27.2rc1).
MTP on XPU: applies the two MTP patches from the B70 inference cookbook (patch_mtp_nightly.py, then patch_mtp_boundary.py) and sets B70_MTP_BF16_DRAFT=1, so the MTP draft is built unquantized.

bash

vllm serve MadSprite/Ornith-1.5-35B-A3B-BigBang-MTP-GPTQ-Int4-sym-G128 \  --quantization gptq --dtype float16 --kv-cache-dtype fp8 \  --max-model-len 131072 --gpu-memory-utilization 0.96 \  --max-num-seqs 1 --max-num-batched-tokens 8192 --enable-prefix-caching \  --limit-mm-per-prompt '{"image":64,"video":0}' \  --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 \  --speculative-config '{"method":"mtp","num_speculative_tokens":4}' \  --override-generation-config '{"temperature":0.6,"top_p":0.95,"top_k":20,"min_p":0.0}'

Memory: with attention, shared experts and vision in full precision, the weights take 22.7 GiB. At --gpu-memory-utilization 0.90, a 32 GB card can't fit KV for one 131K request, so use 0.96, or lower --max-model-len.
Sampling: the Ornith-1.5 card recommends temperature 0.6, top_p 0.95, top_k 20 for general tasks, and temperature 1.0 to reproduce its benchmarks.
Reasoning: always on (<think>). Use --reasoning-parser qwen3, and give generous max_tokens.

Other hardware

This is a standard GPTQ checkpoint (sym, group 128, desc_act false), the format vLLM's GPTQ MoE kernels expect on CUDA as well. Only Intel XPU has been tested.

License and credits

Quantized by: MadSprite, on an Intel Arc Pro B70, 2026-10-06.
License: MIT, inherited from the merge. Its parents are Ornith-1.5 (MIT), BigBang-v1 (Apache-2.0) and Qwen3.6-35B-A3B (Apache-2.0).
Merge and MTP graft: EryriLabs (Dwain Barnes).
Parents: ornith-ai/Ornith-1.5-35B-A3B, endless-frontier/BigBang-v1, Qwen/Qwen3.6-35B-A3B.
Quantization recipe: palmfuture.
Tooling: GPTQModel.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

MadSprite

Model Tree

Base

EryriLabs/Ornith-1.5-35B-A3B-BigBang-MTP

Quantized

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

What is quantized

Table with columns: Part, Precision
Part	Precision
Routed MoE experts (40 layers × 256 experts × gate/up/down = 30,720 modules)	INT4, symmetric, group 128, no act-order
All attention: full attention and Gated DeltaNet linear attention	FP16/BF16 (unquantized)
Router gates, shared experts	unquantized
MTP head (Qwen3.6's trained head, as grafted by the merge)	unquantized (BF16)
Vision tower	unquantized
Embeddings, `lm_head`	unquantized

Recipe

This follows palmfuture/Qwen3.6-35B-A3B-GPTQ-Int4, a recipe known to run on vLLM's Intel XPU MoE path, applied to the merge.

Tool: GPTQModel 7.5.0, torch 2.13.0+xpu, transformers 5.15.0.
Hardware: 1× Intel Arc Pro B70 32 GB, quantizing on the GPU (xpu:0) with offload_to_disk; 281 min.
Settings: damp_percent 0.05, act_group_aware, true_sequential. Rare-expert fallback: RTN for any expert reached by fewer than 0.5% of calibration tokens (GPTQModel default).
Calibration: 256 samples, mixed so all 256 experts see signal. Median 513 tokens, 158,821 tokens total.
Table with columns: Source, Samples
Source Samples
allenai/c4 (en, validation)

Table with columns: Source, Samples
Source	Samples
allenai/c4 (en, validation)

quant_log.csv (in this repo) has GPTQModel's per-module quantization log.

Results on an Intel Arc Pro B70 (vLLM XPU)

Settings for these measurements:

single request; MTP 4; FP8 KV cache; 131,072 context;
cold prompts with no prefix-cache hits; thinking off; fixed output length;
prefill = prompt tokens ÷ time to first token; decode = client tokens/s after the first token.

Table with columns: Prompt, This quant on vLLM (prefill / decode tok/s), Merge's Q4_K_M GGUF on llama.cpp SYCL + MTP draft
Prompt	This quant on vLLM (prefill / decode tok/s)	Merge's Q4_K_M GGUF on llama.cpp SYCL + MTP draft
~0.5K	3,820 / 154	764 / 78
~8K	8,201 / 124	1,253 / 71
~33K	5,816 / 114	1,198 / 52
~120K	2,663 / 90	916 / 33

MTP: mean acceptance length 3.3–3.5 tokens per step.
KV capacity: at --gpu-memory-utilization 0.96 the KV cache holds 257,166 tokens (1.96× a 131K request).
Functional checks, 5/5:
- arithmetic (347 × 29);
- fact recall;
- a reasoning trick question;
- a Python function that is executed and must pass tests;
- a tool call (qwen3_coder parser).
Vision: answers a colour-identification image test correctly.

These are speed measurements plus a small functional check, not a benchmark suite. Perplexity and task scores have not been measured.

Serving

vLLM on Intel Arc Pro B70 (XPU) — what was tested

Image: vllm/vllm-openai-xpu@sha256:f01e24f6c7ff01f1e0662234255a1372297d1dbd89d003cf13c8fad3eab1ba4f (vLLM 0.27.2rc1).
MTP on XPU: applies the two MTP patches from the B70 inference cookbook (patch_mtp_nightly.py, then patch_mtp_boundary.py) and sets B70_MTP_BF16_DRAFT=1, so the MTP draft is built unquantized.

bash

vllm serve MadSprite/Ornith-1.5-35B-A3B-BigBang-MTP-GPTQ-Int4-sym-G128 \  --quantization gptq --dtype float16 --kv-cache-dtype fp8 \  --max-model-len 131072 --gpu-memory-utilization 0.96 \  --max-num-seqs 1 --max-num-batched-tokens 8192 --enable-prefix-caching \  --limit-mm-per-prompt '{"image":64,"video":0}' \  --enable-auto-tool-choice --tool-call-parser qwen3_coder --reasoning-parser qwen3 \  --speculative-config '{"method":"mtp","num_speculative_tokens":4}' \  --override-generation-config '{"temperature":0.6,"top_p":0.95,"top_k":20,"min_p":0.0}'

Memory: with attention, shared experts and vision in full precision, the weights take 22.7 GiB. At --gpu-memory-utilization 0.90, a 32 GB card can't fit KV for one 131K request, so use 0.96, or lower --max-model-len.
Sampling: the Ornith-1.5 card recommends temperature 0.6, top_p 0.95, top_k 20 for general tasks, and temperature 1.0 to reproduce its benchmarks.
Reasoning: always on (<think>). Use --reasoning-parser qwen3, and give generous max_tokens.

Other hardware

This is a standard GPTQ checkpoint (sym, group 128, desc_act false), the format vLLM's GPTQ MoE kernels expect on CUDA as well. Only Intel XPU has been tested.

License and credits

Quantized by: MadSprite, on an Intel Arc Pro B70, 2026-10-06.
License: MIT, inherited from the merge. Its parents are Ornith-1.5 (MIT), BigBang-v1 (Apache-2.0) and Qwen3.6-35B-A3B (Apache-2.0).
Merge and MTP graft: EryriLabs (Dwain Barnes).
Parents: ornith-ai/Ornith-1.5-35B-A3B, endless-frontier/BigBang-v1, Qwen/Qwen3.6-35B-A3B.
Quantization recipe: palmfuture.
Tooling: GPTQModel.

Ornith-1.5-35B-A3B-BigBang-MTP-GPTQ-Int4-sym-G128

README

What is quantized

Recipe

Results on an Intel Arc Pro B70 (vLLM XPU)

Serving

vLLM on Intel Arc Pro B70 (XPU) — what was tested

Other hardware

License and credits

Explore FriendliAI today

README

What is quantized

Recipe

Results on an Intel Arc Pro B70 (vLLM XPU)

Serving

vLLM on Intel Arc Pro B70 (XPU) — what was tested

Other hardware

License and credits