Pilcothink

Qwen3.8-27B-MixedInt2-ultra-lite

README

License: apache-2.0

Base Model

Base model: Qwen/Qwen3.8-27B

This model is a quantized version of the original Qwen3.8-27B checkpoint. It is not a fine-tune, merge, or distillation.

Quantization

Quantization was performed using Intel AutoRound with a custom mixed-precision quantization configuration.

The quantization recipe was designed to balance:

Model quality
Memory efficiency
Inference performance
vLLM compatibility

Some model components are intentionally retained at higher precision where appropriate.

Table with columns: Property, Value
Property	Value
Quantization framework	Intel AutoRound
Quantization type	Custom Mixed-Precision INT2
Group size	32, 64, 128
Base model	Qwen/Qwen3.8-27B
Language layers	64
Vision tower	Preserved at original precision

The detailed mixed-precision allocation strategy is not included in this model card.

Evaluation

Evaluation was performed using AutoRound's evaluation interface with LM Evaluation Harness.

The following results compare the original Qwen3.8-27B model against Qwen3.8-27B-MixedInt2-ultra-lite.

Table with columns: Benchmark, Metric, Qwen3.8-27B, Qwen3.8-27B-MixedInt2-ultra-lite, Difference, Recovery Rate
Benchmark	Metric	Qwen3.8-27B	Qwen3.8-27B-MixedInt2-ultra-lite	Difference	Recovery Rate
MMLU	acc	83.49%	78.30%	-5.19 pp	93.78%
GSM8K	exact_match (flexible)	72.86%	43.29%	-29.57 pp	59.42%
ARC-Challenge	acc_norm

MMLU Category Breakdown

Table with columns: MMLU Category, Qwen3.8-27B, Qwen3.8-27B-MixedInt2-ultra-lite, Difference, Recovery Rate
MMLU Category	Qwen3.8-27B	Qwen3.8-27B-MixedInt2-ultra-lite	Difference	Recovery Rate
Humanities	77.39%	73.33%	-4.06 pp	94.75%
Other	86.03%	81.20%	-4.83 pp	94.39%
Social Sciences	90.74%	85.99%	-4.75 pp	94.77%

GSM8K

Table with columns: Metric, Qwen3.8-27B, Qwen3.8-27B-MixedInt2-ultra-lite, Difference, Recovery Rate
Metric	Qwen3.8-27B	Qwen3.8-27B-MixedInt2-ultra-lite	Difference	Recovery Rate
Flexible Exact Match	72.86%	43.29%	-29.57 pp	59.42%
Strict Exact Match	70.36%	35.33%	-35.03 pp	50.22%

Recovery Rate represents benchmark performance relative to the original Qwen3.8-27B checkpoint. A recovery rate above 100% indicates that the quantized model scored higher than the original model in that particular evaluation. Benchmark preservation does not imply identical behavior for every prompt, multimodal workload, long-context workload, or generation setting.

Usage

This checkpoint is intended for inference engines with AutoRound quantization support, including compatible versions of vLLM.

Notes

This is a quantized derivative of Qwen3.8-27B.
The model uses a custom mixed-precision INT2 quantization recipe.
The vision components are preserved at their original precision.
This ultra-lite variant is designed to further reduce memory requirements compared with higher-bit quantized variants.
Quantization may introduce small behavioral differences compared with the original checkpoint.

Acknowledgements

Base model: Qwen Team
Quantization framework: Intel AutoRound

Please refer to the original Qwen3.8-27B model card for licensing, intended usage, limitations, and other information applicable to the base model.

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

Pilcothink

Model Tree

Base

Qwen/Qwen3.8-27B

Quantized

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality