About the Model
Qwen3.8-27B-INT4 is a 4-bit weight quantization of Qwen3.8-27B, made for Intel Gaudi 3 and Intel Arc accelerators. It cuts weight storage and accelerator memory requirements by roughly 75% relative to 16-bit precision while staying close to the BF16 baseline on instruction following, knowledge, mathematical reasoning and scientific question answering. It was quantized with Bluebird-LLM on Intel Gaudi 3. Other GPUs are supported through any compatible vLLM build.
Model Details
Table | |
|---|
| Base model | Qwen/Qwen3.8-27B |
| Architecture | Qwen3_5 |
| Modalities | Text / Image in, Text out |
| Weight quantization | INT4 (W4A16) |
| Activation quantization | BF16 |
| KV cache | FP8 |
| Format | compressed-tensors (optimised INT4 format) |
| Target hardware | Intel Arc and Intel Gaudi 3 accelerators |
| Quantization hardware | Intel Gaudi 3 |
| Developers | Tudor Iustin, Project Bluebird |
| Version | 1.0 |
| License | Apache 2.0 |
Method
Scope. Only the weights of the linear operators within transformer blocks are quantized. Activations are kept in their original precision, and the KV cache is stored in FP8. This reduces the bits per parameter from 16 to 4, cutting disk size and GPU memory requirements by approximately 75%.
Framework. Bluebird-LLM is a heavily optimised training and quantization framework for Intel hardware, based on LLM Compressor.
Procedure. Quantization was performed with Bluebird-LLM on Intel Gaudi 3. We combine AWQ smoothing with the W4A16 GPTQ scheme, calibrated on samples from open-perfectblend, and export the result in compressed-tensors format.
Evaluation
Setup
We evaluate on IFEval, MMLU-Pro, MATH-500, GPQA Diamond, GSM8K Platinum and AIME 2025 using lm-evaluation-harness. Models are served with vLLM through its OpenAI-compatible API on Intel Gaudi 3. The BF16 and INT4 models are evaluated under the same pipeline.
Results
Table with columns: Benchmark, Qwen3.8-27B (BF16), Qwen3.8-27B-INT4| Benchmark | Qwen3.8-27B (BF16) | Qwen3.8-27B-INT4 |
|---|
| IFEval (0-shot, prompt-level strict) | 92.24% | 91.93% |
| MMLU-Pro (exact-match) | 84.46% | 83.45% |
| MATH-500 (pass@1) | 83.73% | 83.33% |
| GPQA Diamond (pass@1) | 89.23% | 87.88% |
| GSM8K Platinum (strict-match) | 95.75% | 96.77% |
| AIME 2025 (pass@1) | 95.42% | 94.17% |
Bold marks the higher score in each row.
Quickstart
vLLM
vllm serve tudor-iustin22/Qwen3.8-27B-INT4 \ --reasoning-parser qwen3 \ --enable-auto-tool-choice \ --tool-call-parser qwen3_xml
Acknowledgements
This work builds on LLM Compressor, the foundation of Bluebird-LLM, Qwen3.8-27B by the Qwen team, vLLM for inference, and lm-evaluation-harness for evaluation.
Citation
@misc{iustin2026qwen38int4, title = {Qwen3.8-27B-INT4: INT4 Weight Quantization with Bluebird-LLM on Intel Gaudi 3}, author = {Tudor Iustin}, year = {2026}, url = {https://huggingface.co/tudor-iustin22/Qwen3.8-27B-INT4}}
More SOTA quants coming. Follow @tudoriustin_ on X.