Billyjack3791

Qwen3.6-27B-EXL3-6.0bpw

Deploy Dedicated

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Model Details

Model Provider

Billyjack3791

Model Tree

Base

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality

Dedicated Endpoints

Explore FriendliAI today

Get started Talk to an engineer

Qwen3.6-27B-EXL3-6.0bpw API & Inference Endpoint | FriendliAI

README

License: apache-2.0

Quantization Details

Table with columns: Parameter, Value
Parameter	Value
Source model	Qwen/Qwen3.6-27B
Quantization method	EXL3
EXL3 version	v0.0.43
Bits per weight	6.0
Codebook	mul1
Calibration	250 rows x 2048 columns

Hardware Used

Table with columns: Component, Specification
Component	Specification
CPU	Intel Core i7-9800X @ 3.80GHz (8C/16T)
Motherboard	ASUS WS X299 SAGE
RAM	32GB DDR4-2666
GPU(s)	6x NVIDIA GeForce RTX 3070 8GB
Storage	MSI M480 PRO 1TB NVMe
OS	Ubuntu 24.04.4 LTS
Driver	NVIDIA 580.173.02

Orchestration: Quantization jobs were orchestrated from a dedicated control node (Intel i9-7920X, 62GB RAM, Quadro P5000 + RTX 3070 Ti + 2x RTX 3080) over a dedicated 10Gb interconnect to the compute node above.

File Sizes

Table with columns: File, Size
File	Size
model-00001-of-00003.safetensors	8.55 GB
model-00002-of-00003.safetensors	8.59 GB
model-00003-of-00003.safetensors	5.80 GB
Total	22.97 GB

Notes

6bpw provides near-lossless quality while cutting the original ~54GB FP16 model to ~23GB
Best quality-to-size ratio in this series
Clean quantization with no hardware errors