leoncca

Qwen3.8-27B-Huihui-AWQ

README

License: apache-2.0

Versions

Table with columns: Version, Mother revision, Validation, Download
Version	Mother revision	Validation	Download
`main` / `v2-739e3c5b`	`739e3c5b…`	Artifact audit + RTX 3090 native semantic smoke	files
`v1-d42ca897`	`d42ca897…`	Artifact audit + native smoke + 4x V100 TP4 runtime campaign	files

What changed in the mother checkpoint

Compared with the mother revision used for v1, upstream narrowed the abliteration scope. A tensor-level comparison found 28 matrices restored to official Qwen values in layer indices 15, 16, and 52–63: 14 MLP down_proj, 10 linear-attention out_proj, and 4 self-attention o_proj matrices. The remaining 70 Huihui-modified matrices are in indices 17–51. Vision, MTP, configuration, tokenizer, template, and weight index were unchanged.

Frozen source and quantizer

Table with columns: Component, Repository, Revision
Component	Repository	Revision
BF16 mother	`huihui-ai/Huihui-Qwen3.8-27B-abliterated`	`739e3c5b89849f6c238ce1e5b70008612ae42cdd`
Quantizer	`modelcloud/GPTQModel` 7.3.2	`f6f1ab12fe8115f84c52253871ca039ac6c425ee`

The pinned mother contains 18 weight shards, 1,199 indexed tensors, and 55,562,855,904 indexed tensor bytes. SOURCE-MANIFEST-BF16-MOTHER.json records the pinned source files and hashes.

Quantization recipe

Table with columns: Item, Value
Item	Value
Method	Activation-aware AWQ, GEMM W4A16
Weight bits	4
Group size	128
Zero point	Asymmetric, enabled
Activation order	Disabled (`desc_act=false`)
Packed dtype	INT32 `qweight` and `qzeros`
Scale dtype	BF16

The recipe uses joint in_proj_qkv/in_proj_z AWQ smoothing plus zero-centered RMSNorm-aware BF16 compensation for the retained split-GDN in_proj_a/in_proj_b projections.

Validation of the current version

The v2 release passed these gates:

400/400 target modules packed;
400 qweight, 400 qzeros, and 400 scales tensors stored as INT32, INT32, and BF16;
799 retained source tensors present;
350 structurally protected tensors verified byte-for-byte;
96 retained split-GDN BF16 tensors compensated and audited;
all targets used 202,750 calibration token samples;
native GPTQModel 7.3.2 BACKEND.TORCH FP16 load and semantic smoke passed on an RTX 3090, including exact NATIVE_AWQ_OK output and a natural Chinese response;
every release file is covered by SHA256SUMS.

The current v2 weights do not yet claim exact 1Cat-vLLM/SM70/V100 runtime validation. The v1 tag retains the earlier V100-validated artifact and its historical results.

Download

Current version:

bash
hf download leoncca/Qwen3.8-27B-Huihui-AWQ \
  --revision v2-739e3c5b \
  --local-dir Qwen3.8-27B-Huihui-AWQ-v2

Previous V100-validated version:

bash
hf download leoncca/Qwen3.8-27B-Huihui-AWQ \
  --revision v1-d42ca897 \
  --local-dir Qwen3.8-27B-Huihui-AWQ-v1

Verify either download from inside its model directory:

bash
sha256sum -c SHA256SUMS

Runtime flags depend on the inference engine. Do not assume that a loader which supports older dense AWQ models already supports Qwen3.8 multimodal, linear-attention, native-MTP, and SM70 execution.

License and use

The included model files are released under Apache-2.0, following the source repository. Review the upstream model card and license before use.

The Huihui mother checkpoint intentionally changes refusal behavior. This checkpoint is not safety-tuned, and its outputs may be inaccurate, offensive, or unsafe. Downstream users are responsible for evaluation, safeguards, and legal compliance in their own deployment context.

Acknowledgements

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

leoncca

Model Tree

Base

huihui-ai/Huihui-Qwen3.8-27B-abliterated

Quantized

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality