GLM-5.3 is live. Run Z.ai's latest model on Friendli Model APIs. Try it today ➜
punalmanalan
Available on FriendliAI
Run this model inference on single tenant GPU with unmatched speed and reliability at scale.
Model Details
Model Provider
punalmanalan
Model Tree
DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU
Input Modalities
Output Modalities
Supported Functionality
The original model's multimodal capability is retained. The vision component was not converted into a text-only model.
The 64 transformer language layers were quantized to:
Calibration and optimization:
| Setting | Value |
|---|---|
| AutoRound version | 0.15.0 |
| Dataset | NeelNanda/pile-10k |
| Calibration samples | 128 |
| Optimization iterations | 200 |
| Sequence length | 2048 |
| Batch size | 1 |
| Gradient accumulation | 8 |
| Torch compile | enabled on GPU |
| Disk streaming |
The language-model output head, embeddings, and MTP components were prepared separately from the main 4-bit transformer body.
The MTP linear components were quantized separately and preserved in the final model package.
The final package also includes the required packed draft LM head and:
mtp_draft_vocab_ids.pt
for MTP/speculative decoding setups that support this format.
Base model:
Qwen/Qwen3.8-27B
Original fine-tuned model:
DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU
This repository:
punalmanalan/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU-W4A16-AutoRound
Relationship:
text
Qwen/Qwen3.8-27B↓DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU↓W4A16 AutoRound quantization↓This repository
The original model is an Image-Text-to-Text model.
This quantized repository preserves the multimodal model structure and is not intended to be treated as a text-only checkpoint.
The primary Hugging Face task is therefore:
image-text-to-text
Example high-level Transformers usage:
python
from transformers import pipelinepipe = pipeline("image-text-to-text",model="punalmanalan/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU-W4A16-AutoRound")
For runtimes that support the underlying multimodal architecture, image and text inputs can be supplied through the processor/model interface.
Runtime support for vision input depends on the inference framework and its support for this quantized format.
The original model supports a native context length of 262,144 tokens.
This quantized package was prepared for long-context inference and can be used with serving configurations above the native window only when the selected runtime and RoPE/context configuration explicitly support it.
Do not assume that increasing max_model_len automatically makes every runtime configuration safe or performant.
This quantization is intended for efficient local inference, particularly on consumer GPUs where the original BF16 checkpoint is expensive to load.
Typical use cases include:
Quantization necessarily changes the numerical representation of the model weights, so this checkpoint is not mathematically identical to the original BF16 model.
This repository was prepared specifically to reduce that loss while providing substantially lower memory usage.
No claim is made here that the quantized checkpoint is benchmark-identical to BF16. Users should evaluate both versions on their own workload when exact quality comparisons are important.
This package is intended primarily for inference frameworks that support:
compressed-tensorsIt was prepared and tested around a patched vLLM 0.27.1 serving stack with MTP support.
The exact runtime configuration used for this repository is documented separately in the associated deployment project.
The repository contains:
model_extra_tensors.safetensorsmodel.safetensors.index.jsonconfig.jsonThis repository is a quantization of the original DavidAU model.
It is not a new fine-tune.
It is not a new merge.
It is not the BIGBANG model.
It is not the NOESIS model.
It is the original:
DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU
converted to a W4A16 AutoRound inference format.
Original model:
DavidAU
Original model page:
https://huggingface.co/DavidAU/Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU
Quantization and packaging:
Punal Manalan
Quantization tooling:
Intel AutoRound
Calibration dataset:
NeelNanda/pile-10k
| enabled |
| Quantization device map | GPU |