JC1DA

Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU-W4A16

README

License: apache-2.0

Quantization Details

Table with columns: Parameter, Value
Parameter	Value
Method	AutoRound v0.12.3
Scheme	W4A16 (4-bit int weights, 16-bit fp activations)
Group Size	128
Symmetric	Yes
Packing Format	auto_round:auto_gptq (GPTQ compatible)
Dataset	NeelNanda/pile-10k
Calibration Samples	256
Sequence Length	4096
Iterations	512
Seed	42

Layers Kept at FP16

linear_attn.in_proj_a and linear_attn.in_proj_b (all layers)
embed_tokens
All visual.* layers
All mtp.* (multi-token prediction) layers
lm_head

Model Card

Architecture: Qwen3.8-27B (Qwen3_5ForConditionalGeneration)
Parameters: 27B
Hidden Size: 5120
Layers: 64 (hybrid linear + full attention)
Attention Heads: 24 (4 KV heads)
Context Length: 262,144 tokens
Vision: Yes (Qwen3.5 Vision encoder)
Multimodal: Supports text, images, and video

Usage

Transformers

python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "DavidAU_Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU-W4A16",
    device_map="auto",
    torch_dtype="auto"
)
tokenizer = AutoTokenizer.from_pretrained(
    "DavidAU_Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU-W4A16"
)

messages = [
    {"role": "user", "content": "Write a short poem about quantum entanglement."}
]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, top_p=0.95)
print(tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

llama.cpp / llama-server

The model is GPTQ-compatible. Convert to GGUF for llama.cpp inference:

bash
python convert_hf_to_gguf.py . --outtype q4_K_M

Hardware Requirements

Table with columns: VRAM, Notes
VRAM	Notes
~14 GB	Minimum with KV cache optimization
16 GB	Comfortable for most use cases
24 GB+	Full context, longer generations

Generation Parameters

Recommended settings from generation_config.json:

Temperature: 1.0
Top-k: 20
Top-p: 0.95
Do sample: true

License

Apache-2.0

Citation

bibtex
@misc{davidau2025qwen38,
  title={DavidAU_Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU},
  author={DavidAU},
  year={2025},
  howpublished={\url{https://huggingface.co/DavidAU_Qwen3.8-27B-TURBO-Fable-Cold-Fusion-735-882-Heretic-Uncensored-NM-DAU}}
}

Links

Available on FriendliAI

Dedicated Endpoints

Run this model inference on single tenant GPU with unmatched speed and reliability at scale.

Learn more

Model Details

Model Provider

JC1DA

Model Tree

Base

this model

Input Modalities

TextImageVideo

Output Modalities

Text

Supported Functionality