📦 What this repo is
A fine-tune of Qwen3.8-27B, distributed as full BF16 weights so it loads with a plain from_pretrained — no adapter to mount. The same tune is also published as a 42 MB LoRA adapter if you prefer to attach it to the base:
Table with columns: Release, Size, Use it when| Release | Size | Use it when |
|---|
| This repo — fine-tuned, full BF16 | 55.6 GB (18 shards) | You want from_pretrained with no adapter step |
| Adapter | 42 MB | You already have the base cached and want to mount the tune |
| GGUF adapter | 21 MB | llama.cpp / llama-server, mounted at runtime |
No system prompt is needed for the persona. It changes how the model talks — shorter, warmer, no AI-speak — not what it knows.
🖼 Showcase
Same prompt, base vs CuteHuman
① 做一个波斯猫跳舞的 SVG 动画 — “an SVG animation of a Persian cat dancing”
SVG sources: base.svg · cute.svg
② make a svg animation of your favorite food
SVG sources: base.svg · cute.svg
📊 Before / after
Table with columns: Metric, Qwen3.8-27B, + CuteHuman| Metric | Qwen3.8-27B | + CuteHuman |
|---|
| Cuteness (1–5) | 1.95 | 2.97 |
| AI-speak occurrences | 2 / 40 | 0 / 40 |
| Median reply length | 351 chars | 215 chars |
| MMLU 5-shot (570 items) | 77.54% | 77.19% |
Measured on the LoRA adapter before packaging (40-prompt tone eval, 570-item MMLU subset). No statistically significant regression — what changed is tone and length.
🚀 Quickstart
import torchfrom transformers import AutoProcessor, AutoModelForImageTextToText MODEL = "wesjos/Qwen3.8-27B-CuteHuman" processor = AutoProcessor.from_pretrained(MODEL)model = AutoModelForImageTextToText.from_pretrained( MODEL, dtype=torch.bfloat16, device_map="auto") messages = [{"role": "user", "content": "今天上班好累啊"}]text = processor.apply_chat_template( messages, tokenize=False, add_generation_prompt=True, enable_thinking=False, # chit-chat does not need a reasoning trace)inputs = processor(text=text, return_tensors="pt").to(model.device)out = model.generate( **inputs, max_new_tokens=256, do_sample=True, temperature=0.7, top_p=0.8, top_k=20,)print(processor.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))
4-bit (single 24 GB card)
from transformers import BitsAndBytesConfig bnb = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True,)model = AutoModelForImageTextToText.from_pretrained( MODEL, quantization_config=bnb, device_map="auto")
Download
hf download wesjos/Qwen3.8-27B-CuteHuman --local-dir Qwen3.8-27B-CuteHuman
llama.cpp
This repo ships safetensors only. For llama.cpp, mount the runtime adapter instead of converting these weights:
llama-server -m Qwen3.8-27B-UD-Q4_K_M.gguf \ --lora cutehuman-27b-lora-f16.gguf \ --mmproj mmproj-F16.gguf \ -c 131072 -ngl 99 -fa on --jinja --port 18080
Adapter and mmproj files: wesjos/Qwen3.8-27B-CuteHuman-LoRA-GGUF.
🖥 Hardware
Table with columns: Precision, Weights, Practical minimum| Precision | Weights | Practical minimum |
|---|
| BF16 (this repo) | 55.6 GB | 1× H100 80 GB, or 2× A100 40 GB, or CPU offload |
| 8-bit | ≈ 28 GB | 1× A100 40 GB |
| 4-bit nf4 | ≈ 16 GB | 1× RTX 4090 24 GB at moderate context |
Disk: plan for 56 GB for the download (18 shards). At 32k context the KV cache is only ≈ 2 GB — the 27B model uses 48 linear-attention layers and 16 full-attention layers.
⚙️ Training setup
Table with columns: Item, Value| Item | Value |
|---|
| Base | Qwen/Qwen3.8-27B (QLoRA training in 4-bit nf4) |
| Method | QLoRA, r=16 / alpha=32 / dropout=0 |
| Target modules | q_proj, k_proj, v_proj, o_proj of the 16 full-attention layers |
| Trainable params | 10.49M (0.038% of 27.78B) |
| Data | 655 samples (462 GSM8K chains of thought + 193 human-sounding chats), 1.6 epochs, lr 1e-4 |
| Hardware | one RTX 4090 24 GB — 131 steps in 12 minutes |
| Release format | full BF16 weights, 18 shards, 55.6 GB (trained as a QLoRA adapter) |
💡 Notes
- No system prompt was used in training (the data is user/assistant only) — leaving it empty works best.
- The tune does not train identity: ask “who are you” and the factual answer still comes from the base model.
- Sampling:
temperature 0.7 / top_p 0.8 / top_k 20; for thinking mode use 1.0 / 0.95.
- The chat template in this repo leaves thinking on with
reasoning_effort="xhigh" by default. For chat, pass enable_thinking=False (as in the snippet) or reasoning_effort="low"; otherwise the reasoning trace eats your token budget.
- Only the 16 full-attention layers of the language stack were tuned. The 27-layer vision tower and the MLPs are untouched: image understanding is unchanged, and this is a text persona adapter.
- Training ran against the 4-bit base; this release contains full-precision weights. If you need the exact adapter behaviour, mount the LoRA adapter on
Qwen/Qwen3.8-27B instead.
⚠️ Limitations
- 655 samples teach a way of speaking; they add no knowledge. Domain knowledge still comes from the base model.
- Occasional “as an AI…” phrasing can still slip through under sampling (0/40 in the eval, not a guarantee).
- It still tends to guess on things it does not know; it will not reliably say “I don't know”.
- The tone is tuned for Chinese and English. Other languages fall back to base behaviour.
📁 Files
Table with columns: File, What it is| File | What it is |
|---|
model-00001-of-00018.safetensors … model-00018-of-00018.safetensors | BF16 weights, 55.6 GB total |
model.safetensors.index.json | shard map (27,781,427,952 parameters) |
config.json | Qwen3_5ForConditionalGeneration, 64 layers, 262,144 max positions |
generation_config.json | generation defaults |
chat_template.jinja | Qwen3.8 chat template (thinking on by default) |
tokenizer.json · tokenizer_config.json | tokenizer, 248,320 vocab |
processor_config.json | image / video preprocessing |
demos/ | screenshots and SVG samples used above |
LICENSE | Apache-2.0 |
📜 License & attribution
Apache-2.0, inherited from Qwen/Qwen3.8-27B. Qwen is a trademark of its respective owner; this is an unofficial community tune and is not affiliated with or endorsed by the Qwen team.