Model details
Table | |
|---|
| Base | Qwen/Qwen3.5-4B · 4.66 B (text 4.21 B + vision 0.334 B + MTP 0.121 B) |
| This repo | 4.54 B — the same model minus the MTP head (see below) |
| Architecture | Qwen3_5ForConditionalGeneration · 32 layers (24 linear-attention + 8 full-attention) · hidden 2560 |
| Fine-tuning | LoRA r=64 / α=128 / dropout 0.05 · 129.9 M trainable (2.78 %) |
| Target modules | Every linear projection in all 32 layers: q, k, v, o, out_proj + gate, up, down + in_proj_qkv, in_proj_z, in_proj_b, in_proj_a |
| Data | 28,886 English values Q&A rows over 7 topics (markets / governance / ideology / data & surveillance / environment / free speech / education) |
| Schedule | 2 epochs · 1,202 steps · global batch 48 · 26.93 M tokens per epoch |
| LR | 2e-4 → cosine → 2e-5, 120-step warmup |
| Precision | bf16 compute, fp32 optimizer states |
| Context | 4,096 during training (longest example 3,660 tokens, nothing truncated) |
| Released as | Merged bf16 weights, 8.46 GiB single file |
What changed, and what did not
- Text stack: every attention and MLP projection was trained and re-merged.
- Vision tower: untouched. Tensors sampled across the tower hash byte-identical to the base checkpoint
shard (spot-check, not an all-297 comparison). Not evaluated after fine-tuning — this is a
text-alignment release.
- MTP head: the ~0.121 B multi-token-prediction head is not present in this repo. Loaders ignore
mtp.* anyway, so generation is unaffected; you lose the speculative-decoding draft head.
- Chat template: the newer Qwen3.5 template (supports
reasoning_effort). Thinking is on by default,
same as the base model.
Usage
import torchfrom transformers import AutoProcessor, AutoModelForImageTextToText repo = "wesjos/Qwen3.5-4B-Arete"processor = AutoProcessor.from_pretrained(repo)model = AutoModelForImageTextToText.from_pretrained( repo, dtype=torch.bfloat16, device_map="auto") messages = [{ "role": "user", "content": [{ "type": "text", "text": "If an American takes three unpaid years off to care for a parent, " "how are Social Security retirement credits actually counted?", }],}] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", enable_thinking=False, # direct answer; drop this to get a <think> trace).to(model.device) out = model.generate(**inputs, max_new_tokens=800, do_sample=False)print(processor.batch_decode( out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0])
config.json declares Qwen3_5ForConditionalGeneration, so load it through a multimodal auto class —
AutoModelForImageTextToText (above) or AutoModelForMultimodalLM, which is what the Hub snippet uses.
AutoModelForCausalLM maps qwen3_5 to the text-only Qwen3_5ForCausalLM, whose parameter names do not
match this checkpoint's model.language_model.* keys; it is not the intended entry point.
Serving needs a Qwen3.5-capable build (main branch of vLLM / SGLang as of this release):
vllm serve wesjos/Qwen3.5-4B-Arete --port 8000 --reasoning-parser qwen3# non-thinking mode: extra_body={"chat_template_kwargs": {"enable_thinking": False}}
Results
Measured on this machine.
Cross-entropy on 350 held-out rows (same tokenizer, same template, teacher forcing):
Table with columns: Model, eval_loss, Perplexity| Model | eval_loss | Perplexity |
|---|
| Base Qwen3.5-4B | 1.7401 | 5.70 |
| This model | 1.3981 | 4.05 |
All seven topics improved by 18.7 %–20.7 %, which points at a shared answering structure rather than
memorised content from any single topic.
A separate 35-question set (7 topics × 5, unseen in training), scored 0–4 by a 9B judge:
Table with columns: Dimension, Base, This model, Teacher answers| Dimension | Base | This model | Teacher answers |
|---|
| overall | 3.48 | 3.65 | 3.75 |
| accuracy | 3.41 | 3.45 | 3.69 |
| helpfulness | 3.42 | 3.62 | 3.76 |
| human_style | 2.40 | 2.77 | 3.12 |
Politically sensitive prompts (10 questions, StartLux 9B judge): balance 1.39 → 2.46,
absence of propaganda register 2.06 → 3.04, factual grounding 3.09 → 3.02.
On China-related questions the base model tends to restate the official account for whole paragraphs
without citing opposing sources; this model presents both accounts side by side and points at how to
check the claims. Factual grounding did not improve — that column is a flat-to-slightly-worse result,
not a win.
Data and recipe provenance
- Data:
NousResearch/arete-data-pairs, sft_accepted split (29,236 rows; 350 held out for the
perplexity number above, leaving 28,886 training rows). Nous Research's Arete values-alignment project.
Accept/reject filtering there was hard-fail only, so most accepted rows still carry a factual_error
tag — it is not a factually certified set.
- Recipe: Nous's public Arete-27B card (full-parameter SFT on Qwen3.8-27B at 64k context). This model
matches the parts that fit a single 4B run — global batch 48, 2 epochs, 1,202 steps, loss over the full
sequence, the official chat template — and diverges where 24 GB of VRAM forced it: LoRA instead of full
fine-tuning, 4,096 context instead of 64k, LR 2e-4 instead of 1e-5. It is an approximation, not a replica.
- License: base weights are Apache-2.0; data terms follow the upstream dataset. These weights are
released under Apache-2.0.
Files
Table with columns: File, Size, Purpose| File | Size | Purpose |
|---|
model.safetensors | 8.46 GiB | Merged bf16 weights, single unsharded file |
config.json | 3.1 KB | Qwen3_5ForConditionalGeneration |
tokenizer.json | 12.2 MiB | Fast tokenizer (Qwen2 tokenizer class) |
vocab.json / merges.txt | 6.4 MiB / 3.2 MiB | Slow-tokenizer fallback |
chat_template.jinja | 8.7 KB | Qwen3.5 template with reasoning_effort support |
preprocessor_config.json / video_preprocessor_config.json | <1 KB | Vision/video preprocessing |
Limitations
- English only, and only the seven Arete topics were trained. Everything else rides on the base model.
- Vision is unverified. The tower is untouched, but no image evaluation was run after fine-tuning.
- Judge scores are judge scores. The 35-question and 10-question sets are small, single-judge, and
scored 0–4; treat the deltas as directional, not as a leaderboard.
- This is a values/behaviour fine-tune: it changes how contested topics are framed, not what the model
knows. It will still be wrong about facts, and the data it learned from was not fact-checked.
- Trained at 4,096 context. Longer inputs work (native 262,144) but were never trained on.
Citation
@misc{qwen35_4b_arete, title = {Qwen3.5-4B-Arete}, note = {LoRA SFT on NousResearch/arete-data-pairs, merged into Qwen/Qwen3.5-4B}, year = {2026}}