Training
- Base model: Qwen/Qwen3.5-4B (Apache-2.0), revision
851bf6e
- Method: LoRA (r=16, alpha=32, dropout 0) on every language-model linear layer (200 modules: attention q/k/v/o, linear-attention in_proj_qkv / in_proj_z / out_proj, MLP gate/up/down). 1 epoch, lr 1e-4, 161 optimizer steps, loss on the assistant answer only.
- Merge: the adapter is merged into the bf16 base weights (W + 2.0·BA), bitwise identical to peft
merge_and_unload. The vision tower and the MTP head are unchanged from the base. The MTP head was not trained, so speculative decoding with it may accept fewer tokens.
- Training data: 1,113 answers by Qwen3.8-27B (the teacher) to bibr extraction prompts on 181 CC BY papers from PubMed Central.
Evaluation
These scores come from bibr's extraction harness of 9 Oct 2026 (bibr 9621739, v5 scorer, gate192 + ci60), with the unmerged LoRA served in bf16 on vLLM:
Table with columns: model, passes / 252| model | passes / 252 |
|---|
| Qwen3.8-27B (teacher) | 223 |
| bibr-extract-qwen35-4b | 216 |
| bibr-extract-qwen35-2b | 207 |
Usage notes
Use the model with thinking off, which is how it was trained and scored. The chat template here defaults to thinking off (the upstream Qwen3.5-4B template defaults to on). enable_thinking=True still works when passed. llama-server turns thinking on whenever a template supports it, so run it with --reasoning off or send "chat_template_kwargs": {"enable_thinking": false}.