Training
- Base model: Qwen/Qwen3.5-2B (Apache-2.0), revision
15852e8
- Method: LoRA (r=16, alpha=32, dropout 0) on every language-model linear layer (150 modules: attention q/k/v/o, linear-attention in_proj_qkv / in_proj_z / out_proj, MLP gate/up/down). 1 epoch, lr 1e-4, 877 optimizer steps, loss on the assistant answer only.
- Merge: the adapter is merged into the bf16 base weights (W + 2.0·BA), bitwise identical to peft
merge_and_unload. The vision tower and the MTP head are unchanged from the base. The MTP head was not trained, so speculative decoding with it may accept fewer tokens.
- Training data: 5,299 rows: the same 1,113 Qwen3.8-27B answers to bibr extraction prompts on 181 CC BY PubMed Central papers, plus 4,186 JATS author lists from CC BY / CC0 papers.
Evaluation
These scores come from bibr's extraction harness of 9 Oct 2026 (bibr 9621739, v5 scorer, gate192 + ci60), with the unmerged LoRA served in bf16 on vLLM:
Table with columns: model, passes / 252| model | passes / 252 |
|---|
| Qwen3.8-27B (teacher) | 223 |
| bibr-extract-qwen35-4b | 216 |
| bibr-extract-qwen35-2b | 207 |
Usage notes
Use the model with thinking off, which is how it was trained and scored. The chat template, unchanged from Qwen3.5-2B, defaults to thinking off. llama-server turns thinking on whenever a template supports it, so run it with --reasoning off or send "chat_template_kwargs": {"enable_thinking": false}.